MIDTERM EVALUATION REPORT September 2018 USAID’s Development Food Security Activity (DFSA): “Nobo Jatra/New Beginning” DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect those of World Vision Bangladesh and the United States Agency for International Development or the United States Government NOBO JATRA PROJECT/“NEW BEGINNING”: MID-TERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 i Table of Contents Executive Summary ..............................................................................................................................1 1. Introduction ............................................................................................................................... 13 2. Project Overview ........................................................................................................................ 14 2.1. Objectives and expected results .......................................................................................... 14 2.2. Performance indicators ....................................................................................................... 14 2.3. The NJP Theory of Change ................................................................................................... 15 2.4. NJP implementing partners and management structure ....................................................... 15 3. Objectives of the evaluation ....................................................................................................... 16 4. Methodology .............................................................................................................................. 17 Phase 1: Preparation ...................................................................................................................... 17 Phase 2: Data collection and synthesis ............................................................................................ 17 Phase 3: Analysis and reporting ...................................................................................................... 20 Challenges, limits and solutions applied .......................................................................................... 20 Quality assurance ........................................................................................................................... 20 5. Evaluation findings and discussions ............................................................................................. 21 Objective 1: Quality and effectiveness ............................................................................................ 21 Objective 2: Evidence of Change ..................................................................................................... 58 Objective 3: Collaborative Learning and Action ............................................................................... 60 Objective 4: Sustaining Project Impact ............................................................................................ 61 Objective 5: Cross-Cutting Themes .................................................................................................. 64 Objective 6: Project Framework ...................................................................................................... 66 6. Recommendations ...................................................................................................................... 69 Annex A. NJP theory of change ....................................................................................................... 78 Annex B. Evaluation Matrix ............................................................................................................ 84 Annex C. Data collection tools ........................................................................................................ 94 Annex D. List of documents received from WV .............................................................................. 155 Annex E. Terms of reference ……………………………………………………………………………………………………….. 155 Annex F. Ratanpur Union budget activities ..…………………………………………………………………………………178 NOBO JATRA PROJECT/“NEW BEGINNING”: MID-TERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 ii List of Figures Figure 1. NJP working areas ………………………………………………………………………………………………….. 13 Figure 2. NJP management structure ……………………………………………………………………………………. 15 Figure 3. BRAC graduation model …………………………………………………………………………………………. 55 List of Tables Table 1. Summary number of NJP indicators ………………………………………………………………………… 14 Table 2. Summary of Respondents ………………………………………………………………………………………. 18 NOBO JATRA PROJECT/“NEW BEGINNING”: MID-TERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 iii ACKNOWLEDGEMENTS Development Technicians would like to acknowledge that this Mid-Term-Evaluation (MTE) was made possible through the active support of World Vision (WV), and its partners – Winrock and the World Food Program (WFP). Special thanks are extended to Rafique Islam (NJP M&E Manager) for his continuous support in providing documents and information to the MTE team throughout the evaluation, and for his active support during the data collection period in the field. Appreciation is also extended to all the Field Office Coordinators for hosting the MTE team, and the field staff that directed the evaluators to their sampled village sites, and introduced the team to government key informants. Development Technicians also thanks the WV management – Rakesh Katal (NJP COP), and Henrik Sturve and Kristi Pearson, from WVI headquarters, for providing support on administrative issues that were proficiently handled such that the MTE team could stay on schedule. The support from USAID/FFP teams, from both Washington and Bangladesh was appreciated for their active engagement for part of the field data collection, and their invaluable comments and insights of their field observations, and of learnings they had acquired from similar MTEs conducted elsewhere in the world. Finally, the MTE team would like to acknowledge the wonderful determination of the NJP beneficiaries who aspire to provide a resilient life, free of poverty, for their children and families. With appreciation of the MTE team - Julia Ahmed (Gender & MCHN Specialist) - Samia Ahmed (Governance & Social Accountability Specialist) - Shameem Dawla (Logistics and Translation Specialist) - Alex Diouf (Promotional Graduation and TOC Specialist) - Zakir Hossain (WASH Specialist) - Faheem Khan (Promotional Graduation, Agriculture and AIGA Specialist) - Abdul Wadud (M&E Specialist) NOBO JATRA PROJECT/“NEW BEGINNING”: MID-TERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 iv ACRONYMS AIGA Alternative Income Generating Activity AIRN Agro-Input Retailers Network AIRP Arsenic Iron Removal Plant ANC Ante-Natal Care BCC Behavior Change and Communication BEO Bureau Environmental Officer BSCIC Bangladesh Small and Cottage Industries Corporation CCT Conditional Cash Transfer CDMAP Comprehensive Disaster Management Action Plan CHCP Community Health Care Project CNF Community Nutrition Facilitator COEL Carbon Monoxide Exposure Limiter COP Chief of Party CPC Child Protection Committee CPP Cyclone Preparedness Program CRA Community Risk Assessment CREL Climate Resilient Ecosystem and Livelihoods CSA Climate-Smart-Agriculture CSG Community Support Group CU2 Children under 2 years CU5 Children under 5 years CVA Citizen Voice and Action DCOP Deputy Chief of Party DDM Department of Disaster Management DFSA Development Food Security Activity DGFP Directorate General of Family Planning DGHS Directorate General of Health Services DMC Disaster Management Committee DPHE Department of Public Health and Engineering DRR Disaster Risk Reduction DQA Data Quality and Assurance DTW Deep Tube-Well ELT Entrepreneurial Literacy Training NOBO JATRA PROJECT/“NEW BEGINNING”: MID-TERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 v EPI Extended Program for Immunization FFBS Farmer Field and Business School FGD Focus Group Discussion FFP Food for Peace F2F Farmer to Farmer FtF Feed the Future GBV Gender Based Violence GMP Growth Monitoring and Promotion GoB Government of Bangladesh HCA Host Country Agreement HDRC Human Development Research Center IDI In-Depth Interviews IGA Income Generating Activity IPHN Institute of Public Health and Nutrition IPTT Indicator Performance Tracking Table IYCF Infant and Young Child Feeding LGD Local Government Division LLC Limited Liability Company LSBE Life Skill Based Education LSP Local Service Provider LOA Life of Award MAM Moderately-Acute Malnutrition MAMA Mobile Alliance and Maternal Action M&E Monitoring and Evaluation MCHN Maternal Child Health and Nutrition MEG Male Engagement Group MIS Management Information System MNP Micro-Nutrient Powder MoDMR Ministry of Disaster Management and Relief MoF Ministry of Finance MoFHW Ministry of Family Health and Welfare MoLGRD&C Ministry of Local Government, Rural Development and Cooperatives MoU Memorandum of Understanding MoWCA Ministry of Women and Children Affairs NOBO JATRA PROJECT/“NEW BEGINNING”: MID-TERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 vi MTE Mid-Term Evaluation MUAC Mid Upper Arm Circumference NID National Identification Card NJP Nobo Jatra Project NRM Natural Resource Management PIO Project Implementation Officer PITA Participation, Inclusion, Transparency and Accountability PLW Pregnant and Lactating Women PNC Post-Natal Care PRA Participatory Rural Appraisal PSF Pond Sand Filter RRAP Risk Reduction Action Plan SAE Sub-Assistant Engineer SAM Severely-Acute Malnutrition SBCC Social Behavior and Change Communication SMC School Management Committee SOD Standing Order on Disaster SRHR Sexual Reproductive Health and Rights SUAP Safer Use Action Policy Tk Taka ToC Theory of Change ToF Training of Facilitators ToT Training of Trainers UDMC Union Disaster Management Committee UN United Nations UPSC Union Parishad Standing Committee USAID United States Agency for International Development USD United States Dollar VDC Village Development Committee WASH Water, Sanitation and Hygiene WATSAN Water and Sanitation WBA Well-Being Analysis WFP World Food Program WHO World Health Organization NOBO JATRA PROJECT/“NEW BEGINNING”: MID-TERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 vii WVB World Vision Bangladesh WVI World Vision International NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 1 EXECUTIVE SUMMARY The Mid-Term Evaluation (MTE) of the Nobo Jatra Project (NJP), conducted by Development Technicians LLC, took place over March to August 2018. A team of six specialists conducted the evaluation, performing desk reviews over March-April, and conducting field data collection from April 23 to May 11, in 20 villages across the four working Upazilas of the project. A total of 50 Focus Group Discussions (FGD) and 168 In-Depth Interviews (IDI) were completed as part of the data collection. The team also completed interviews with government counterparts and staff within each of the Upazilas, as well as with District staff based in Satkhira and Khulna regional offices. NJP targets 123,794 households in 40 Unions, within Shyamnagar, Kaliganj, Koyra and Dacope Upazilas in the two Districts of Khulna and Satkhira. A total of 856,116 individuals are expected to directly benefit from the project. The goal of NJP is “to improve gender equitable food security, nutrition and resilience of vulnerable people within Khulna and Satkhira Districts in Bangladesh.” To achieve this goal, the project has four purposes:  Purpose1 – To improve the nutritional status of children under five (CU5) years of age, pregnant and lactating women (PLW), and adolescent girls.  Purpose 2 – To increase equitable household income.  Purpose 3 – To strengthen gender equitable ability of people, households, communities and systems to mitigate, adapt to, and recover from natural shocks and stresses.  Purpose 4 (Crosscutting) – To improve social accountability and national policy engagement of service provision for vulnerable men and women. Although NJP was approved in September 2015, the project could not begin active implementation until October 2016. The delays were primarily related to signing of the Host Country Agreement (HCA), before which NJP could not legally start interventions. The HCA was signed on August 7, 2016, resulting in a delay of one year. Hence, at this mid-point, the project has only intervened for a period of 1.5 years. Activities promoted by NJP have been piloted or implemented by other projects in and around the same areas. These include activities from the Climate Resilient Ecosystem and Livelihoods (CREL) and Agro-Input Retailers Network (AIRN) projects, both funded by USAID, with the goal of increasing food availability and access, and diversifying livelihoods. Although these projects have now concluded, the MTE team observed evidence of activities still on-going, such as the Tower Garden which was piloted and introduced by CREL. The MTE team found it commendable that NJP is building on models from these projects. The major findings of the MTE are as follows. Purpose 1: WASH: Community members were appreciative of the SBCC-WASH sessions, where the MTE team directly observed that the project has made progress in changing behaviors and beneficiaries were adopting better health and hygiene practices, especially in areas where improved latrines had been installed. Evidence of these behavior changes were seen when MTE team members visited latrines, with no prior warning, regardless of whether or not these were provided by NJP, and found the majority of latrines were clean and contained soap, sandals, and water nearby. The NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 2 SBCC-WASH sessions combined with the construction/rehabilitation of latrines, the establishment of water points, and the strengthening of WATSAN committees have all contributed to improving the hygiene and sanitation status of project beneficiaries. However, it was evident that the needs of the broader NJP participants were not being met as the project had only a limited number of beneficiaries who had received improved latrines, and the evaluators were constantly requested by community members to inform the project of their need for a latrine. This indicated that participants were not fully conversant on how to be included for getting support on improved sanitation structures, and also indicated that the project did not have the resources to satisfy the needs for all its participants in this intervention. Access to safe drinking water is the existential threat across the south-west of Bangladesh, due to the high levels of salinity across this region. NJP is attempting to ease access to water through constructing/rehabilitating Pond Sand Filters (PSF) and Deep Tube-Wells (DTW). Most of the community-based water options provided by the project have managed to develop bank accounts as maintenance funds. However, there has been mixed progress in collecting maintenance funds from users, which if not strengthened and formalized may impact on the future sustainability of these water points. With regards to sanitary latrines, NJP faced a challenging prospect due to the scarcity of land, making it difficult for the project to maintain the required environmental safeguards such as having latrines 10 meters away from water sources. This reality has contributed to the limited progress of NJP to install and rehabilitate sanitary latrines. NJP has taken steps to minimize this problem where it has requested approval from the FFP BEO that this requirement should only be applicable for “potable water” sources, and not be applicable to all water sources. The MTE team also observed many of the latrines that were rehabilitated were not always of good quality. However, the MTE team did find hygiene practices were being better observed in improved latrines, regardless of whether they were provided by NJP or other organizations, and found these to consist of hand washing areas, soap, sandals and water. MCHN: The MTE found that NJP has strengthened Community Clinics and developed better linkages with Upazila Health Complexes. Health staff from the government were provided basic training at these complexes, which was followed on by further training at the Union Parishad level. When interviewing health staff at the Upazila level and at Community Clinics, the staff were appreciative of these trainings. Growth Monitoring and Promotion (GMP) sessions were highly regarded by attending mothers. The number of children participating in GMP sessions have already exceeded targets (134 percent achievement). The MTE team when interviewing mothers attending GMP sessions and SBCC￾MCHN sessions, observed that the project has made progress in raising awareness on appropriate mother and child care, especially during pregnancy and for taking care of children under two years of age (CU2). On interviewing mothers, the MTE team found that they were appreciative of the growth monitoring activities in GMP sessions where CU2 children were weighed and height measurements were taken, and screening was done for MAM and SAM children. It is to be noted that although the MTE team did not encounter any MAM or SAM children in the sample sites that were visited, they were told by mothers that CNFs were prepared to use MUAC measurements if such cases were found. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 3 Mothers also shared that they appreciated the learning that took place from CNFs sharing essential messages on mother and child care, and in the discussions that took place between mothers on these issues. During these sessions, CU2 were provided MNP – however, the disposal of these packets was not being carefully monitored that could negatively impact on the environment. The SBCC-MCHN sessions were found to be an effective platform to sensitize mothers on proper child care practices, and on health, hygiene and nutrition. The courtyard sessions observed by the team were found to be engaging, and the CNFs were found to be effectively demonstrating various issues on child feeding practices. The Conditional Cash Transfer (CCT) intervention by far appeared to be the most popular activity which beneficiaries, government counterparts and partners consistently highlighted. This was not surprising considering that CCT beneficiaries received Tk 2,200 per month, for a period of 15 months. The CCT cash transfer amounts varied significantly with the GoB sponsored CCT intervention, where NJP provided Tk 2,200 compared to the government’s Tk 500 per month to eligible beneficiaries. The Tk 2,200 amount was derived from WFP’s Transfer Modalities Research Initiative study, which determined this sum of money would enable Pregnant and Lactating Women (PLW) to consume a nutritionally adequate diet. However, the MTE team consistently came across tension where mothers sponsored by the GoB were unhappy that they too were not receiving similar sums of money as provided by NJP. NJP needs to be careful that the higher transfers provided by the project does not create issues for the Government of Bangladesh (GoB) sponsored CCT intervention, which are implemented nationwide. Gender: The MTE team found that women empowerment has been one of the more successful components of the project to date. NJP gender teams have invested time to ensure that Male Engagement Groups (MEG) were formed and trained, and that recommendations from the Gender Analysis report were being considered, including the utilization of local resources for reducing child marriage. When interviewing MEG participants, there was common agreement that women and men need to have more equitable distribution of household chores, and that household decisions should be taken equally. The MTE team observed that men appeared to be more sensitive to gender issues, and it was evident that women who were interviewed were confident and comfortable to be speaking on different programmatic issues. A module titled ‘Family Wellbeing and Prosperity’ has been developed through extensive gender research in NJP working areas for use in MEGs, consisting of 12 sessions in the module. The MTE team found that the Affirmative Action approach was well accepted by communities, allowing women to participate in relevant committees and groups in connection to Maternal Child Health and Nutrition (MCHN), Water Sanitation and Hygiene (WASH), Disaster Risk Reduction (DRR), Agriculture, Alternative Income Generating Activities (AIGA) and Promotional Graduation, all of which contributed to the implementation of the National Women Development Policy 2011. Affirmative Actions were promoting gender issues to influential committees and Village Development Committees (VDC) where NJP was already demonstrating evidence of impact on empowering women where the MTE team directly observed significant women representation in management committees such as VDCs. Progress in women empowerment was also evident through the confidence they demonstrated when discussing issues with MTE team members. Many of these discussions took place in mixed female and male settings, where women frequently pressed their points across. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 4 Female Producer Group members were connected with collection points, and female beneficiaries engaged with the Entrepreneurial Literacy Training (ELT) were clearly very proud and appreciative of their improved capability to read and write. Importantly, connections were found to link women agency building with economic empowerment through the Promotional Graduation, AIGA and Agriculture components. Purpose 2: Agriculture & Alternative Livelihoods – AIGA: As a result of the 6-month ELT and trainings on AIGA, the MTE found that the majority of women respondents shared that they were now earning additional incomes of between Tk 800–1,200 per month, with the exception of one woman who shared that she had not earned any income even after 6-months of completing her training. When considering the responses from the majority of women who claimed to be now earning an income, this sum of Tk 800-1,200 per month was not insignificant considering these participants belonged to the poorest quintile, and this additional income certainly contributed to the overall well-being of their family. The MTE team was particularly impressed that these women could plan and choose how much time they could devote to the AIGA. This was important as women shared with the MTE team that they had control over their time, and that they planned their day so as not to become overly burdened. The MTE team were informed that NJP has recently adopted vouchers as the mechanism for AIGA and Agriculture participants, who receive start-up capital, to purchase inputs from enlisted vendors. The MTE team considered this as an important initiative to boost the local economy, and for beneficiaries to establish relationships with input/output markets. Over the Life of Award (LOA), should local markets provide quality services, it is reasonable to assume that NJP participants will continue to strengthen their relationships with these vendors, leading to a more sustainable systemic structure. Agriculture & Alternative Livelihoods – Agriculture: The Lead Farmer approach was considered by the MTE team to be particularly challenging. Having one Lead Farmer be responsible to develop the capacity of up to 200 other farmers, with no incentives asides from receiving two rounds of seeds per annum, and minor inputs to boost their poultry and aquaculture production, seemed overly ambitious. All Lead Farmers interviewed expressed their desire to help their fellow farmers, however, when probed as to how much time they devoted to the various activities to work with their Producer Groups, mixed responses were received. Trainings and follow up monitoring of Lead Farmers by NJP was of questionable quality as none of the Lead Farmers interviewed conducted the Farmer Field Day, and only two of nine had conducted pre-season planning meetings. The MTE team had mixed observations of the demonstration plots where Lead Farmers were not practicing optimal Climate-Smart-Agriculture (CSA) techniques. From home visits to Homestead Producers, the MTE team observed small patches of gardens, most of them bare, indicating poor training and capacity building. Even if these gardens were producing, the areas were too minimal to generate any significant impact. These gardens may be meeting NJPs strategy to provide limited additional food to the household and to promote diet diversity – however, the MTE team considered these to be too insignificant to make any meaningful changes to the overall nutrition of the household. These were unlikely to sustain beyond the life NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 5 of the project as Homestead Producers lacked the skills to maintain these gardens when project support and monitoring ends. 62 of the 124 trained Lead Farmers were also members of the Value-Chain Producer groups they were responsible for. These Value-Chain Producer groups were more likely to sustain as these Lead Farmers were motivated to ensure their success considering they had a direct stake in these groups. In general, Value-Chain Producer members were selected only if the community recognized them as commercial small holder farmers, which invariably meant these were wealthier farmers. This afforded them greater opportunity to invest in commercial farming, and when working as collectives, were able to generate greater volumes of commodities giving them more negotiation power to agree on optimum sale prices to markets. In addition, Producers involved in bitter-gourd, bottle-gourd and brinjal value-chains were provided seeds over two seasons (valued at Tk 700 or approximately US$ 9), water-melon Producers were provided seeds for one season, fingerlings were provided to GIFT Producers, and poultry provided to Poultry Value-Chain Producers – all of which were valued at Tk 700 (US$ 9) per participant. NJP’s strategy to develop Homestead Producers was primarily to diversify their homestead production to contribute towards an improved nutritionally balanced diet within their own households. Homestead Producers received either seeds over two seasons, fingerlings or poultry, valued at Tk 700 (US$ 9) per participant. Even though the input provided by the project was similar to that of the Value-Chain group members, Homestead Producers could use the inputs to develop small home-gardens to only produce small surplus food to diversify their household meals. Unlike Value-Chain Producers, Homestead Producers did not have any additional resources of their own to further boost production, which effectively disabled them to participate in any viable value￾chains. Although it was argued that these participants had little to no agricultural land to participate in value-chains, alternative options of leasing “khas”1 land could have been considered. Agriculture and Alternative Livelihoods – Promotional Graduation: NJP approaches the graduation model differently to the traditional BRAC approach. The 24-month BRAC model is staged where in the first 3-months the targeting is completed, consumption support and formation of micro-credit groups take place between months 3-6, and skills training and asset transfer is around month six. The remaining 1.5 years is devoted to support on-the-job capacity building. The NJP approach differs where the 24-months only begin after the targeting has been completed and the Promotional Graduation groups have been formed. The project focused considerable time on the ELT component over the first 9-months, and start-up capital and start of IGAs only began from month seven (whereas the BRAC model provides these from month three). The impact of the BRAC approach has been widely analyzed and considered to be an impactful approach. However, the NJP approach to graduation could not be critically assessed due to the early stages of its implementation, and hence it was not possible to determine the level of impact this will bring over the LOA. Promotional Graduation participants were universally appreciative of the ELT, where they took great pride in their basic ability to read and write. The Savings and Lending component was another intervention which clearly motivated these participants, where all groups were able to show the evaluators their savings boxes, with the mandatory three keys. Each group had elected a leader and a treasurer, and all members generated savings through the purchase of shares at Tk 50 per 1 Khas land are government owned fallow land, over which no individual has property rights NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 6 share, and it was evident that loans were being dispersed to members. By-laws had been established, where members who were facing particular hardship were granted extensions to repay loans, and should they not be able to repay due to unforeseen circumstances, then loans would be waived. The Savings fund served as a “social fund,” where by-laws allowed funds to be used for community needs such as paying for funerals. The engagement of numerous partners in the Promotional Graduation intervention – WVB, Winrock and WFP, supported by their three national partners Codec, Shushilan and Nobolok, has led to confusion. Field staff, from both Winrock and Codec, relayed that WFP’s intervention from month six, to identify IGAs, whilst the ELT was taking place, was disruptive. There were also misgivings as to why WFP were engaging in the Promotional Graduation IGA component, considering Winrock were already implementing the 6-month AIGA component. It was not clear to the MTE team what the difference was between the 24-month Promotional Graduation IGA and the 6-month AIGA components, as both engaged in “on-farm” and “off-farm” income generating activities, where the primary difference was that the former invested in a longer support period as part of the graduation approach. However, the technical content could be considered to be similar, which led the MTE team to believe this to be a duplication of much of the work. Start-up capital was considered by all beneficiaries to be important. This was not surprising considering the poor status of these households, where this seed capital enabled beneficiaries to purchase inputs for their chosen business plans. However, inconsistency in start-up capitals provided to different sets of participants was found to be causing major issues in the field. NJP provided Tk 24,000 (approximately US$ 300) to graduation participants (Tk 1,000 per month over the first 9-months, followed by a start-up capital of Tk 15,000); Tk 1,000 (approximately US 13) as a one-time start-up capital to 18 percent of the 6-month AIGA participants (3,200 participants); and no start-up capital to the remaining 82 percent (14,800) AIGA participants. Although the project has taken great effort to target the poorest (among the Ultra Poor) to be included in the Promotional Graduation component, only 14,000 of the identified 62,137 Ultra Poor could be accommodated as Promotional Graduation participants. The remaining Ultra Poor participants were then included in the other components of the project, including the AIGA component. Hence, it could be inferred, that the socio-economic characteristics of the Promotional Graduation and the AIGA participants were similar. This has raised questions among the AIGA members why Promotional Graduation participants receive such higher levels of resource, considering they are both from the Ultra Poor category. This has been further complicated by the split within the AIGA group where only 18 percent received Tk 1,000, and the remainder received no inputs asides from the training. Front line staff repeatedly expressed this as a challenge, where participants were questioning the logic of this approach. This indicated that an effective communication strategy needed to be developed and unrolled explaining to participants that Promotional Graduation participants were part of a larger programmatic effort to pull them sustainably out of poverty, and hence were provided significantly more resources to reach this goal. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 7 Purpose 3: Disaster Risk Reduction: Community Risk Assessments (CRA) were being developed, together with Union Disaster Management Committees (UDMC), at both Ward and Union levels. This was especially well described by the Bajua Union Parishad Chairman, who had actively participated in the development of the CRA and presented this to the Upazila level for final approval before proceeding with the development of the Risk Reduction Action Plan (RRAP). Three training modules, in both Bengali and English, have been developed as capacity building tools for Disaster Management Committees (DMC), following the guidelines of the Standing Order on Disaster (SOD). NJP has made good links with the Cyclone Preparedness Program (CPP), where NJP Youth Groups, VDCs and UDMC members have conducted several simulation exercises as part of the CPP. To formalize the partnership, an MoU has been signed between WFP and CPP on September 25, 2017, to include NJP Youth Groups to become members of CPP committees in 28 Unions where NJP operates. Purpose 4: Good Governance & Social Accountability: The Union Parishad Act-2009 formalized the formation of 13 Standing Committees in each Union Parishad, where Union Parishad Standing Committees (UPSC) represent communities in Union level decision making processes. To support this structure, NJP works with five standing committees: a) Education, Health and Family Planning, b) Agriculture Fisheries, Livestock and Economic Development, c) Sanitation, Water Supply and Sewerage, d) Social Welfare and Disaster Management, and e) Family Arbitration Women and Children Welfare. On interviewing government counterparts, it was evident to the MTE team that NJP’s partnership was valued, and that the project had brought positive changes in the frequency and timeliness of local government meetings, the documenting of meetings, and follow up on decisions. The project has initiated the Citizens Voice and Action (CVA) approach to mobilize communities and service providers, with similar principles to the Participation, Inclusion, Transparency and Accountability principles (PITA), adopted by USAID (2016). Project monitoring reports indicated that NJP has unrolled the CVA approach across 40 Unions, prompting greater participation of communities to identify their own village related problems, and in the formulation of Village Development Plans detailing how they intended to overcome these issues. Actions plans were being shared at Union and Upazila meetings, and different support services were planning how best to provide assistance in line with these plans, thereby strengthening the social accountability process. Local media played an important role in the CVA approach, widely sharing Village Development Plans and the commitments made by local government to address these issues. As a result, the MTE team found that 39 Union Parishads have now included budgets to support activities based on Village Development Plans. NJP is continuing to work on this approach to ensure that the remaining one Union also includes a budget based on its Village Development Plan. As an important sustainability approach of continued support, NJP had facilitated the formation of 622 VDCs in 40 Union Parishads, and has actively strengthened the participation of VDCs in observation days, and in the Wardshava open budget meeting, to raise their demand that the Union Parishad includes their needs in the annual planning and budgeting process. The MTE team were impressed with the level of engagement of women in VDCs, both in terms of numbers and NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 8 participation, where their enthusiasm and self-worth was evident. NJP is commended for their contribution in strengthening the role of women empowerment across these remote, and in most cases, conservative areas. From the interviews with VDC members, the MTE team found both women and men to be aware of the importance of the VDC as an overall coordinating body, and accordingly were found to be engaging with the various platforms that existed within the community. VDC members also appeared to be linked with local government and elected officials, where the MTE team was taken to meet different local elites after concluding VDC interviews. These were positive indications that these VDC members will continue to maintain the VDC after the closure of NJP. The key recommendations from the MTE are as follows. Overall: It is recommended that a clear Exit and Sustainability Strategy is developed. The strategy should clearly relay how the project intends to build in sustainability parameters against different activities from the onset, and how the project intends to gradually hand-over activities to be taken up by communities and surrounding support structures. It is recommended that findings from the Tufts study2 on exit strategies be referenced for this work. Notable findings from the Tufts assessment were that  Project success at the end of the project did not necessarily translate to longer-term sustainable impact. The study found that although some projects appeared to be successful during the project, gains were quickly eroded when project support ceased as communities and support structures had not been sufficiently strengthened to continue making progress on their own.  To achieve long-term sustainable impact, developing resources, capacity and motivation were critical.  Linkages, especially vertical linkages, such as those between community-based organizations or individuals and existing public or private sector institutions, were usually critical for successful phase-over of responsibilities.  In the exit process, a gradual transition from project-supported activities to independent operation was important for sustainability.  Providing free resources threatened sustainability, unless replacement of those resources both as project inputs and as incentives had been addressed. The MTE team also recommends that NJP develop clear strategies on how to include households which are not directly participating in project activities, so that they too benefit from the project. It is recommended that the current practice of sensitizing the entire community through platforms such as SBCC-WASH be continued and expanded into other technical areas. This could include encouraging non-participating households to observe and learn from demonstration plots, and take an active interest in technical groups operating in their village such as the VDC, MEG, and DRR groups. 2 B Rogers, J Coates “Sustaining Development: A Synthesis of Results from a Four-Country Study of Sustainability and Exit Strategies among Development Food Assistance Projects,” December 2015 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 9 WASH: Access to safe drinking water is a significant challenge target communities are facing. NJP is exploring a number of technologies including reverse osmosis. While that technology was adequate in certain areas, several communities have said that they will not be able to pay for its maintenance. Certain Upazilas have nonetheless found it adequate for their situation. From discussions held with the Additional Chief Engineer – Works of the Department of Public Health and Engineering (DPHE), and the Project Director and Deputy Project Director of the Bangladesh Rural Water Supply and Sanitation Project of DPHE, several possible alternative sources of water were discussed. It is recommended that NJP makes a careful selection of water technology to install and get the concurrence of the beneficiary community before any final determination is made and consider different rain-water harvesting options; scaling-up the construction / rehabilitation of PSFs and DTWs; and introducing water purifying tablets at times of water scarcity to reduce the spread of disease. The MTE team recommends that Water Management Committees are trained more rigorously on maintenance processes of water points by introducing a “fee-for-use” approach; developing a checklist of what needs to be maintained and when; training Water Management Committees on maintenance functions; and providing tools to these committees, immediately after training, to enable them to effectively maintain their water points. It is also recommended that NJP actively links with other agencies involved in developing improved latrines. This could include NJP taking on the role of developing soft skills of communities through the SBCC-WASH platform, to compliment the installation/rehabilitation of improved latrines by the partner agency. At the same time, NJP is recommended to continue working with WATSAN committees and advocate for government to plan for appropriate budgets and support for the scale-up of improved latrines. MCHN: From the MTE, it was evident that CCT mothers were using their cash transfers, not only for purchasing food, but also for other household needs. From the primary data collection, the MTE found that approximately 20 percent of CCT respondents said they saved Tk 1,000 per month, to contribute in building other assets within their households. This indicated that there remained a lapse in understanding of the purpose of the cash transfer where this was intended to enable the purchase of nutritious food to supplement the diets of mothers and CU2. However, given the reality of the poverty that existed within these households, it is reasonable to assume that most mothers would use a portion of the monthly Tk 2,200 to either share food among other household members, or for the expenses to cover other essential needs of the household. Considering this reality, it is recommended that NJP determine a fixed threshold of how much of the monthly CCT must be used for purchasing food, and what is an acceptable level of the cash that may be used for other purposes. It is also recommended that messages are reinforced through the GMP and SBCC-MCHN sessions, that the purpose of the cash is to purchase sufficient nutritious food for the mother and child, and that this will have a life-changing impact for their child. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 10 Gender: The project has made commendable progress in its women empowerment effort, which was directly observed by the MTE team when interviewing women in groups such as VDCs, Savings and Lending groups, and AIGA groups. The MTE found that women were confident, and speaking freely in front of their male counterparts on the range of issues raised by different MTE team members. However, the MTE found that there remained a gap where relevant gender related oversight from the government was largely missing. Accordingly, the MTE team recommends that NJP advocates for greater engagement of the government Gender Monitoring Committees engaging District and Upazila Women Affairs Officers who are based in NJP working areas. Agriculture and Alternative Livelihoods – AIGA: The MTE team considers the engagement of Winrock and WFP, together with their national partners, for the Promotional Graduation component to be a duplication of effort and leading to confusion between partners. The MTE team also considers the differences in definition between the IGA and AIGA components does not justify why two different agencies should be involved in these components. It is recommended that given Winrock’s expertise in this area, and the fact that they already have a full technical team based in Khulna focused on AIGA, the project considers Winrock to be wholly responsible for all IGA and AIGA components. It is also recommended that the project use one terminology – AIGA, to describe this activity, rather than using two terms which can be misleading. The term AIGA seems most appropriate as the aim of the project is to diversify the income base (both on-farm and off￾farm) of its participants, which directly impacts on developing the resilience of these households. Agriculture and Alternative Livelihoods – Agriculture: It is recommended that NJP’s current strategy, with its 12,000 Homestead Producers, be reconsidered. The MTE revealed that the gains made thus far were too minimal for these activities to be sustainable. The inputs currently provided were insignificant to bring about transformational change, and likely would not sustain after the project ends. It is recommended that the current strategy of only increasing subsistence production to contribute to household meals be re-strategized where these participants are provided sufficient support to break their cycle of poverty. This may entail looking beyond what these subsistence farmers are able to produce in their homesteads, as in many cases these are too small to effectuate larger change. The MTE team recommends that the Lead Farmer approach be reconsidered. The current approach is too ambitious for one Lead Farmer to develop capacity of up to 200 other farmers through demonstration plots, Farmer Field Days and pre-season planning. A more substantial effort is required to develop the capacity of mainly subsistence farmers (12,000 Homestead Producers), as well as the 8,000 Value-Chain Producers. Approaches such as the Farmer Field and Business School (FFBS), and the Farmer to Farmer (F2F) approach can be considered by NJP as a more effective means of reaching out to its 20,000 Agriculture participants. This would enable a greater number of FFBS leaders to be developed, who could work with their groups of 20-25 farmers in the range of agricultural practices being promoted by NJP. The F2F approach can also be an effective means of reaching out to further sets of participants, such that the entire community benefits. This is important to ensure that a “population-based” impact is achieved. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 11 Agriculture and Alternative Livelihoods – Promotional Graduation: The MTE team recommends that NJP develops an effective communication strategy to explain to participants why it offers three different types of start-up capital – Tk 1,000 per month for a total of 9-months during ELT sessions, together with a start-up capital of Tk 15,000 to Promotional Graduation participants; Tk 1,000 to 3,200 AIGA participants; and no start-up capital to 14,800 AIGA participants. This will reduce the tension among AIGA and Agriculture participants as to why the Promotional Graduation participants receive significantly more resources. With regards to the duration of ELT, it is recommended that the project reviews whether or not the 9-month duration for the Promotional Graduation participants can be reduced to 6-months which is similar to what the AIGA and Agriculture participants receive. This recommendation stems from the fact that AIGA and Agriculture participants are from a similar socio-economic group to that of the Promotional Graduation participants, as only a portion of the Ultra Poor could be accommodated in the graduation component, and the remaining beneficiaries were placed in the other two components. DRR: The MTE team recommends that NJP advocates, at the National level Steering Committee, for greater resources to be allocated for the development of RRAPs and CDMAPs. It is important to have these discussions at this strategic management level as funds can only be allocated from the national budget to support these initiatives. It is recommended that NJP works with the Disaster Ministry to determine the minimum structure that is required to realistically prepare DRR departments at the District and Upazila levels for upcoming natural shocks and bring this up as a priority agenda in the National Steering Committee meeting. Having appropriate structures in place will help develop long term sustainable support to withstand shocks and stresses, thereby developing the overall resilience of these areas. The Acceleration Fund, although a good initiative, is vastly under-resourced for many of the Unions in which NJP operates. The MTE recommends that this be discussed by the National Steering Committee to seek additional funding sources for Unions which are under greater threat of river erosion, water-logging and salinity. Good Governance and Social Accountability: One of the greatest challenges to projects such as NJP, is the deliberate expansion of “unregulated” saline gher farms across the south-west of Bangladesh. Gains made by development projects will be short-lived unless active measures are taken to advocate against local power structures, who include powerful businessmen and elected officials, abusing their influence to illegally expand gher farms. It is recommended that NJP seek greater coalitions, together with the support of the World Vision country office. The MTE team does not consider NJP as an “activist” project, however, it wields considerable influence given the large number of participants it reaches, in some of the remotest and hardest to reach areas of the country. NJP and WVB are in a unique position to invite other credible organizations, actively engaged in “rights-based programming,” to form coalitions at multiple levels to influence government to take greater action to mitigate the deliberate salination of agricultural land. It is recommended that an Expert Consultant be engaged to investigate this component further, and to work with the Advocacy specialists of NJP and WVB to identify potential partners, and to develop platforms and strategies on how best to work in this advocacy arena. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 12 As part of the sustainability effort, government extension and support structures need to be effective and responsive to the communities they serve. To strengthen these systemic structures, it is recommended that the National Steering Committee meet twice a year. It is recommended that these meetings review action plans and budgets developed at the local Upazila and Union levels, and obtain commitments from the different ministries and departments to support these local level efforts through technical and budgetary support as needed. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 13 1. Introduction The Nobo Jatra Project (NJP) is a 5-year project funded by the United States Agency for International Development / Office for Food for Peace (USAID/FFP). The project is implemented by World Vision Bangladesh (WVB), Winrock International and the United Nation’s World Food Program (WFP), in collaboration with the Government of Bangladesh (GoB) and three national partners. The goal of NJP is “to improve gender equitable food security, nutrition and resilience of vulnerable people within Khulna and Satkhira Districts in Bangladesh.” NJP targets 123,794 households in 40 Unions, within four Upazilas, in the two Districts of Khulna and Satkhira. A total of 856,116 individuals are expected to directly benefit from the project. The project has identified two livelihood zones as primary geographic targets in the southwestern coastal areas: the Khulna Fisheries and Rice Livelihood Zone, and the Sundarban Forest Product Collection and Crab-Fishing Livelihood Zone. Within these two zones, the consortium has identified four geographically contiguous Upazilas in Khulna and Satkhira Districts, characterized by chronic food insecurity. Although this region does not show the highest rates of malnutrition in Bangladesh, the rates, nonetheless, are high, and the percentage of families living under the lower poverty line ranges between 33 to more than 50 percent.3 Development Technicians LLC was contracted by World Vision International (WVI) to conduct an independent Mid-Term Evaluation (MTE) of the project from March to September 2018. 3 documents.wfp.org/stellent/groups/public/documents/ena/wfp240049.pdf http://documents.worldbank.org/curated/en/916761468211763695/pdf/904870WP0WB0Po00Box385319B00PUBLI C0.pdf Figure 1 – NJP working areas NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 14 2. Project Overview 2.1. Objectives and expected results The project has four primary purposes: Purpose1: Is “to improve the nutritional status of children under five (CU5) years of age, pregnant and lactating women (PLW), and adolescent girls.” Interventions focus on improving utilization of Water, Sanitation and Hygiene (WASH) practices; reducing adolescent pregnancy; increasing equitable nutritious food intake; and increasing practice of gender-equitable norms in the household, including food distribution, workload, a supportive environment, and greater decision￾making. Purpose 2: Is “to increase equitable household income.” Interventions focus on increased income and equitable access to nutritious food, for both male and female, through increased production of safe, diverse, nutritious, and high-value foods. Activities focus on increasing the diversification of livelihoods through strengthened linkages to the private sector, to identify and select market￾based livelihoods that improve employment opportunities and income. Purpose 3: Is “to strengthen gender equitable ability of people, households, communities and systems to mitigate, adapt to, and recover from natural shocks and stresses.” Efforts are made to improve the capacity of both the targeted communities and the governmental institutions mandated for delivering Disaster Risk Reduction (DRR) services. Purpose 4 (Crosscutting): Is “to improve social accountability and national policy engagement of service provision for vulnerable men and women.” Interventions aim to improve responsiveness of market-based local service providers (WASH and agriculture), and increase responsiveness of government service providers (health and nutrition, DRR, agricultural extension, and social protection). 2.2. Performance indicators NJP’s performance indicators specify data to be collected, where the monitoring and evaluation (M&E) plan includes 120 indicators to be used to monitor progress and manage performance through the Life of Award (LOA) (Table 1). These include USAID/FFP required (R and RiA) indicators to facilitate USAID/Bangladesh’s roll-up reporting, as well as indicators that USAID/Bangladesh has established as part of its Development Objectives Performance Management Plan. In cases where these pre-established R and USAID/Bangladesh indicators do not fully measure progress toward NJP’s planned results, the project has included customized indicators. Together, these indicators provide a comprehensive measurement of NJP’s effectiveness in meeting targeted results. Details on each of the indicators, including definitions, data sources, and frequency of data collection and reporting, are provided in the Performance Indicator Reference Sheets that can be found in the project monitoring and evaluation manual. Table 1: Summary number of NJP indicators Level of indicator FfP-R FFp-RiA Custom Total Impact 6 1 - 7 Outcome 34 30 64 Output 3 8 38 49 Total 9 43 68 120 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 15 2.3. The NJP Theory of Change The NJP Theory of Change (ToC) has been developed as part of the project design and includes several levels of logic for each thematic area as follows: 1. Increasing access to clean water and improved sanitation. 2. Improving Mother, Child, Health and Nutrition (MCHN) practices, including infant and young child-feeding (IYCF). 3. Increasing use of Climate-Smart-Agricultural (CSA) techniques and natural resource management (NRM). 4. Improving income diversity, asset maintenance, and savings. 5. Developing alternative livelihood opportunities for youth. 6. Increasing mobility and voice for women to promote gender equality. 7. Helping communities influence change at national policy level, through their greater engagement in local and regional level decision making. 8. Reducing chronic poverty, with households gaining greater resilience leading to transformative change in the lives of individuals of Khulna and Satkhira Districts. 2.4. NJP implementing partners and management structure The NJP consortium is primed by WVB, with WFP and Winrock as subs, together with three national partners – Shushilan, Codec and Nabolok. WVB is in overall management of the project and takes the lead on MCHN, WASH, Promotional Graduation, adolescent, community and stakeholder mobilization, compliance and quality assurance, and local partnership management. Together with Shushilan, WVB are responsible to intervene in the areas of good governance and government relations, policy engagement, social accountability, gender, and the development of Savings and Lending groups. Winrock works with Codec to implement the Entrepreneurial Literacy Training (ELT), and the Agriculture and Alternative Livelihoods activities. WFP, working with Shushilan and Nabolok, is overall responsible for transferring cash under the Conditional Cash Transfer (CCT) and Promotional Graduation components, and developing Income Generating Activities (IGA) for Promotional Graduation participants. Figure 2 – NJP management structure NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 16 3. Objectives of the evaluation The MTE explores how effectively NJP is following implementation plans and meeting targets, the acceptability of interventions by beneficiaries, and early signs of change that beneficiaries associate with project interventions. The objectives of the MTE are: Objective 1 – Quality and effectiveness: To evaluate the quality and effectiveness of service delivery, the strengths and weaknesses of implementation and management, and the quality of outputs in terms of adherence to terms agreed with FFP. Evaluate their appropriateness and perceived value to target communities and identify factors that appear to enhance or detract from the quality, acceptability, and usefulness of implementation and outputs. Objective 2 – Evidence of change: To present early evidence of change (intended and unintended, positive and negative) associated with activity interventions and outputs, assess how well the observed changes reflect the ToC and logframe, and identify factors in the implementation or context that impede or promote the observed and intended changes. Objective 3 – Collaborative learning and action: To review systems for capturing and documenting lessons learned, assess the extent to which they are used in project implementation and to refine project design, and including feedback from stakeholders and participants. This includes assessing processes and evidence, baseline results and monitoring data, for adjusting project strategies. Assess how well the project is seeking out, testing, and adapting new ideas and approaches, to enhance its effectiveness or efficiency. Objective 4 – Sustaining project impact: To determine the extent to which outcomes, systems, and services are designed and being implemented to continue after the project ends, and assess progress made on implementing sustainability strategies. Objective 5 – Crosscutting themes: To determine the appropriateness and effectiveness of support for gender equity in terms of access to, participation in, and benefit from project interventions. Assess the extent to which project interventions target youth, support greater capacities for local governance, and address sources of environmental risk. Objective 6 – Project framework: To recommend adjustments to the ToC and logframe, activity design, resource allocation, activity management, M&E plan, or implementation that could improve the likelihood of achieving desired results by the activity’s end, based on the evidence collected and conclusions drawn for the evaluation objectives above. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 17 4. Methodology Phase 1: Preparation Phase I began in mid-February 2018, following contract award. The Team Leader and the Deputy Team Leader had an introductory call with the project Chief of Party (COP) and M&E Manager on February 27, 2018, followed by a wider introductory meeting with USAID and World Vision on March 5, 2018. Key NJP documents were shared with the MTE team in February, enabling the team to start on the desk review and prepare the Inception Report. During the preparation phase, extensive discussions were held between the MTE team members to develop a common understanding, and for the different specialists to draft tools and approaches in their respective areas of expertise. The Deputy Team Leader arrived in Dhaka in mid-April to lead, together with the MTE M&E Specialist, a two-day training workshop (April 18-19) with all the national MTE team experts. Three technical staff from FFP/Washington and the FFP Desk Officer from USAID/Bangladesh overseeing NJP also attended the training session, to share experiences of MTEs conducted in other countries implementing Development Food Security Activities (DFSA). During training, the evaluation methodology and data collection tools were discussed, as well as anticipated challenges and limits for the evaluation. The FFP team accompanied the MTE team for the field data collection in Satkhira for a period of nine days. At the end of each day, during the debriefing sessions, the MTE team benefited from the observations of the FFP team, as the team discussed the findings from the day. Phase 2: Data collection and synthesis Following preparation, the MTE team started the data collection and synthesis phase. This followed a three-stage process: 1) identification of the criteria to be used for site selection, 2) identification of the sites to be visited and respondents to be interviewed, and 3) analysis and report preparation. The MTE adopted a “purposive” approach to select the sample frame. Stage 1: Villages were clustered within the Dacope, Koyra, Shyamnagar and Kaliganj Upazilas, using the criteria below (based on information received from NJP).  Village Progress: Clustered according to NJP’s internal assessment, rating villages as “Good Progress” and “Limited Progress”. These were subjective assessments conducted by each of the regions where NJP management and field staff graded their villages based on their own observations on how effectively they considered different activities were being implemented.  Markets: Clustered according to NJP’s internal assessment based on level of access to markets – rated as “Good Accessibility” and “Limited Accessibility”.  Risk: Clustered according to NJP’s internal assessment on proneness to disasters – rated as “High” or “Low”. MTE Team with USAID Colleagues NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 18 Stage 2: After clustering villages using the above parameters, a simple random number generator was used to select 13 villages for each Upazila. If a village appeared more than once, it was replaced with another village using the simple number generator, until 13 unique villages were selected for each Upazila. The generator then selected five villages, from the list of 13 villages, within each of the four NJP Upazilas. Accordingly, a final list of 20 villages was selected as the sample frame for the MTE primary data collection. NJP was only informed of the villages to be visited shortly before the field visits to maintain an element of surprise. Stage 3: The MTE team conducted the field data collection from April 23 to May 11, in all of the 20 sample villages (five villages per Upazila). The team also completed interviews with government counterparts and staff within each of the Upazilas, as well as with District staff based in Satkhira and Khulna regional offices. A total of 50 Focus Group Discussions (FGD) and 168 In-Depth Interviews (IDI) were completed (Table 2). Table 2 – Summary of Respondents Interview Satkhira District Khulna District National / Regional Total Shyam nagar Upazila Kaliganj Upazila Koyra Upazila Dacope Upazila Water Point Management Committee – FGD 1 1 2 0 4 Water Point Management Committee Member – IDI 2 2 0 0 4 Conditional Cash Transfer Mother Group – FGD 0 1 1 1 3 Conditional Cash Transfer Mother Beneficiary – IDI 2 1 0 1 4 WASH SBCC Group – FGD 2 2 1 0 5 WASH SBCC Beneficiary – IDI 1 4 3 1 9 MCHN SBCC Group – FGD 0 3 0 0 3 Male Engagement Group – FGD 2 0 0 0 2 Village Development Committee – FGD 4 0 0 2 6 Village Development Committee – IDI 1 4 0 1 6 Promotional Graduation Group – FGD 3 2 1 2 8 Promotional Graduation Beneficiary – IDI 5 2 3 1 11 Producer Group – FGD 0 2 0 1 3 Producer Group Member – IDI 2 2 0 0 4 Lead Farmer – IDI 3 2 2 2 9 Alternative Income Generating Activity Group – FGD 0 0 1 1 2 Alternative Income Generating Activity Beneficiary – IDI 0 1 0 3 4 Youth Member – IDI 0 3 1 0 4 Life Skills Based Education – FGD 1 0 0 1 2 Non-Beneficiary – IDI 1 2 0 1 4 Government Representatives – FGD 0 1 1 3 5 Government Representatives – IDI 4 6 1 5 16 Elected Officials and Influentials – IDI 4 3 1 6 14 Child Protection Committee – FGD 0 1 0 1 2 Local Service Provider – IDI 1 3 1 3 8 Community Clinic Staff – FGD 0 3 0 2 5 Digital Center Staff – IDI 0 1 0 0 1 WVB Staff 31 31 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 19 Winrock Staff 7 7 WFP Staff 12 12 Partner Staff 20 20 TOTAL FGDs 50 (Shyamnagar=13; Kaliganj=16; Koyra=7; Dacope=14) TOTAL IDIs 168 (Shyamnagar=26; Kaliganj=36; Koyra=12; Dacope=24; Staff=70) TOTAL INTERVIEWS 218 For the field data collection, the MTE team was divided into two sub-teams – one led by the Team Leader and the other by the Deputy Team Leader. Each team had three members, with one additional translator to assist the Team Leader to conduct interviews. The MTE team informed the NJP M&E Manager of the sample sites that would be visited the night before the team was scheduled to visit those sites. The short notice was deliberately taken to minimize the risk of the project “show-casing” the areas to be visited before the team arrived. Each morning, on receiving the sample villages and/or government and elected representatives that were to be visited and interviewed that day, the concerned NJP field office ensured that front line staff responsible for the selected villages were present to assist the MTE team in locating the villages. The field office also arranged appointments with government and elected representatives, or their designates, to meet with the MTE team. The MTE team travelled each morning in two separate vehicles directly to the Upazila field office under which sites had been selected. The Upazila Field Coordinator received the team and introduced the field staff that would accompany the two MTE sub-teams to their respective areas. The NJP staff explained the logistical issues to consider, including the time it would take to reach these sites, and whether the team would need to also travel by alternative means such as boats, motorcycles and rickshaw vans for the more remote areas. The MTE sub-teams then separated to travel to their respective sites. On reaching the sample sites, the project field staff located one to three volunteers working in the village, or project participants. These individuals were introduced to the MTE team at which point the project field facilitator broke off from the team and remained at a distance from the village, enabling the MTE team to have candid discussions with NJP participants. Once all planned interviews were concluded, the team returned back to their vehicles where the field facilitator waited for the team to guide them back to the Upazila field office, where the two MTE sub-teams came together again. A similar approach was taken when interviewing government staff at the Upazila and Union levels, where field facilitators introduced the MTE team and then left allowing the evaluators to have candid discussions with government respondents. Each evening, all MTE team members met for up to two hours to discuss the findings from the day, issues encountered, and trends emerging. The same pattern was repeated for the duration of the primary data collection period. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 20 Phase 3: Analysis and reporting Following the field data collection, the team conducted preliminary analysis from May 12–19, based on the four thematic areas of the MTE. The evaluation team used a “content analysis” approach using information gathered from the desk review, IDIs and FGDs. “Contribution analysis” was used to assess the logic within the ToC used by NJP, and its effectiveness in achieving activities, outputs, and outcome results under each purpose statement. This involved reviewing the causal linkages in the ToC, and how different activities would lead to changes, together with their associated risks and assumptions. The MTE considered two possible outcomes – a. changes were occurring as planned, and there was evidence that NJP was contributing to this change; and b. changes were not taking place as planned. For the latter outcome, the MTE team attempted to understand why these changes were not occurring, and what areas of the ToC needed to be re-engineered. The analysis was facilitated using the evaluation matrix developed prior to the data collection. Information from various sources was “triangulated” as part of the analysis, and to draw conclusions and recommendations. Challenges, limits and solutions applied During the development of the inception report, the MTE team considered that potential resistance, when collecting information, may be an issue. Such resistance is normal as many respondents and community members would not be aware why data was being collected, whereas others may feel threatened by the information being gathered. Logistics and coordination would also be challenging in Bangladesh, particularly when traveling in the remote areas where NJP operates. To overcome these potential challenges, the following approaches were adopted:  Involve stakeholders: To deal with potential resistance, the MTE team took time to explain the value of the data being collected, and how this would positively serve to improve their own functions and responsibilities. The MTE team stressed that the intent of the MTE was not to criticize, but rather to constructively suggest recommendations for improvement.  Enlist the assistance of WVB, WFP, and Winrock personnel: Involving this group required being sensitive to their pressures and obligations, arranging with senior management to authorize their support, and minimizing demands on their time.  Minimize respondent burden: Advance notice, keeping interviews short, and conducting interviews at times and places convenient to respondents was important, to not only obtaining reliable information, but also to encourage others within their network to cooperate. Quality assurance The team adopted the following quality assurance approaches to ensure reliability of data collected:  Identification of respondents: As the MTE team reached selected sites, respondents were identified and validated by the different team members.  Daily review of field visits and challenges: At the end of each day, the MTE team held a meeting to discuss field visits and challenges encountered, and what approaches were taken to overcome issues. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 21  Adherence to the “Do No Harm” principle: The Team Leader ensured that MTE team members adhered to the “Do No Harm” principles and ensured that data collection protected confidentiality and participation for all interviewees.  Adherence to the evaluation matrix: The Team Leader ascertained throughout the field data collection that all evaluators adhered to the evaluation protocol and collected appropriate data to respond to the evaluation questions. 5. Evaluation findings and discussions Objective 1: Quality and effectiveness Key Evaluation Question 1.1: Effectiveness in meeting planned schedules, targets and outputs; factors promoting or inhibiting adherence to schedules Although NJP was approved in September 2015, the project could not begin active implementation until October 2016. The delays were primarily related to signing of the Host Country Agreement (HCA), before which NJP could not legally start interventions. The HCA was signed on August 7, 2016, resulting in a delay of 1-year. Hence, at this mid-point, the project has only intervened for a period of 1.5-years. This section provides an overview of the progress made to date by NJP in different interventions, where qualitative observations are elaborated in further detail under Key Evaluation Question 1.3. WASH Strategy as per Implementation Guideline: The World Health Organization (WHO) estimates that there is a strong causal link between poor WASH conditions and undernutrition,4 and studies have demonstrated that diarrhea and intestinal nematode infections are key drivers of low nutrition indicators. Accordingly, through the WASH component, NJP aims to improve these areas by reaching 523,640 beneficiaries with integrated Social Behavioral and Change Communication (SBCC) messaging, 167,200 beneficiaries to gain access to safe drinking water services, and 107,000 beneficiaries to access basic sanitation services. The NJP WASH activities include:  Strengthening and reactivating local Water and Sanitation (WATSAN) committees through training 3,400 WATSAN committee members, participating in meetings, and assisting in the development of WASH plans and community-based monitoring of WASH services.  Facilitating behavior change to improve essential hygienic practices through SBCC platforms led by WASH facilitators and promoters.  Training 10,914 water management committee members to manage and maintain water supply facilities. 4 “Improving nutrition outcomes with better water, sanitation and hygiene,” WHO / UNICEF / USAID 2015 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 22  Facilitating linkages with the Department of Public Health and Engineering (DPHE) and the University of Khulna for water quality testing of all new and rehabilitated water facilities.  Improving access to community and household water facilities through appropriate alternative water options.  Rehabilitation of 8,800 existing latrines, and construction of 8,800 new latrines which meet hygienic sanitary standards, using elevated slabs or sealants to improve their durability.  Promoting and facilitating linkages between consumers and WASH businesses.  Orienting 1,002 entrepreneurs on WASH products to engage more people as pump mechanics and in the construction of sanitary latrines. Progress to date The MTE observed from discussion with WATSAN members that they were knowledgeable and supportive of WASH initiatives in their areas. This was indicative of the good progress made by NJP in the strengthening and reactivation of WATSAN committees where the project has already trained 3,219 of its targeted 3,400 committee members. Early progress in this activity will allow the project to continue providing support to committee members, assisting and developing their skills to manage and support WATSAN efforts in the community. In the area of behavior change, from the SBCC-WASH sessions observed, the MTE found that WASH Facilitators were engaging the group going through picture cards on various themes on WASH and calling out to participants for their active participation. Female participants were found to be confident and actively participating, together with their male counterparts which the MTE team found commendable. Evidence of change was observed particularly in the hygiene conditions of upgraded latrines, where soap, sandals and access to water was apparent in the majority of these latrines when visited by the team. Less progress was observed in the maintenance of water points, where Water Management Committees had yet to ensure that maintenance funds were being routinely collected from users, even though each of the water points had established their own bank accounts for this purpose. NJP intends to tackle this issue through stricter monitoring to ensure Water Management Committees collect water usage fees and deposit these earnings in the established bank accounts for each of the water points. The MTE also observed that many of the Water Management Committees had not been provided tools immediately after completing their training which limited their ability to perform maintenance operations. The slow progress can be attributed to the fact that NJP has only completed training of 1,570 of its targeted 10,914 Water Management Committee members. The construction of all of the water points visited by the MTE team were found to be satisfactory, except for one structure which was a tubewell constructed in the Gabura village (within Gabura Union under Shyamnagar Upazila). Here, the team found that the platform constructed for the tubewell was only four feet by three feet and was not of sufficient area for users to pump water comfortably. Further, the tubewell was of a natural red color, which could be misconstrued as being arsenic affected, as the tubewell had not been painted. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 23 A major shortcoming observed was on water quality testing. Water quality, as per the NJP Program Implementation Guideline, were being tested in DPHE laboratories at the time of being established. However, when visiting sample water points, the MTE found that routine monitoring was being conducted by NJP staff themselves with standard testing kits. When probed, staff shared that the DPHE and Khulna University laboratories were too far away and hence this self-testing approach was adopted. When examining the water quality reports, the MTE observed these to be of poor quality. For example, in Bedkashi Union under Koyra Upazila, field records indicated that arsenic presence was zero, and that salinity measured through chloride content was 1040 ppm. These readings were obvious inaccuracies as the arsenic content cannot be zero, and the allowable limits for chloride content is 600 ppm, going up to 1000 ppm in coastal areas. Apart from the quality of the water tests, the MTE found that routine monitoring was not always being followed, or not being shared with the community. This was evident in Paschim Banshipur Union in Shyamnagar where the President of the Water Management Committee reported he was not aware of any water quality tests that had been performed on this site. When interviewing the Additional Chief Engineer (Works) of DPHE, it was evident that the DPHE recognized their own limitations to conduct water quality tests, and that greater coordination was needed between the local and national DPHE units, as well as with projects such as NJP. NJP and DPHE are already taking steps to improve the quality and reporting of water quality tests, where the agencies were due to sign an MoU in August 2018, agreeing that DPHE will take over all water quality testing in their own labs. Maintenance of water points was another issue that the MTE found to be weak due to several factors. The first was that there was little motivation from community members to generate maintenance funds, through a “fee-for-use” approach, which has critical implications for the future sustainability of these assets. Second, tools were not always being handed over to Water Management Committees immediately after training, which limited the capacity of these committees on maintenance operations. Third, when reviewing the Water Management Training Manual, it was observed there was no checklist for routine maintenance, thereby depriving Water Management Committees of a guide that could be easily referred to for maintenance operations. From discussions between the MTE team and the Project Director and Deputy Project Director of the Bangladesh Rural Water Supply and Sanitation project of DPHE, as well as conversations with the Additional Chief Engineer (Works) of DPHE, the evaluators were informed none of the Ground Water Studies had identified good layers of aquifers which could be used for extracting clean water. Accordingly, alternative approaches such as reverse osmosis treatment plants had been installed, but with limited success due to their high operation and maintenance costs. This remains a significant challenge for NJP and the project has started to collaborate with national, divisional and district DPHE departments and NGOs, with the intention of developing coordinated action plans from FY 2019. As part of this process, NJP has already initiated monthly coordination meetings with the Coastal Water Alliance based in Khulna. NJP also has plans to further explore options including adding Arsenic Iron Removal Plants (AIRP) to existing tubewells which have been identified as arsenic contaminated, and Sky Hydrants to treat pond surface water. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 24 MCHN Strategy as per Implementation Guideline NJP targets 23,600 PLW and 26,506 CU2 with its MCHN interventions. The first intermediate outcome is to increase the adoption of MCHN practices through the following activities (as per the NJP MCHN Program Implementation Guide):  Facilitating Growth Monitoring and Promotion (GMP) monthly sessions, for mothers with CU2, in Extended Program for Immunization (EPI) centers and Community Clinics. One Community Nutrition Facilitator (CNF), one WASH Promoter and one EPI Volunteer lead each GMP group, to facilitate sessions and to conduct home visits. 960 EPI centers and 113 Community Clinics are targeted under the project.  Providing Micro-Nutrient Powder (MNP) to CU2. MNPs are distributed to caregivers at Community Clinics and EPI centers through the monthly GMP sessions.  Implementing the mHealth5 pilot project, first in Dacope Upazila, and later to expand across all NJP Upazilas.  Providing Conditional Cash Transfer (CCT) to PLW, where mothers receive Tk 2,200 per month, for a total of 15 months.  Conducting Behavior Change and Communication (BCC) outreach activities on nutrition (MAMA messages and demonstrations), through SBCC-MCHN groups facilitated by CNFs, and participating in broader awareness campaigns. The second intermediate outcome is to improve nutrition and health related GoB services via:  Capacity building for government health service providers through a 5-day Training of Trainers (ToT) of existing Ministry of Family Health and Welfare (MoFHW) staff, and 3-day training of MoFHW front-line staff.  Supporting coordination for nutrition at Community Clinics, through quarterly meetings between the Directorate General of Health Services (DGHS) and the Directorate General of Family Planning (DGFP).  Reactivating Community Clinics, where each clinic has one Community Group and three Community Support Groups (CSG). Progress to date The MTE found that the project has strengthened Community Clinics and developed better linkages with Upazila Health Complexes by building the capacities of their health staff through targeted training. When interviewing health staff at Upazila Health Complexes and at Community Clinics, the staff were appreciative of these trainings, and stated that their capacity to treat patients had improved as a result. This was confirmed by patients who were interviewed by the MTE team when they were visiting these centers. Visiting patients shared that these centers had improved their services after receiving training from NJP, and community members as a result had increased their visits to these sites. In addition to skills training, NJP had also provided equipment such as 5 The mHealth pilot project is designed to build the capacity of government frontline health staff to capture data and report in real-time on issues related to MCHN and GMP NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 25 weighing and height scales, basic medical apparatus such as stethoscopes, and furniture including chairs and tables, all of which aided these centers to conduct their services more efficiently. GMP sessions were highly regarded by attending mothers. 35,638 CU2 were participating in GMP sessions, exceeding its target of 26,506, which is a good proxy indicator showing how valued these sessions were to mothers and caretakers. The MTE team, when interviewing mothers attending GMP and SBCC-MCHN sessions, observed that the project has made progress in terms of raising awareness on appropriate mother and child care, especially during pregnancy, and in taking care of CU2. From interviews with mothers, the MTE team found that they were appreciative of the growth monitoring activities in GMP sessions where U2 children were weighed and height measurements were taken, and screening was done for MAM and SAM children. It is to be noted that the MTE team did not encounter any MAM or SAM children in the sample sites that were visited, but were told by mothers that CNFs were prepared to use MUAC measurements if such cases were found. Mothers also shared that they appreciated the learning that took place from CNFs sharing essential messages on mother and child care, and in the discussions that took place between mothers on these issues. During these sessions, CU2 were provided MNP – however, the disposal of these packets was not being carefully monitored that could negatively impact on the environment. The CCT component was highly appreciated by all recipients who were participants of this intervention. The MTE observed that mothers were very pleased to be receiving Tk 2,200 per month, over a period of 15 months. Most mothers were using this to purchase supplementary food – however, the MTE found evidence that a portion of the money was being used to purchase non-food commodities which indicated that mothers needed better awareness raising of what the prime intention of the CCT was. This was also an indication that the poor status of these households forced mothers to make difficult choices where a portion of the money was expended for other essential household needs. The MTE found that the SBCC-MCHN sessions were being proficiently conducted by CNFs. On interviewing mothers and caretakers attending SBCC-MCHN sessions, the MTE found them to be highly appreciative of these sessions, where they expressed their satisfaction in the improved child and mother care practices that they learned in these sessions. They also appreciated the safe space for them to freely converse with CNFs and peer mothers/caretakers on issues they were facing, and learning how different people were approaching and solving their challenges. The follow-on house visits by CNFs and SBCC Promoters was also highly appreciated. In the area of improving GoB services, NJP has made progress where the MTE observed that communities were significantly more aware of the services provided by Community Clinics. As mentioned above, during visits to clinics, the evaluators came across patients visiting these centers for various services, and when interviewed, they all shared a positive reflection of the support these clinics provided. These were affirmative indications that communities were being effectively mobilized, and that support structures were being responsive to the needs of their clients. However, the project has made less progress on the number of government health staff receiving nutrition related professional training, having trained 2,493 of its targeted 2,982 staff, as reported in NJP’s FY 2017 Quarter 4 Report. A 5-day ToT was provided to Medical Officers, Health Inspectors and Statisticians; and a 3-day Training of Facilitators (ToF) was provided to Sub￾Assistant Community Medical Officers, Family Welfare Volunteers, Assistant Health Inspectors, Health Assistants, and Family Welfare Assistants (FWA). The ToTs were conducted by IPHN staff NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 26 and supported by the NJP MCHN Manager; and the ToFs were conducted by Upazila Medical Officers, Health Inspectors and Statisticians, with the support of NJP MCHN Officers. The training followed the IPHN Basic Nutrition Guide and covered the topics of food and nutrition, micronutrients, adolescent nutrition, nutrition for PLW, Infant Young Child Feeding (IYCF), reasons for malnutrition, GMP, safe food, direct nutrition interventions, disaster response, BCC and counselling, and malnutrition referral mechanisms. GENDER Strategy as per Implementation Guideline The gender strategy of NJP compliments the GoB’s National Women Development 2011, and the National Plan of Action for the Advancement of Women 2011-2015. NJP focuses on the following major interventions:  Awareness raising by training project staff and conducting orientation sessions for NJP participants.  Gender integration in ELT through the addition of gender modules, material, case studies and guidelines in course material.  Skills development and capacity building by ensuring women’s participation in need-based, market-oriented agricultural and non-agricultural skills development training.  Resource generation and savings formation for investment through the development of self￾help savings groups.  Market linkages and value-chains by establishing forward and backward linkages with organized markets, where women receive fair price and face reduced exploitation by middle-men.  Institution building to increase the participation of women in community activities and committees including Village Development Committees (VDC), Child Protection Committees (CPC), School Management Committees (SMC), WATSAN Committees, Union Disaster Management Committees (UDMC), and Union Parishad Standing Committees (UPSC) of Family Conflict Resolution, Women and Children Welfare.  Male engagement through participation in group educational activities where men are encouraged to reflect on rigid gender norms, examine their personal attitudes and beliefs, and question traditional ideas on household decision-making, division of labor, caring for children and sharing household tasks.  Reduction of adolescent marriages through its “marriage not before 18” campaign.  Life Skill Based Education (LSBE) for adolescent girls and boys.  Leadership training for women and youth. Progress to date NJP has been careful to include gender modules in its various training materials, including a dedicated session on Gender-Based-Violence (GBV) for the MEG group sessions, and including modules in the LSBE curriculum on Sexual Reproductive Health and Rights (SRHR), trafficking and sexual harassment. Through these various activities, the MTE team observed that women in the sample areas were confident in relaying their views, frequently debating with male counterparts sitting in VDC group discussions, and in IDIs with women where men had no objections to have NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 27 their female counterparts discussing in private with the evaluators. When interviewing men who were members of MEGs, the evaluators found them to be supportive of women taking more leading roles within the community, and in taking a greater role in decision-making within their own homes. The MTE found that the project has commendably taken forward the ELT component, where the universal response from female respondents was that they were very proud of what they had learned in the ELT sessions. This was evidenced when women who were previously illiterate, proudly wanted to show their newly acquired reading and writing skills to the evaluators after having attended the ELT sessions. The project has also made good progress in developing women’s savings and lending groups, as a part of the Promotional Graduation component. From interviews held with women members, the MTE found that women in these groups were confident, and actively participating in contributing to and management of the savings account. Although formal savings accounts have not as yet been established, NJP is now exploring the option of opening savings account as part of the bKash package which is currently being used to transfer funds under the CCT component. From interviews with CPCs, the MTE team were impressed to hear various examples of how the CPC have taken up the issue of early marriage, where four examples were cited of how CPCs stopped early marriages from taking place. Agriculture and Alternative Livelihoods – Alternative Income Generating Activity Strategy as per Implementation Guideline The project aims to reach a total of 18,000 Ultra Poor and Poor participants in this component, of which 65 percent or more will be female. According to the NJP Project Implementation Guideline, AIGA participants will undergo the following:  A 6-month ELT course. These participants are considered to be semi-literate, and hence requiring three fewer months (i.e. a total of 6-months) of training compared to the 9-months ELT training conducted with Promotional Graduation participants who are considered to be mostly illiterate.  During the 6-month ELT training, Winrock supports participants to explore and select AIGAs, based on their feasibility and profitability. Potential AIGAs include self-employment as well as higher-skilled formal employment.  Field staff are provided an intensive 2-day training on the concept of AIGA, different implementation methodologies, and training on “Field Guide to Integration of Poor and Extreme Poor into Markets.”  Of the 18,000 participants, only 3,200 participants are eligible to receive a one-time Tk 1,000 start-up capital for their selected AIGA, to enable them to purchase inputs for their selected business plans. The remaining 14,800 participants will only benefit from the training provided during the 6-month period.  Developing Public-Private-Partnerships to increase employment opportunities for NJP beneficiaries. Proposed partnerships include Pebble Child, Prokritee, PRAN Agro, ACI Agribusiness, Paragon Group, Rahman Renewable Energy Company, and Bangladesh Poultry Industries Coordination Committee. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 28 Progress to date The MTE found that AIGA participants attending the ELT were very satisfied with what they had learned in these sessions. The evaluators were impressed to observe how motivated women were where previously illiterate women were keen to demonstrate to the evaluators their newly acquired reading and writing skills. 14,898 participants, of the targeted 18,000 participants, have already completed the 6-month ELT. Although the MTE team did not have the opportunity to directly observe any AIGA trainings, from interviews held with AIGA participants, the evaluators found that they were engaging in new trades. It was a common trend to observe that women participating in these activities felt empowered and were eagerly expressing how their overall status within their household had been elevated, where the majority of women interviewed shared that were now earning an additional income of between Tk 800–1,200 per month. This was not insignificant considering these participants belonged to the poorest quintile. However, there was one exception who shared that she earned no income at all, even after 6-months of completing her training. To date, 16,994 of the 18,000 participants having completed training on AIGA techniques. Project data listed the top three AIGAs selected by females as hand embroidery, turkey rearing, and paper￾box making. Among male participants, the most preferred AIGAs were bamboo product making, followed by small grocery stores and nursery production. NJP, to aid in this process, contracted Change Maker to conduct a comprehensive market assessment to identify potential trades that participants can engage in, where a total of 26 trades were identified. Agriculture and Alternative Livelihoods – Agriculture Strategy as per Implementation Guideline The Agriculture component is implemented by Winrock, reaching a total of 20,000 Producer Group participants, divided into 12,000 participants (minimum 60 percent female) as Homestead Producers and 8,000 participants (minimum 40 percent female) as Value-Chain Producers. 75 percent of beneficiaries from the Homestead Producer group are trained on vegetable production techniques, and the remaining 25 percent in aquaculture techniques. The goal of NJP is to ensure that these subsistence farmers are able to produce additional nutritious food to contribute to the overall balanced diet within these households. The Value-Chain component targets the more resourceful participants6 who possess sufficient resources to actively engage in commercial production activities. As per the NJP Agriculture Program Implementation Guideline, the following activities are associated with this component:  Women friendly homestead production options such as livestock/poultry, homestead production and aquaculture.  6-month ELT.  150 Lead Farmers (more than 30 percent to be female), each responsible to develop the capacity of between 180-200 farmers from within, and adjoining communities.  6-day training of Lead Farmers on integrated farming practices (vegetable production, fish cultivation and poultry rearing). 6 Refer to NJP Agriculture and Alternative Livelihoods Program Implementation Guideline for definition of targeting criteria NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 29  Lead Farmers to establish demonstration plots (using Climate-Smart-Agriculture (CSA)7 techniques), as practical training sites for their Producer Groups.  Lead Farmers provided two seasons worth of seed annually (valued at Tk 700 or approximately US$ 9), over the LOA, to establish demonstration plots.  25 percent Lead Farmers expected to organize Farmer Field Days each year, together with the GoB and the private sector; and conduct pre-season planning meetings with their Producer Groups and input/output market actors.  Develop and operationalize 20 (previously 75) Collection Points, comprised of members from 8-10 surrounding Producer Groups.  Producer Group participants provided 3-days of intensive training on improved production technology, CSA, NJP Safer Use Action Policy (SUAP), poultry rearing and aquaculture techniques.  Homestead Producers receive two seasons worth of seeds (valued at Tk 700 or approximately US$ 9) only for the first year to help establish their homestead gardens. Value-Chain Producers involved in bitter-gourd, bottle-gourd and brinjal value-chains receive seeds over two seasons, water-melon Producers receive seeds for one season, GIFT Producers receive fingerlings, and Poultry Producers receive. The monetary value per participant, for both Homestead and Value￾Chains, of the inputs provided equated to Tk 700 (US$ 9) per participant.  Local Service Providers (LSP) developed in Agriculture, Livestock/Poultry, and in Aquaculture techniques through a 3-day training course for agriculture and aquaculture input suppliers and a 5-day training for animal health service providers. LSPs are then connected to Homestead and Value-Chain Producers. Progress to date NJP has trained 124 Lead Farmers (54 female and 70 male), who reach 11,634 Homestead Producers (11,062 female and 572 male) in 521 groups. In the Value-Chain component, NJP has reached 6,034 Value-Chain Producers (5,680 female and 354 male) in 305 groups, where each participant has received a one-time capital of Tk 700 (US$ 9) to purchase inputs. All 124 Lead Farmers have established demonstration plots. To develop demonstration plots, each Lead Farmer was provided inputs including a signboard, spray machine, hand gloves, boots, protective glasses, apron, poultry shed, fencing net, bamboo, rope, 10 chickens and one rooster, fingerlings and fish feed, and agricultural seed, all of which amounted to Tk 18,000 (or approximately US$ 225) per Lead Farmer. When interviewing Lead Farmers, the MTE team did not consider that one Lead Farmer being responsible for up to 200 Homestead or Value-Chain Producers to be viable. This was an overly ambitious approach, and discussions with Lead Farmers revealed that they did not have the time, interest, or capacity to devote themselves to training such large numbers of other farmers. Several Lead Farmers interviewed were also found to be significant distances away from their Homestead Producer and Value-Chain Producer groups, which raised concerns as to how they could viably train their groups from such large distances away. This observation became more apparent when visiting the homes of Homestead Producers where the MTE team came across small patches of 7 FAO defines CSA as “agriculture that sustainably increases productivity, enhances resilience (adaptation), reduces/removes green-house gases (mitigation) where possible, and enhances achievement of national food security and development goals,” NJP Program Implementation Guideline NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 30 gardens, mostly bare, indicating that not much progress had been made in developing their capacities in agricultural production techniques. The MTE team did not come across any Value-Chain Producers in their sample sets, and hence lacked direct observations of how Value-Chain Producers were progressing. However, from discussions with project staff and Lead Farmers, it appeared that for the Value-Chain Producer groups in which Lead Farmers were also members, these may be more sustainable due to the vested interest of the Lead Farmers to ensure that optimum profits were generated from these collectives from production to markets. NJP MIS data showed that all 54 female and 70 male Lead Farmers were practicing two or more CSA techniques. The progress was less evident for Producer Groups, where data indicated that 9,955 female-led homestead gardens had been established, but only 1,106 of 11,062 female (10 percent) Homestead Producers were practicing two or more improved CSA techniques. A further 514 male-led homestead gardens had been established, but only 55 of 572 male (10 percent) Homestead Producers had adopted two or more CSA techniques. The progress for Value-Chain Producers was similarly not much better where 300 of 5,680 female (5 percent), and 18 of 354 male (5 percent) were practicing two or more CSA techniques. This was indicative of the poor support and capacity building provided by Lead Farmers to their Producer Group members. NJP deliberately strategized to engage females in homestead production as these were less time intensive to manage, and were in direct proximity to their household thereby enabling women to tend their gardens without having to disrupt their other household activities. With respect to LSPs, a total of 215 (104 Agriculture, 30 Livestock / Poultry, and 81 Aquaculture) LSPs have been trained/mentored. From field interviews, the MTE came across several Livestock/Poultry LSPs, who were based in and around the communities they served. This was a positive development as these remote communities could now access these essential services. However, Agriculture and Aquaculture LSPs were harder to find. With regards to Collection Points, the project has reduced its target from 75 down to 20 as the project recognized that the previous target was overly-ambitious. 12 Collection Points have been formed to date, serving 48 Homestead Producer groups and 150 Value-Chain groups. At the start of the project, four commodities were considered as part of the value-chain, but later expanded to eight commodities. The MTE commends this flexibility as the project evolves to the opportunities of emerging markets within its target zones. Disaster Risk Reduction Strategy as per Implementation Guideline NJP has several interventions to develop the resilience of its target populations as follows:  Strengthening Disaster Management Committees (DMC) as per the GoB Standing Order on Disaster (SOD).  Training 50,000 people in disaster preparedness activities.  Mobilizing 620 VDCs to participate in Disaster Risk Reduction (DRR) activities.  Advocating for increased participation and contribution from public organizations in designing and implementing activities against the Comprehensive Disaster Management and Action Plan (CDMAP) and the Risk Reduction Action Plan (RRAP). NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 31  Advocating for increased participation of private organizations to provide resources to implement CDMAP and in joint responses.  Simulations on cyclone response within each Union. Progress to date A total of 1,127 volunteers have been trained to date on DRR activities, all 620 VDCs have been trained and are now committed to participate in DRR activities, and a total of 28,364 community people have been trained on how to respond when disasters strike. A 2-day training was provided to 2,026 Youth Volunteers on disaster preparedness and risk reduction issues, how to conduct CRAs and to prepare RRAPs, and on topics such as good governance and gender sensitive preparedness and response activities. A 1-day training was also given to 620 VDCs on similar training topics, after which they were actively linked to their respective DMCs and Youth Volunteer Groups. NJP has had several achievements in the area of DRR to date including its strengthening of the five UPSCs it works with in the Unions where the project operates, effectively linking with the CPP, and supporting the GoB SOD through its strengthening of DMCs, which is a critical element of the SOD. From interviews with Upazila and Union Parishad Chairmen, local influentials, and youth volunteers, the MTE found that Youth Groups were being formed and trained, and that VDCs were engaging in disaster related activities. This included conducting Community Risk Assessments (CRA), which was a participatory process where local communities assessed hazards, vulnerabilities and risks faced by their communities, followed by discussing possible risk reduction approaches to these challenges. These findings then led to the development of the RRAPs, which was the reporting template to frame and organize the information derived from the CRAs. The MTE found that NJP has coordination mechanisms with the Cyclone Preparedness Program (CPP) where joint simulation exercises have been performed. Upazila and Union Parishad government representatives were particularly appreciative of the partnership between NJP and five of the UPSCs, as this directly contributed to the GoB’s DRR strategy and SOD with activities such as developing the capacity of DMCs, developing the capacity of communities and local government DRR offices to conduct CRAs and develop RRAPs, and contributing to framing Union Parishad action plans to support DRR activities at the community level. The MTE found that the preparation work was going well, however, given that there have not been any major disasters since the inception of NJP, the MTE could not evaluate how effectively these interventions have built the resilience of these communities to withstand and adapt to major environmental shocks. Good governance and social accountability Strategy as per Implementation Guideline NJP’s Good Governance and Social Accountability Program Implementation Guideline employs an evidence-based social accountability approach, utilizing Citizen Voice and Action (CVA), alongside capacity building to community members that are representatives on the UPSCs. The committees will be the key agents of change to ensure social accountability in the working areas throughout the project lifetime. The major activities under this intervention include: NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 32  Conducting 117 Scorecard sessions and interface meetings in Community Clinics.  Conducting 40 Scorecard sessions with UPSCs and VDCs.  Facilitating reactivation of 622 VDCs, where VDCs will be the center of community development planning and implementation.  Assisting VDCs develop Community Development Plans.  Facilitating linkages between citizens and Ward, Union and Upazila governance structures as part of the CVA approach in 117 Community Clinics, 40 Union WATSAN Committees and 40 Union Agricultural Service Units.  Using local media, disseminate information on the availability of government services from Community Clinics, Union Agriculture Units and Union Parishads. A total of 152 media spots will be aired over the LOA (one message to be aired over a period of 40 weeks). Progress to Date The MTE found the CVA approach to have played a major role in mobilizing communities, and in engaging government support structures to be more responsive to the needs of communities. Evidence of this was observed as the MTE found Community Clinics now to be providing services to the local communities, where the evaluators came across a stream of patients visiting these clinics. When interviewed, patients favorably reflected on these centers saying that the clinics had become more helpful and supportive after project intervention. The CVA approach has been unrolled across all 40 NJP Unions, prompting greater participation of communities to identify their own problems, and for Village Development Action plans to be formed. These plans were taken up to Union and Upazila meetings for both technical and budgetary support. As a result of this effort, 39 Union Parishads have now included budgets and resources to support Village Development plans. These budgets include the construction/renovation of PSFs, DTWs, rain water harvesting systems, access roads, Community Clinics, and construction of latrines for extreme poor households. Annex F contains a copy of the action plans of Ratanpur Union showing the list of activities that have been included influenced by their RRAPs. Union Parishads will be held accountable to these commitments through two primary platforms. The first is at the bi-monthly UPSC meetings, chaired by the Union Parishad Chairman, and attended by all 13 UPSCs, as well as community representatives from VDCs. In these meetings each UPSC are required to provide an update of progress against their respective components, and how the Union Parishad budget is being utilized against commitments made. The second platform is the quarterly Union Parishad and Ward Shava meetings with VDC leaders, where communities are updated on the utilization of budgets against planned activities. The Union Parishad Act-2009 formalized the formation of 13 Standing Committees in each Union Parishad, where UPSCs represent communities in Union level decision making processes. To support this structure, NJP works with five standing committees: a) Education, Health and Family Planning, b) Agriculture Fisheries, Livestock and Economic Development, c) Sanitation, Water Supply and Sewerage, d) Social Welfare and Disaster Management, and e) Family Arbitration Women and Children Welfare. From interviews with various UPSC members, the MTE found that these local government structures appreciated the support from NJP to develop their management NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 33 and technical capacity. Respondents were eager that NJP also engages the remaining eight UPSCs, to ensure that all 13 Standing Committees remain coordinated in their development efforts At the policy level, the MTE found that the project was engaging with three critical branches of central government: (1) the Local Government Division (LGD) of the Ministry of Local Government, Rural Development and Cooperatives (MoLGRD&C), (2) the Department of Disaster Management (DDM) of the Ministry of Disaster Management and Relief (MoDMR), and (3) the Ministry of Women and Children Affairs (MoWCA). NJP has retained senior level staff to create and reinforce inter-ministerial links and hold national level workshops, and to analyze existing policies and seek ways to improve inter-ministerial collaboration on planning and budgeting which are pro-poor. However, at a systemic level, since the start of the project in September 2015, progress under the Government Relations and Capacity Building portfolio has been slow. As a result, WVB took the decision, in January 2018, to rescind the Government Relations and Capacity Building component from WFP and take on the role itself. Since then, considerable effort has been invested by WVB to build a skilled team, including the creation and recruitment of the Head of Policy, Advocacy and Research; convening of a thematic study tour with senior level representatives from the MoDMR, DPHE, DGHS, and LGD; facilitating two policy advocacy workshops; and the development of a strategy along with two policy briefs. Overall Issues A key shortcoming that the MTE came across was the lack of a detailed Exit Strategy, developed based on the experiences of the project. At the time of project design, a summary Exit Strategy was attached as an annex to the proposal, however none except one NJP staff during the MTE was able to refer to this strategy, making apparent that project staff were not considering the longer￾term view of how the project would eventually exit and ensure interventions sustained. Further, the Exit Strategy developed at the start of the project was based on initial assumptions and had not been updated based on the experiences of the project after implementation started, and also lacked detail of how the project would responsibly exit out of communities. The MTE finds this to be problematic which will impact on the sustainability of interventions, platforms and processes that the project has introduced. Early signs of this issue were apparent to the MTE team when interviewing beneficiaries and government counterparts where the universal hope was that NJP would be extended and continue to provide support over the next several years. This was concerning to hear at only the mid-point of the project and is symptomatic that the project is not sufficiently addressing sustainability factors among beneficiaries and support structures. The project also lacked a strategy for secondary adoption. Not having a secondary adoption strategy puts the project at risk as this will invariably affect NJP’s ability to bring about a “population-based impact” which is a key requirement of FFP Title II projects. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 34 Key Evaluation Question 1.2. Implementation, management, communication, collaboration strengths / challenges Implementation: Although NJP faced a setback of over a year before active implementation of activities in the field, the project has managed to meet the majority of its “activity” targets, with the exception of WASH, by this mid-point. However, the quality of its interventions is of mixed results and is discussed in detail under Evaluation Objective 1.3. Activities promoted by NJP have been piloted or implemented by other projects in and around the same areas. These include activities from the Climate Resilient Ecosystem and Livelihoods (CREL) and Agro-Input Retailers Network (AIRN) projects, both funded by USAID, with the goal of increasing food availability and access, and diversifying livelihoods. Although these projects have now concluded, the MTE team observed evidence of activities still on-going, such as the Tower Garden which was piloted and introduced by CREL. The MTE team found it commendable that NJP is building on models from these projects. Accessibility of many of the target areas is a challenge, as NJP works in some of the remotest and inaccessible areas of Bangladesh. The MTE team experienced this first hand, where multiple modes of transport were required to reach communities including driving long distances by road, crossing waterways and rivers by local boats and ferries, travelling country roads on motorcycles or by local rickshaw vans, and walking long distances. One of the sample areas had to be dropped (Sutarkhali Union under Dacope Upazila) as this was a 4-hour boat ride either way. NJP manages these complexities through their local partners who already have a presence in these areas, where WV, Winrock and WFP conduct frequent field visits to support and monitor the activities of their national partners. However, some of the remoter areas lack the frequency of management and technical visits afforded to more accessible areas. WVB’s network of VDCs served as essential platforms to initiate development activities, and the consortium’s strong ties with local government bodies has helped accelerate the acceptance of the project by local power structures. One of the more significant challenges that NJP had gone through was the initial targeting of beneficiaries. NJP adopted an approach where front-line staff visited villages, conducted well-being analysis, followed by household visits to finally register beneficiaries from what they found to be the poorest households. Government counterparts were not included in the process. The project faced several issues with this approach. First, from interviews with NJP staff, the project recognized that the staff responsible to conduct the selection were not adequately trained and hence did not follow proper protocol in the targeting process. This coincides with feedback from communities, where the MTE evaluators encountered numerous complaints that staff only conducted well-being sessions with limited numbers of community people. Rather than investing further effort, lists were drawn up from these sessions, without the due process of follow-up household visits. This resulted in the potential exclusion of some of the poorest beneficiaries who were not present at the time when front-line staff were conducting the well-being analysis. The MTE team found several MTE evaluators travelling to sample villages; Koyra Upazila NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 35 examples of such people, who were clearly from the poorest category, but when interviewed, they shared they had been excluded due to not being present during the well-being analysis. The second issue that was evident was the interference of local power structures, primarily elected officials, who insisted that certain households be included, who would not have otherwise be registered. Third, not engaging government counterparts had led to disgruntlement and tension between local administrations and the project. The project has since identified targeting errors as a critical issue and taken steps to address this by re-initiating the targeting process to verify the validity of all participants. Given the ambitious results framework, the project was forced to juggle with many activities at the same time. Front-line staff were constantly under pressure to chase target numbers, which at times compromised on the quality of implementation. Even though NJP was claiming to have triggered an increase in farmers revenue, these were likely short-term as key systemic structures, such as Collection Points, Agriculture and Aquaculture LSPs, and resources and staffing of local Agriculture and Livestock departments, were not being strengthened. It was evident that the level of effort varies widely from village to village, with no clear strategy as to how these were determined. The MTE team concluded that the project was missing opportunities by not strategizing how activities could build from one another. For example, the MTE team found that although the SBCC-WASH messaging had only reached 27 percent of its target, the communities which had benefited from this activity demonstrated to the MTE team an improved knowledge and practice on WASH related activities (these are discussed at greater length under Evaluation Objective 1.3). However, the MTE team did not find any clear plan of how the SBCC-messaging would be scaled up, together with the provision of latrines which were essential to promote a healthy environment. As a project, NJP is serving between 5–20 percent of households in a given community (except for activities where a potable water source has been established impacting on a greater number of households). At mid-term, it is not clear how other households in the target communities, who were not participants of NJP, would have access to project interventions or benefits, such as the adoption of CSA techniques, AIGAs, and savings and lending groups, as there was no clear secondary adoption strategy. This will hamper NJP’s effort to have a “population-based” impact, thereby limiting the overall impact of the project. While NJP partners had dedicated front line staff, in most cases, the training of 3–5 days was too minimal for them to be fully prepared for their roles. This was evident from discussions that the MTE team had with various field staff, who shared that the training was not sufficient. Following this short training, front-line staff were entrusted as the primary interface to bring impact within communities. Refresher trainings have been planned, but as yet have not taken place as only one year of implementation has elapsed. The MTE team were especially concerned as they came across, on more than one occasion, gender insensitive comments being made by relatively senior field level staff. These experiences have been shared with the NJP Chief of Party (COP) for his follow￾on action. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 36 Management: The COP is overall accountable and responsible for NJP. Based at WVB national headquarters in Dhaka, the COP is supported by a management team comprised of the Technical Program Director, Finance Manager, M&E Manager, Supply Chain Manager, Knowledge Management and Communication Manager, Head of Policy, Advocacy and Research, and a part￾time Government Relations Adviser. The Technical Program Director is based at the WVB national headquarters in Dhaka and is in charge of all technical leads, whereas the Deputy Chief of Party (DCOP) is located at the Khulna office, and is in charge of all operational activities. The DCOP coordinates project activities with consortium partners and with government counterparts at both the Divisional and District levels. At the field level, NJP has sub-district offices at each of the four Upazilas it works in. Each of these offices are led by a Field Office Coordinator, who is responsible for field coordination and implementation. The Upazila field office also provides space for Technical Officers, the Finance and Administration team, and the Value-Chain team. Although NJP was locating its partner staff in one location in the effort to improve coordination, the MTE team found that this was not taking place as intended. Evaluators frequently came across staff who were unaware of other technical activities and plans and appeared siloed in reaching their own targets. Further, as each technical team had their own budget and targets, staff appeared to be “tunnel-visioned” in reaching their activity targets, but without the same rigor in the quality of implementation. The Lead Farmer intervention is one example highlighting this issue. Field staff were committed to selecting and training Lead Farmers as per plan. However, there was little attention to the follow-up of Lead Farmers to ensure that they were providing quality training and advice to the Homestead and Value-Chain Producers they were responsible for. NJP is governed by a steering committee at the national level, chaired by the Secretary of the Ministry of Disaster Management and Relief (MoDMR) and the NJP COP. Members of the steering committee include the Ministry of Local Government and Rural Development and Cooperatives (MoLGRD&C), Ministry of Finance (MoF), Ministry of Family Health and Welfare (MoFHW) and Ministry of Women and Child Affairs (MoWCA). USAID/Bangladesh, WFP and Winrock are also part of the steering committee. The steering committee provides strategic oversight and is a platform to strengthen inter-ministerial coordination from the national to the local levels and meets once annually. The MTE did not have the opportunity to observe this meeting as there were none scheduled at the time of the evaluation. However, meeting minutes of previous meetings were shown to the evaluators. Given the scale of NJP and its goal of strengthening and developing government systems, the MTE team considered only one meeting per year as inadequate. From the field visits, it was apparent that government departments including Agriculture, Livestock, Fisheries and Health were challenged with limited resources and capacity. These issues were directly linked to the central governance system, requiring support from the steering committee. Also, issues emerging from the field required discussion by the different ministerial partners to strengthen coordination and integration of strategies and activities. Accordingly, it is recommended that national steering committee meetings are held more frequently for these important discussions and coordination to take place. Decisions taken by the Steering Committee should be immediately relayed down to the District, Upazila and Union Parishad administrations to ensure all levels of government remain coordinated. It is recommended that NJP continues to work with the different government tiers, ensuring that strategic level decisions are discussed in regular coordination meetings, and that issues requiring national level support are relayed back to the National Steering committee for discussion and action. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 37 Executive leadership of WVB, WFP and Winrock provide oversight of NJP through the Strategic Oversight Committee, which is comprised of the Country Representatives/National Directors of these organizations. This committee meets periodically and undertakes field visits to strengthen coordination within the consortium. From discussions with the WVB Executive Leadership, and the Technical Director responsible for the country technical team, it was apparent that although NJP is considered to be one of the priority projects of the mission, there was little coordination between NJP and other teams within the country office. This is a concern as NJP is not taking full opportunity of other technical strengths existing in the country office, and especially not taking advantage of a coordinated mission response on policy and influencing issues that NJP is dealing with. On discussions with the leadership of WFP, it was made apparent to the MTE team that as a United Nations (UN) organization, WFP does not come under the same criteria as an NGO. This is an issue that transcends NJP, as the UN does not fall under the normal Regulation 11 requirements. There were obvious frustrations between WVB and WFP, where WVB as prime have been concerned with the progress made by WFP in the areas of government relations and policy issues – both of which have now been taken out of the WFP portfolio and absorbed directly by WVB. NJP staff turnover was high standing at over 36 percent. The remote and difficult areas in which NJP operates made it difficult for the project to recruit and retain qualified staff. Another unexpected reason for staff attrition has been the Rohingya crisis in the south-east of Bangladesh which has brought unprecedented attention to the area, with an influx of humanitarian agencies. Given the urgent need for the Rohingya crisis, openings for qualified professionals were numerous, together with attractive packages that NJP could not compete with. Staff turnover was also associated with high workloads as mentioned by numerous staff during interviews. Communication: NJP has developed a branding strategy that has been approved by USAID. Based on the branding plan that was shared by NJP, the branding of latrines was waived by USAID as these were culturally insensitive. However, shortcomings were evident where the MTE team visited five Community Clinics to find that none of the assets provided by the project were branded. This included weighing machines, height scales, chairs and tables. Similarly, water points were in most cases not adequately branded. The MTE team were nonetheless, impressed by the fact that every beneficiary and government counterpart were well aware that the funds were from the American People. Collaboration: NJP operates in a complex environment and several factors have an impact (positive or negative) on the project’s likelihood to achieve its planned objectives. Among the strengths that the MTE team noted, were the partners technical expertise in supporting farmers’ food security interventions, maternal and child health and nutrition, and interventions to diversify income sources and livelihoods. All partners – WVB, Winrock, WFP, Sushilan, Codec and Nabolok, were recognized for their expertise in working in these domains, at strategic as well as local levels, where national partners had a strong presence on the ground. VDC center; Dacope Upazila NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 38 The NJP management has taken steps to adopt an inclusive partnership approach where partners meet monthly to discuss plans, progress and challenges. From discussions with the various partners, the MTE team found that meetings were taking place. However, discussions with field staff surfaced coordination issues where different staff were not aware of each other’s activities, approaches and timeframes, which could lead to issues in the overall integrated approach that NJP adopts. A commendable approach taken by the project was its flexibility to host different partner staff within the WVB regional and local offices, which has assisted in the communication between partners. The MTE were also impressed by the strong relationships with government counterparts, clearly evidenced from the interviews held with government staff. This collaboration has been important for the rapid progress made by NJP in reaching its activity targets, which was evident from the IPTT, and the quarterly and annual reports. There is no direct food distribution in NJP, and commodities brought in were all monetized as a consortium involving the three DFSAs operating in Bangladesh (i.e. NJP, SHOUHARDO III and SAPLING). All three projects sold commodities to the GoB at a pre-determined fixed price, thereby minimizing the risks of market distortion, and the complications of an open-tendering process including the private sector. The three projects conducted regular meetings with their line ministry to manage issues as they arose, so that the monetization of commodities was performed efficiently. The commodity of the three projects was sold and transferred to the government, and the request for payment was initiated as a consortium. NJP is satisfied with the relationship with the government, and the three DFSA partners have been able to find common solutions to their problems as needed. Key Evaluation Question 1.3. Quality and efficiency in implementation and community acceptance WASH The MTE conducted five FGDs and nine IDIs with SBCC-WASH members, four FGDs and four IDIs with members of Water Point Management Committees, and IDIs with three WASH LSPs. Respondents revealed that salination of water and soil in the south-west of Bangladesh was the major challenge faced by the local population. Although options such as reverse-osmosis water purifiers were being used, these were limited in scale. Further, respondents shared that reverse￾osmosis plants were expensive to implement and maintain. The review team repeatedly came across government counterparts who were not in favor of these plants for the same reasons – i.e. they were expensive to maintain for which they did not have the budget. PSFs were gaining traction but had limited private sector engagement, and hence limited in its coverage. There was also a misguided belief that PSFs removed salinity from water, which was not accurate. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 39 With regards to sanitary latrines, NJP faced a challenging prospect due to the scarcity of land, making it difficult for the project to maintain the required environmental safeguards such as having latrines 10 meters away from water sources. This reality has contributed to the limited progress of NJP to install and rehabilitate sanitary latrines. NJP has taken steps to minimize this problem where it has requested approval from the FFP Bureau Environmental Officer (BEO) that this requirement should only be applicable for “potable water” sources, and not be applicable to all water sources. The MTE team also observed many of the latrines that were rehabilitated were not always of good quality. However, the MTE did find hygiene practices were being better observed in improved latrines, regardless of whether they were provided by NJP or other organizations, and found these to consist of hand washing areas, soap, sandals and water. From interviews with beneficiaries and Water Management Committee members, the evaluators found that there was little progress to motivate communities to invest their own resources to scale-up the implementation of hygienic latrines across the community. Although community members understood the importance of these hygienic latrines, these areas continued to suffer from the “dependency” attitude where expectations were that NJP, or other development agencies, would provide these latrines for free. This attitude will invariably impact on the both the sustainability and the scale-up of these structures as poor households continue to be reluctant to invest in the construction and maintenance of hygienic latrines. The MTE team did not come across any WASH Entrepreneurs during the field data collection. Although the MTE team did not observe any “on-going” latrine rehabilitation or construction work, households interviewed shared that youth were engaging in this work by providing labor to clean pits, dig new pits, and engage in the repair and construction of latrines. NJP’s strategy included capacity enhancement of WASH Entrepreneurs where to date it has trained 242 of its targeted 440 Entrepreneurs. The project shared that it had been difficult to find and motivate people to become WASH Entrepreneurs as there continues to be a belief that this will not generate sufficient income as a core profession. MCHN The MTE interviews revealed that government representatives were satisfied with the 5-day ToT training received, and that these were followed through with 3-day trainings to relevant front-line staff including CNFs, community volunteers and government staff. The evaluators did find that the Institute of Public Health and Nutrition (IPHN) had an agreement with NJP, however, there was little evidence of their active engagement in the field with capacity building activities. Random checks of growth charts were impressive, where GMP cards were completed showing growth rates against wasting and stunting thresholds. When observing GMP sessions, the evaluators found that CNFs were able to interpret results, and that they took the opportunity to promote mother and child care practices, and to discuss case specific counselling between normal, underweight (moderate and severe) cases. When talking to mothers, they were appreciative of these sessions and shared with the evaluators that given the importance of these issues, they had adopted many Rehabilitated latrine in Amadi Union; Koyra Upazila NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 40 of these improved practices. The mothers were pleased that after the GMP sessions, within the next 1-2 days, SBCC-MCHN sessions were held within their communities, together with household visits by CNFs and MCHN Promoters. The evaluators found that NJP commendably approached GMP interventions by building on the strengths of local volunteers and CNFs. However, project specialists felt that screening and measuring at GMP sessions needed to be improved, together with improvements in follow-on counselling sessions. Mothers shared that they were receiving MNP sachets during GMP sessions, however, there were concerns regarding the disposal of empty packets. Although the MCHN Program Implementation Guideline includes the incineration of these sachets, the MTE team found that mothers were unaware of this which could lead to environmental issues as sachets were randomly being thrown away. The review team attended three SBCC-MCHN sessions and from these limited observations found these to be an effective platform to sensitize mothers on proper child care practices, and on health, hygiene and nutrition. The courtyard sessions observed by the team were found to be engaging, and the CNFs were found to be effectively demonstrating various issues on child feeding practices. One such demonstration included the use of a dummy breast, where the female CNF placed the model breast over her own chest and continued to demonstrate how infants should be breastfed. Although this was an effective means of relaying messages to mothers, the MTE team was concerned that such vivid demonstrations needed to be conducted with caution, especially in known conservative areas where men may react negatively to these demonstrations. Risk levels were raised considering SBCC-MCHN sessions were usually conducted in open, communal areas, where bystanders frequently came to observe what was taking place. At the end of one of these sessions, an MTE evaluator was approached by a community elder saying that the community appreciated NJP’s activities – however, advised that women sessions should be secluded as religious elders may not appreciate such free mixing of women and men. One area that appeared to need attention was the concept of exclusive breastfeeding as the MTE evaluators found several mothers who continued to believe that providing water, together with breastmilk, to babies was essential. Mothers also shared that several of them were not able to attend courtyard sessions due to workload. The MTE team was unable to determine the extent of this problem due to a lack of data. The MTE team were repeatedly informed from all levels of staff related to the MCHN component, that workloads were particularly high, leading to burn-out. Added to this, the opportunities for employment with alternative agencies offering higher salaries, or locations which were better suited for different individuals, had led to staff opting for opportunities elsewhere. The MTE team was unable to determine what the turnover rate was for MCHN staff, however, the project overall had a turnover rate of 36.7 percent which appeared to be unusually high. Hence the issue of high SBCC-MCHN session NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 41 workload and stress may not be isolated to only MCHN staff. This is a complex issue for NJP as securing qualified staff to be based in the remote areas in which the project operates is not easy. NJP’s strategy of revitalizing Community Clinics was in line with the GoB strategy, where the government 3rd Health Sector Plan (2009-2011) mandates the revitalization of these clinics, and the current 4th Health Sector Plan (2016-2021) continues to support this effort with an independent Line Director and Operational Plan. The 4th Sector Plan elevates the role of Community Groups and Community Support Groups (CSG) for improved health outcomes, especially in rural Bangladesh. The project to date has attended 128 bi-monthly meetings with CSGs, and monthly Community Groups to support the mobilization of local resources to strengthen Community Clinics. Within the sample frames, the MTE found that the Unions of Choto Antihara and Dokhin Bitkashi (under Koyra Upazila) were suffering from a lack of Community Clinics. However, after discussions with the Union Parishad, it was found that budgets have been allocated for the construction of Community Clinics over the coming year in these two Unions. From discussions with field staff, it was evident that the CVA approach contributed in this achievement to effectively raise the voices of the community to demand the establishment of these clinics. The MTE found that NJP was effectively supporting the existing referral mechanism, where CNFs referred Moderately-Acute Malnutrition (MAM) children to the Community Health Care Project (CHCP) within the Community Clinic using a referral slip, who then, if required, were referred to the Union Health Complex at the District hospital. However, respondents shared that issues still remained with regards to Severely-Acute Malnutrition (SAM) cases, primarily due to the poor service at the Upazila Health Complex where such cases were referred to. Despite the presence of a Nutrition corner, interviewed mothers complained that these facilities lacked trained medical professionals to provide the treatment needed. Further, mothers were reluctant to leave their other children alone at home, and hence preferred not to stay in the Upazila Health Complex. These were weaknesses already identified by the project, and measures were being taken by NJP to address these issues. The NJP Annual Results Report 2017 indicated that CVA sessions were conducted with government service providers to identify the current status of services offered in Community Clinics and setting monitoring standards to continue to hold these clinics accountable. The Community Score Card approach was found to be an effective activity, where the project facilitated four FGDs across 118 Community Clinics, including service users and local community members in the process. The CCT intervention by far appeared to be the most popular activity which beneficiaries, government counterparts and partners consistently highlighted. This was not surprising considering that CCT beneficiaries received Tk 2,200 per month, for a period of 15 months. At the onset of the targeting process, there were inevitable delays due to identifying eligible beneficiaries and registering them in bKash – however, it appeared from discussions with GMP mothers, these delays have now been largely overcome. The MTE team conducted three FGDs with CCT groups, and four IDIs with CCT mothers, where 20 percent of women shared that they saved up to Tk 1,000 per month for asset building purposes and spent the remaining funds to purchase food. This indicated that mothers were not fully aware that the purpose of the cash was to ensure adequate nutrition for them and their CU2, and accordingly the full Tk 2,200 was supposed to be spent in purchasing food. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 42 NJP’s guidelines required that mothers have mobile SIM cards in their own names to be eligible to be registered for the CCT intervention. While the intention of this approach was to elevate women’s status by ensuring women received money directly through the use of her own phone, the project needed to take a pragmatic approach considering the complex requirements imposed by the government to obtain a SIM card. Any citizen wishing to procure a SIM card was required to present their National Identification Card (NID). The NID card is a relatively new undertaking, where considerable numbers of the population still do not have these – a problem which is even more pronounced in the rural, remote areas where NJP operates. Considering this reality, there was a risk of eligible mothers to be excluded. To overcome this risk, NJP commendably sought approval from USAID for the project to be able to use SIM cards of close family members of eligible mothers, should they not have their own SIM cards. Carbon Monoxide Exposure Limiter (COEL) bracelets have also played an important role in promoting messages. Women interviewed wearing COEL bracelets were appreciative of this technology and shared that the recorded messages were clear and easy to understand. The MTE team encourages NJP’s piloting of COEL bracelets, for women who do not own phones, where voice messages can be played on different MCHN and Mobile Alliance and Maternal Action (MAMA) related topics. COEL bracelets may be an effective alternative for women who do not have access to mobile phones. The CCT cash transfer amounts varied significantly with the GoB sponsored CCT intervention, where NJP provided Tk 2,200 compared to the government’s Tk 500 per month to eligible beneficiaries. The Tk 2,200 amount was derived from WFP’s Transfer Modalities Research Initiative study, which determined this sum of money would enable PLW to consume a nutritionally adequate diet. The two groups only received resources from either the GoB or the NJP CCT intervention and it was beyond the scope of the evaluation to determine how effective the GoB program was in comparison to the NJP program. However, the MTE team consistently came across tension where mothers sponsored by the GoB were unhappy that they too were not receiving similar sums of money as provided by NJP. NJP needs to be careful that the higher transfers provided by the project does not create issues for the GoB sponsored CCT intervention, which are implemented nationwide. Gender The MTE team found that women empowerment has been one of the more successful components of the project to date. This was evident from the numerous interviews held with women, both in mixed and homogenous groups, as well as from IDIs. The MTE team were impressed with the confidence women demonstrated in these meetings, frequently and comfortably arguing with their male counterparts in open forums. The team was also impressed with the acceptance from the male population of women taking more central roles in different community level discussions and groups. IDIs revealed a similar pattern where women were confident, and men appreciative of how their spouses were developing and contributing to their households. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 43 NJP gender teams have invested time to ensure that MEGs were formed and trained, and that recommendations from the Gender Analysis report were being considered, including the utilization of local resources for reducing child marriage. When interviewing MEG participants, there was common agreement that women and men need to have more equitable distribution of household chores, and that household decisions should be taken equally. The MTE team observed that men appeared to be more sensitive to gender issues, and it was evident that women who were interviewed were confident and comfortable to be speaking on different programmatic issues. A module titled ‘Family Wellbeing and Prosperity’ has been developed through extensive gender research in NJP working areas for use in MEGs, consisting of 12 sessions in the module. The first four sessions were exclusively reserved for men, to acclimatize them to concepts which they may initially find difficult, and subsequent sessions were then mixed groups. The MTE team found this to be an effective approach, and observations from interviews with women indicated that they were feeling a greater sense of empowerment and confidence through these sessions. The MTE teams also observed positive signs of knowledge and attitudes from men, where they were accepting of women’s non-traditional roles in committees/groups with management responsibilities. The Family Arbitration and Women and Child’s Welfare Standing Committee, and the Upazila Women and Child Development Committee, both constitute NJP’s Child Protection Committee (CPC), where preventing early marriage was a priority for CPCs. The MTE team was impressed to learn of four examples of where CPCs had successfully halted such marriages. Although child marriage is one form of GBV, the MTE found that NJP has taken a broader approach to GBV where it has included a dedicated session on this topic as part of its MEG module, as well as including topics on SRHR, trafficking and sexual harassment in its LSBE sessions. To further strengthen this effort, the MTE team recommends NJP discuss this issue with the MoWCA and local government offices on how best to ensure support to survivors of Domestic Violence against Women. As part of the support network, it is important to mobilize local police stations, health facilities, legal aid organizations, to bring perpetrators to justice for criminal offences in accordance with the Domestic Violence (Prevention and Protection) Act 2010, and the recent court verdict of the High Court of Bangladesh.8 The MTE team found that the Affirmative Action approach was well accepted by communities, allowing women to participate in relevant committees and groups in connection to MCHN, WASH, DRR, Agriculture, AIGA and Promotional Graduation, all of which contributed to the implementation of the National Women Development Policy 2011. Affirmative Actions were promoting gender issues to influential committees and Village Development Committees (VDC) where NJP was already demonstrating evidence of impact on empowering women where the MTE team directly observed significant women representation in management committees such as VDCs. Progress in women empowerment was also evident through the confidence they demonstrated when discussing issues with MTE team members, many of which were in mixed groups and arguing their points with their male counterparts. Further, female Producer Group 8 The High Court of Bangladesh in a full verdict released in April 2018 directed police station officers-in-charge to record complaints about sexual assaults and rapes on women and children without any delay, The Daily Star, May 27, 2018 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 44 members were connected with Collection Points, and female beneficiaries engaged with the ELT were clearly very proud and appreciative of their improved capability to read and write. Importantly, connections were found to link women agency building with economic empowerment through the Promotional Graduation, AIGA and Agriculture components. LSBE for both school and community-based interventions were found to be working well and accepted by youth, school authorities and communities. Most of the school based LSBEs were conducted at multipurpose “school-cum-cyclone” shelters. Initially, one group was formed in each school. However, due to popular demand, the project is now considering expanding LSBEs to form two groups (20+20=40 students), consisting of students from class VII to IX, and school dropouts. With the support from the Upazila Education Officer, school based LSBE programs were connected with School Management Committees (SMC). Key functions of the SMC were to address violence against girls in school, and the prevention of child marriage among school going girls. From MTE interviews, the evaluators were impressed to observe the confidence of students to explain issues on Sexual Reproductive Health and standing up for SRHR rights. Teachers were also very supportive regarding SRHR matters in LSBE sessions. Gender and Sexual Reproductive Health Rights issues were extensively integrated in LSBE modules, complementing implementation of the Adolescent SRHR strategy. NJP has made progress with policy level engagement where in the last quarter of 2017, the project facilitated a visit of key members from the MoFHW, MoWCA, IPHN and MoLGR&D to NJP project sites to promote national level policy dialogue. A formal meeting was also convened with the Secretary of MoWCA to identify complementary activities related to gender and women empowerment, and engagement in the government safety net program. However, the following challenges remain:  Irregular participation of the poorest in MEG gender sessions as they could not afford the time to participate.  Seasonal migration of husbands/men impeded their continued participation in MEGs.  While Promotional Graduation and AIGA interventions provided operational scopes to reduce gender gaps in accessing labor markets, however, changes were slow and wage discrimination was prevalent in NJP working areas (e.g. where it was common practice for a female laborer to earn Tk 150 (approximately US$ 1.90) while a male laborer earned Tk 200 (approximately US$ 2.50) for half a day’s work).  Although the project had initiated meaningful conversations within MEGs and VDCs on gender matters, however, adopting affirmative action in favor of women required flexible family friendly hours and women’s SRHR needed to be considered.  While there were early changes in men and women’s behavior, this was less evident in institutional facilities such as the Union Health Complex, DPHE, MoWCA and the Union and Upazila Parishad.  Although data was being collected, there was no data analysis on GBV issues (Child Marriage, Family Violence, etc) conducted by the project. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 45 Agriculture and Alternative Livelihoods – AIGA From the primary data collection of the MTE, a total of two FGDs and four IDIs were conducted with AIGA participants. The team found that 100 percent of the participants in the sample frame were female, indicating that the project was reaching its target of over 65 percent female participants. Respondents interviewed were found to be engaging in new trades including Omani cap making, producing decoration pieces, and engaging in handicraft production. It was a common trend to observe that women participating in these activities felt empowered and were eagerly expressing how their overall status within their household had been elevated now that they were earning an income. However, one participant did share that she had not earned any income as yet, even after six months of having received the training. The MTE found that the majority of women respondents shared that they were now earning additional incomes of between Tk 800–1,200 per month, with the exception of one woman who shared that she had not earned any income even after 6-months of completing her training. When considering the responses from the majority of women who claimed to be now earning an income, this sum of Tk 800-1,200 per month was not insignificant considering these participants belonged to the poorest quintile, and this additional income certainly contributed to the overall well-being of their family. The MTE team was particularly impressed that these women could plan and choose how much time they could devote to the AIGA. This was important as women shared with the MTE team that they had control over their time, and that they planned their day so as not to become overly burdened. The Kaliganj Upazila has particularly seen as impressive expansion in the turkey rearing AIGA, where all sampled AIGA participants interviewed here shared that they were now involved in this trade and engaging with permanent local markets, and haats.9 Similarly, the Upazila and Union Livestock Officers interviewed were equally excited and praised the expansion of this commodity. However, the Upazila Livestock Officer did share that he was concerned of the increased risk of Avian Influenza due to the scale-up of turkey rearing, which was a major issue across Bangladesh over 2007 to 2012.10 NJP needs to carefully monitor this AIGA to ensure no such outbreak occurs from their programming efforts. Research also shows that expanding poultry farms increases the risk of Environmental Enteropathy, where bird droppings are more abundant, and children are more exposed to these. The project needs to be careful to minimize this risk as Environmental Enteropathy directly impacts on the well-being of a child. NJP was seen to be actively breaking into new markets, where they were participating in International Trade Shows in Khulna to connect with new buyers. The Khulna Trade show opened promising avenues, where NJP is now discussing a partnership with an Australian company, VIP Industries Limited, who have plans to establish a garments factory in the Mongla Export Processing Zone, and have expressed interest to employ up to 1,500 workers. NJP continues to explore this potential. The project is also in process to sign a Memorandum of Understanding (MoU) with the Tech Vocational Education Training Institute where 400 NJP beneficiaries will be trained to meet 9 Haat is the Bengali translation of a make-shift more localized market which operate in fixed days of the week 10 Bangladesh experienced outbreaks of highly pathogenic avian influenza (HPA1, subtype H5N1) in poultry over 2007 to 2012, where a total of 556 outbreaks were reported, leading to 608 humans being infected, and 359 deaths – Bangladesh Journal of Livestock, Volume 19, 2012 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 46 the 360 hours requirement to obtain government certificates. This can be of considerable value as an endorsement directly from the government for these graduates. The MTE team were informed that NJP has recently adopted vouchers as the mechanism for AIGA and Agriculture participants, who receive start-up capital, to purchase inputs from enlisted vendors. The MTE team considered this as an important initiative to boost the local economy, and for beneficiaries to establish relationships with input/output markets. Over the LOA, should local markets provide quality services, it is reasonable to assume that NJP participants will continue to strengthen their relationships with these vendors, leading to a more sustainable systemic structure. Value-Chain Producers were developed across eight commodities, where NJP had specific targets against each commodity value-chain. The MTE team considered this approach may be limiting participants to benefit from other emerging commodities that future market analysis may reveal to be profitable that are not part of the eight commodities already selected by the project. Agriculture and Alternative Livelihoods – Agriculture The MTE conducted a total of three FGDs (all women) and four IDIs (all women) with small-scale Homestead Producers. The review team did not come across any Value-Chain Producers from within the sampled villages randomly selected in the MTE. In addition, a total of nine Lead Farmers were interviewed, together with three Livestock/Poultry LSPs, one Agriculture LSP, and one Aquaculture LSP. From the sample frame, six of the nine Lead Farmers were found to have demonstration plots cultivating a variety of vegetables using different CSA practices. Tower Gardens were being promoted in these plots, where all demo plots visited had constructed one such garden. All Lead Farmers were able to show the MTE team the construction of improved poultry houses which were elevated and protected from water logging, and Lead Farmers shared that the mortality rates of poultry had significantly fallen since the start of NJP. Improved techniques of raising poultry pens, administering saline during the dry/hot season and warm rice during the winter, and improved vaccination all played a major role in this. Aquaculture benefits were less visible. Lead Farmers claimed that using improved techniques such as banana and bamboo trees floating in ponds as contributing to reducing salinity, which was a false belief indicating poor messaging from the project. Lead Farmers also stated that improved techniques such as layering and administration of proper feed was boosting fish production. Lead Farmers spoke highly of Collection Points, where the common trend was that Lead Farmers had connected with several buyers in the local market and bargained for the best prices. Wholesalers collected their produce from predetermined Collection Points at agreed times during the week. However, interviews with NJP project staff revealed that the project technical team considered Collection Points as a challenge requiring greater attention, with hopes that these will Demonstration Plot; Shyamnagar Upazila NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 47 become increasingly vibrant over time as farmers and traders recognize their true value. As for the pre-season planning, only two of the nine Lead Farmers interviewed mentioned they had conducted this. Livestock/Poultry LSPs were readily found during the field data collection. Positive reflections were forthcoming from these LSPs on the 5-day training provided to them by NJP and linking them to government extension services to receive technical support, and to access vaccines. Agriculture and Aquaculture LSPs were harder to find, and only one of each was interviewed during the field data collection. These LSPs, similar to the Livestock/Poultry LSPs, were appreciative of the training and expressed their confidence in the support they received from local government extension workers. When interviewing Homestead Producers and Lead Farmers, all of them were universally appreciative that they had now considerably reduced their exposure to consuming pesticide laden vegetables as their own productions were now boosted (using CSA environmentally friendly techniques). However, observations during the field visit was that CSA techniques were not being effectively implemented to have an impact, which was especially true for Homestead Producers who had only managed to nurture small homestead gardens, which were too minimal to sustain after project support ceases. One particular example stood out, where the MTE team were particularly impressed with an initiative taken by the VDCs in Kaliganj Upazila. Agricultural land was constantly under threat during the rainy season, where saline water from the river would inundate these areas, making it impossible for paddy production. This year, the surrounding VDCs organized and mobilized communities to construct a temporary embankment as a barrier to protect the agricultural land. As a result, these communities were able to harvest paddy from approximately 20 acres of land.11 The MTE team considers this as a best practice and should be replicated where feasible. The Lead Farmer approach was considered by the MTE team to be particularly challenging. Having one Lead Farmer be responsible to develop the capacity of up to 200 other farmers, with no incentives asides from receiving two rounds of seeds per annum, and minor inputs to boost their poultry and aquaculture production, seemed overly ambitious. All Lead Farmers interviewed expressed their desire to help their fellow farmers. However, when probed as to how much time they devoted to the various activities to work with their Producer Groups, mixed responses were received. Trainings and follow up monitoring of Lead Farmers by NJP was of questionable quality as none of the Lead Farmers interviewed conducted the Farmer Field Day, and only two of nine had conducted pre-season planning meetings. The MTE team had mixed observations of the demonstration plots where Lead Farmers were not practicing optimal CSA techniques. It was evident from the interviews that Lead Farmers mostly only engaged with their Producer Groups, when members would come for specific advice. The MTE team further discovered that the Lead Farmers interviewed in Dacope, were located far away from their Producer Groups. When the team asked to be taken to a Homestead Producer’s household, the Lead Farmers shared that the closest community was between 30 minutes to one hour away. Similar responses were forthcoming when speaking to Homestead Producers, who said that their Lead Farmer was similar distances away. This raised significant questions as to how effectively the demonstration plots, and 11 This success has been documented by NJP in the case study “Abdul Gani and VDCs Initiative” NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 48 the corresponding advice and support from Lead Farmers was actually impacting on Producer Group members in the Dacope region. From home visits to Homestead Producers, the MTE team observed small patches of gardens, most of them bare, indicating poor training and capacity building. Even if these gardens were producing, the areas were too minimal to generate any significant impact. These gardens may be meeting NJPs strategy to provide limited additional food to the household and to promote diet diversity – however, the MTE team considered these to be too insignificant to make any meaningful changes to the overall nutrition of the household. Also, these were not likely to sustain beyond the life of the project as Homestead Producers lacked the skills to maintain these when project support and monitoring ends. 62 of the 124 trained Lead Farmers, were themselves members of the Value-Chain Producer groups for which they were responsible for. These groups were more likely to sustain considering their success would directly benefit the responsible Lead Farmer thereby encouraging them to place greater effort in these groups. In general, Value-Chain Producer members were selected only if the community recognized them as commercial small holder farmers, which invariably meant these were wealthier farmers. This afforded them greater opportunity to invest in commercial farming, and when working as collectives, were able to generate greater volumes of commodities giving them more negotiation power to agree on optimum sale prices to markets. In addition, Producers involved in bitter-gourd, bottle-gourd and brinjal value-chains were provided seeds over two seasons, water-melon Producers were provided seeds for one season, fingerlings were provided to GIFT Producers, and poultry provided to Poultry Value-Chain Producers – all of which were valued at Tk 700 or US$ 9 per participant. NJP’s strategy to develop Homestead Producers were primarily to diversify their homestead production to contribute towards an improved nutritionally balanced diet within their own households. Homestead Producers received either seeds over two seasons, fingerlings or poultry, valued at Tk 700 per participant. Given that Homestead Producers were from the poorest category, the increases in homestead production could only be used to improve their own household consumption, and was not of a sufficient volume for them to participate in viable value￾chains. Although it was argued that these participants had little to no agricultural land to participate in value-chains, alternative options of leasing “khas”12 land could have been considered. It was also unclear whether they would be allowed to participate in value-chains, even if they had boosted their production, as NJP had capped their value-chain participants at 8,000, all of whom were selected from the more resourceful poor. This effectively barred the poorest from participating in anything more than subsistence farming. 12 Khas land are government owned fallow land, over which no individual has property rights NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 49 Lead Farmers interviewed were particularly proud of their Tower Gardens, which was piloted under the USAID funded CREL project. The cost of these gardens varied widely where respondents provided a range of Tk 1,000 (approximately US$ 13) to Tk 5,500 (approximately US$ 69) to construct a Tower Garden. Lead Farmers were willing to invest resources behind this, and to invest the labor, as they considered this to be an innovation which would help them continue to produce during water logging periods, when salinity was at its peak. The MTE team was unable to determine the impact of these gardens as there have not been any floods during the NJP period. Given the expense of these gardens, the MTE team did not observe these to be widely adopted by members of Producer Groups. Homestead Producers did appreciate the technology, and claimed they would be building these – however, given the expense it is unlikely there will be major uptake of this among the poorest participants. The deliberate water-logging by pumping saline river water into agricultural land is possibly one of the greatest challenges facing the south-west region of Bangladesh. During the field data collection, in addition to NJP beneficiaries, the MTE team interviewed a broader set of residents such as motorcycle taxi riders, and local shopkeepers, to better understand how the overall landscape had altered over the years. Residents universally said that life was better 10- 15 years ago, when they were able to rely on healthy productions of paddy and were able to cultivate a variety of crops in and around their homestead. Life had now changed, as the soil had lost its production capacity due to the expansion of gher farms, most of which were owned by large businesses and different power structures. The team observed the deliberate salination of good agricultural land in Koyra (see picture), which was being performed by a local power elite. Members of the MTE team brought up this issue when interviewing the Koyra Upazila Chairman, who admitted this was a major issue. Respondents shared that poor families had little option, once the land was water-logged with saline water, apart from selling the land to these power structures at low cost, or becoming part of their workforce. NJP targets 5,000 participants as part of the aquaculture intervention – yet, to date only 2,908 Homestead Producers and 400 Value-Chain Producers have been trained. Given the endemic and expanding challenge of salinity, poor households were unable to access sweet water resources which was a significant impediment to expand on this intervention. Tower Garden; Dacope Upazila Deliberate salination of agricultural land; Koyra Upazila NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 50 Disaster Risk Reduction The MTE team conducted FGDs with two Youth Groups, IDIs with two youth members, and over a dozen IDIs with Chairmen and local influentials. From these interviews, respondents shared that they were pleased with the training offered to Youth Groups, and that they considered VDCs to be actively engaging in disaster related initiatives. Community Risk Assessments (CRA) were being developed, together with Union Disaster Management Committees (UDMC), at both ward and Union levels. This was especially well described by the Bajua Union Parishad Chairman, who had actively participated in the development of the CRA and presented this to the Upazila level for final approval before proceeding with the development of the Risk Reduction Action Plan (RRAP) which detailed the findings from the CRA. Three training modules, in both Bengali and English, have been developed as capacity building tools for Disaster Management Committees (DMC), following the guidelines of the Standing Order on Disaster (SOD). NJP has made good links with the Cyclone Preparedness Program (CPP), where NJP Youth Groups, VDCs and UDMC members have conducted several simulation exercises as part of the CPP. To formalize the partnership, an MoU has been signed between WFP and CPP on September 25, 2017, to include NJP Youth Groups to become members of CPP committees in 28 Unions where NJP operates. Interviews with government and elected representatives, and with Union Parishad members, indicated that NJP’s partnership with five of the 13 Standing Committees at the Union level, were welcomed. The partnership was especially commended considering that this relationship directly contributed to the GoB’s Disaster Risk Reduction (DRR) strategy and SOD, through strengthening capacity of DMCs, building capacity of communities and local DRR offices to conduct CRAs and develop RRAPs, and to assist in the development of action plans at the Union Parishad level to support DRR activities in their constituent communities. Respondents did however mention that NJP should consider extending the partnership to the remaining eight Union Parishad Standing Committees (UPSC), considering the close collaboration required among all 13 UPSCs. These extended relationships do not necessarily have to be of an implementation nature, but could be an information sharing forum such that all 13 UPSCs remain coordinated on NJP activities. NJP has formed Youth Groups and recruited a total of 2,026 volunteers (1,137 male and 889 female). NJP Youth Group Volunteers had to satisfy several criteria including having to be existing volunteers under the CPP, possessing a minimum educational qualification of completing secondary school, aged between 18-35 years, and be a permanent resident of the Union. The women interviewed were motivated and expressed that they felt these positions were important for the overall well-being of the community, and that the trainings were of good quality. From these conversations it was evident that women volunteers felt empowered through their participation in Youth Groups and as DRR volunteers. In terms of challenges, from interviews held with government representatives, the MTE found that the RRAP and CDMAP processes were difficult to conduct due to a lack of resources. The MTE also found that the local level government structures on DRR were particularly weak, where at the Upazila level there were only two regular staff – one Project Implementation Officer (PIO) and one Office Assistant, who were supported by two project staff – a Sub-Assistant Engineer (SAE) and a Work Assistant. Similar structures were also mirrored at the District level. Of these few positions, government respondents shared that many of these positions were vacant further stressing an already lean structure. Additionally, the MTE found that although a DRR database was maintained at the Union level, there was no budget for staff to maintain this. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 51 Apart from capacity building needs, respondents shared that there was a major gap in funding where the acceleration funds of only Tk 1,000,000 per Union (approximately US$ 12,500) was inadequate to deal with the infrastructure needed to mitigate river erosion, water logging and salinity. With respect to private sector engagement, the MTE found little evidence of their engagement in DRR activities. Good Governance and Social Accountability The Union Parishad Act-2009 formalized the formation of 13 Standing Committees in each Union Parishad, where UPSCs represent communities in Union level decision making processes. To support this structure, NJP works with five standing committees: a) Education, Health and Family Planning, b) Agriculture Fisheries, Livestock and Economic Development, c) Sanitation, Water Supply and Sewerage, d) Social Welfare and Disaster Management, and e) Family Arbitration Women and Children Welfare. On interviewing government counterparts, it was evident to the MTE team that NJP’s partnership was valued, and that the project had brought positive changes in the frequency and timeliness of local government meetings, the documenting of meetings, and follow up on decisions. The project has initiated the CVA approach to mobilize communities and service providers, with similar principles to the Participation, Inclusion, Transparency and Accountability principles (PITA), adopted by USAID (2016). Project monitoring reports indicated that NJP has unrolled the CVA approach across 40 Unions, prompting greater participation of communities to identify their own village related problems, and to the development of Village Development Plans detailing how they intended to overcome these issues. Actions plans were being shared at Union and Upazila meetings, and different support services were planning how best to provide assistance in line with these plans, thereby strengthening the social accountability process. Local media played an important role in the CVA approach, widely sharing Village Development Plans and the commitments made by local government to address these issues. As a result, the MTE team found that 39 Union Parishads have now included budgets to support activities based on Village Development Plans. These budgets include the construction/renovation of PSFs, DTWs, rain water harvesting systems, access roads, Community Clinics, and construction of latrines for extreme poor households. Union Parishads will be held accountable to these commitments through two primary platforms. The first is at the bi-monthly UPSC meetings, chaired by the Union Parishad Chairman, and attended by all 13 UPSCs, as well as community representatives from VDCs. In these meetings each UPSC are required to provide an update of progress against their respective components, and how the Union Parishad budget is being utilized against commitments made. The second platform is the quarterly Union Parishad and Wardshava meetings with VDC leaders, where communities are updated on the utilization of budgets against planned activities. NJP is continuing to work on this approach to ensure that the remaining one Union also includes a budget based on its Village Development Plan. As an important sustainability approach of continued support, NJP has facilitated the formation of 622 VDCs in 40 Union Parishads, and has actively strengthened the participation of VDCs in observation days, and in the Wardshava open budget meeting, to raise their demand that the Union Parishad includes their needs in the annual planning and budgeting process. The MTE team were impressed with the level of engagement of women in VDCs, both in terms of numbers and NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 52 participation, where their enthusiasm and self-worth was evident. NJP is commended for their contribution in strengthening the role of women empowerment across these remote, and in most cases, conservative areas. From the interviews with VDC members, the MTE team found both women and men to be aware of the importance of the VDC as an overall coordinating body, and accordingly were found to be engaging with the various platforms that existed within the community. VDC members also appeared to be linked with local government and elected officials, where the MTE team was taken to meet different local elites after concluding VDC interviews. These were positive indications that these VDC members will continue to maintain the VDC after the closure of NJP. NJP operates in some of the remotest areas of Bangladesh, some sites taking up to four hours by boat to reach (e.g. Sutarkhali in Dacope Upazila). From the field visits, it was evident that the remoter operational areas suffered from a lack of visits by government extension workers given the distances and the different modes of transport involved. These trips were expensive, and given the limited resources of the government, it was not surprising that these areas were less frequented by government staff. Key Evaluation Question 1.4. Effectiveness of the promotional graduation intervention Strategy as per Implementation Guideline A total of 14,000 Ultra Poor participants are targeted for this intervention. According to NJP’s database, the project has identified a total of 62,137 Ultra Poor participants – of which only 23 percent of the Ultra Poor could be included for the Promotional Graduation component. These participants were selected through a rigorous Participatory Rural Appraisal (PRA) process, where the project identified the poorest and most vulnerable households – based on different well-being categories identified by the project included in the census data. Promotional Graduation participants are divided into two cohorts – 7,000 participants in each cohort. As per the NJP Promotional Graduation Program Implementation Guideline, a minimum of 60 percent of the participants must be female, grouped in 20-25 members, and trained three times a week in two-hour sessions. The entire duration of the Promotional Graduation intervention with each cohort is 24 months, following the schedule below:  First 9-months participants undergo a rigorous ELT course. Led by Winrock.  From month six, Savings Groups are formed, and participants trained. Savings Groups are required to open savings accounts with formal financial institutions. Led by WVB.  From month six, participants are supported to select two IGAs, and to develop business plans around these. A “primary” IGA and a “complementary” IGA are selected such that a backup IGA is on hand should the primary IGA not be successful. Each of the business plans must meet environmental compliance so as not to negatively impact on the surrounding landscape. Led by WFP.  Promotional Graduation participants receive a compensation allowance of Tk 1,000 per month, for the first 9-months of attendance to ELT. To be eligible to receive this stipend, each beneficiary must have an attendance rate of at least 60 percent. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 53  From month seven, participants are eligible to receive a total of Tk 15,000 as start-up capital to purchase inputs for their chosen IGAs. The funds are dispersed variably, following the business plan, and the investments required at different stages. IGA participants are provided between 2-5 days of training on their selected IGAs, and 3-months of mentorship after the start of their IGA. Led by WFP.  NJP will engage a learning partner to document changes, impact and processes over the 5-year period.  Promotional Graduation participants have the option to join NJP Producer Groups or AIGA groups, after they have started their IGA, for further support.  Once Promotional Graduation participants are selected, NJP will conduct a separate baseline, for each of the two cohorts, through their learning partner to collect socio-economic data of the participants.  To support gender equitable participation, NJP will ensure that a minimum of 60 percent of participants are female, and that child care solutions and mobile trainings are established to maximize female participation.  Promotional Graduation participants are actively engaged in broader community activities by linking them to VDCs, WASH, MCHN and DRR activities.  Participants are considered to be graduated if they meet 80 percent or more of the following criteria (NJP targets 70 percent of its Promotional Graduation participants to graduate by the end of the project life): o Multiple sources of income o Three square meals a day over the past year o Household owns productive assets o Households have developed kitchen gardens o Cash savings in a formal financial institution o Household uses sanitary latrine and drinks safe potable water o Practices knowledge in disaster preparedness activities o No child marriage in the household o School-aged children are going to school o Eligible couples adopt family planning methods Progress to date From project data, NJP reported that the first cohort had registered a total of 7,023 participants (all female). Of this, 355 participants were excluded, as 259 participants had either migrated, were too ill, or simply lacked interest, and another 96 participants were dropped to avoid duplication with the MCHN CCT, AIGA, and Agriculture activities. The remaining 6,668 participants were organized into 338 groups. From the qualitative primary data collection of the MTE, eight FGDs (all female) and 11 IDIs (all female) were conducted with Promotional Graduation participants. From these interviews it was evident that participants were universally appreciative of the ELT, where they took great pride in their basic ability to read and write. For the ELT component, a total of 6,016 participants have now completed their course, and 652 participants were scheduled to complete their training by June 2018. The Savings and Lending component was another intervention which clearly motivated participants, where all groups were able to show the evaluators their savings boxes, with the NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 54 mandatory three keys. Each group had elected a leader and a treasurer, and all members generated savings through the purchase of shares at Tk 50 (approximately US$ 0.65) per share, and it was evident that loans were being dispersed to members. All eight FGDs confirmed that they had a 100 percent repayment to date. By-laws had been established, where members who were facing particular hardship were granted extensions to repay loans, and should they not be able to repay due to unforeseen circumstances, then loans would be waived. Further, all Promotional Graduation groups had formed their own Savings and Lending Groups, and members had opened bKash accounts. However, the project had not facilitated the formation of savings accounts in formal financial institutions as of yet. This approach had been deliberately taken to allow time for the first cohort to develop their skills in generating savings so that they were better prepared to manage savings accounts in formal financial institutions. NJP plans to develop savings accounts in formal financial institutions over the remaining LOA. A separate “social fund” was also established in each group, set at between Tk 500–1,000 (approximately US$ 6.50-13), as decided by the group. All members of the group were responsible to deposit money in equal shares to the social fund where the money was kept in a separate yellow bag inside the savings box. The social fund was dispersed based on group agreement to members in need, such as paying medical bills for a child, or for children’s educational purposes. These funds were disbursed as “interest￾free” loans, which were payable within 30-days of taking the loan. In instance where a member was unable to pay back a loan due to circumstances beyond his control, the group would then write-off the loan and all members would again contribute equally to revitalize the capital. For instances where members requested money from the social fund to pay for a family funeral, these were disbursed as charitable gifts which did not require repayment. Evaluators were shown business plans that had been developed, together with a checklist ensuring these conformed to environmental standards. All FGDs and IDIs sampled under the Promotional Graduation intervention comprised of 100 percent women participants, confirming that NJP was reaching over 60 percent females for this component. Women interviewed were found to be speaking with knowledge on the various graduation activities. As to be expected, there was invariably a section of women who were shyer and needed to be coaxed to speak – but when encouraged to participate, they were found to be equally knowledgeable as their more confident counterparts in the group. Women from both the FGDs and IDIs universally shared that they felt more empowered, and that their spouses acknowledged the benefits they had gained by attending these sessions. Improved literacy and the fact that they were bringing in Tk 1,000 (approximately US$ 13) per month (with the knowledge that they would be provided a further Tk 15,000 (approximately US$ 190) as start-up capital) had earned them greater respect within their households. These participants shared that their spouses were now more receptive to their ideas, and that household decisions were now more collectively being made. Savings and Lending Group; Shyamnagar Upazila NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 55 The flexibility of project staff was seen as a best practice by the MTE team. The three weekly sessions (of two-hours each), were conducted suiting the timings of the participants. This ensured maximum participation in these sessions. An external research agency, Human Resources Development Center (HDRC), has been retained recently in December 2017, to conduct operational research on the impact of the Promotional Graduation intervention, where the firm is now finalizing their baseline report for the first graduation cohort. The project strategy to allow Promotional Graduation participants to later join other project interventions such as the AIGA and Agriculture components was considered to be a sound approach as this would continue to build the capacity of these destitute populations to break the cycle of poverty. However, it was too early to observe the uptake of this strategy considering the first cohort had only just completed their 9-month ELT, and the majority were now awaiting their seed capital to begin their selected IGAs. The MTE team were satisfied from the primary data collection that Promotional Graduation participants were being effectively sensitized on issues such as WASH, MCHN and DRR. From the sampled FGD and IDI interviews, participants were able to confidently speak about a variety of issues on these topics, and the importance they bear on the overall well-being of their households and communities. These interviews, in addition to interviews held with VDCs confirmed that graduation groups were connected to VDCs and were participating in different community level activities. NJP approaches the graduation model differently to the traditional BRAC approach (Figure 3). The 24-month BRAC model is staged where in the first 3-months the targeting is completed, consumption support and formation of micro-credit groups take place between months 3-6, and skills training and asset transfer is around month six. The remaining 1.5-years is devoted to support on-the-job capacity building. Figure 3 – BRAC graduation model13 13 Syed M. Hashemi and Aude de Montesquiou, “Reaching the Poorest: Lessons from the Graduation Model,” 2011 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 56 The NJP approach differs where the 24-months only begin after the targeting has been completed and the Promotional Graduation groups have been formed. The project focuses a considerable time on the ELT component over the first 9-months, and start-up capital and start of IGAs begins from month seven. Differences between the two approaches can be observed where BRAC starts developing micro￾credit groups immediately after targeting, whereas NJP starts developing Savings and Lending groups from month six after targeting. Another important distinction between the two approaches is that BRAC begins its skills training, asset transfers and start of IGAs from month three after targeting, whereas NJP does the same from month seven after targeting. The impact of the BRAC approach has been widely analyzed and considered to be an impactful approach. However, the NJP approach to graduation could not be critically assessed due to the early stages of its implementation, and hence it was not possible to determine the level of impact this will bring over the LOA. With regards to the ELT, the MTE team considers that the initial effort on raising literacy capabilities at the start of the intervention certainly is positive to build confidence among these beneficiaries – however, the duration of 9-months may be reconsidered to make this shorter in line with the 6-month ELT afforded to the AIGA participants. This is discussed at greater length in the AIGA section. Challenges The engagement of numerous partners in the Promotional Graduation intervention – WVB, Winrock and WFP, supported by their three national partners Codec, Shushilan and Nobolok, has led to confusion. Field staff, from both Winrock and Codec, relayed that WFP’s intervention from month seven, to identify IGAs, whilst the ELT was taking place, was disruptive. There were also misgivings as to why WFP were engaging in the Promotional Graduation IGA component, considering Winrock were already implementing the 6-month AIGA intervention. It was not clear to the MTE team what the difference was between the 24-month Promotional Graduation IGA and the 6-month AIGA components, as both engaged in “on-farm” and “off-farm” income generating activities, where the primary difference was that the former invested in a longer support period as part of the graduation approach. However, the technical content could be considered to be similar, which led the MTE team to believe this to be a duplication of much of the work. Start-up capital was considered by all beneficiaries to be important. This was not surprising considering the poor status of these households, where this seed capital enabled beneficiaries to purchase inputs for their chosen business plans. However, inconsistency in start-up capitals provided to different sets of participants was found to be causing major issues in the field. NJP provided Tk 24,000 (approximately US$ 300) to graduation participants (Tk 1,000 per month over the first 9-months, followed by a start-up capital of Tk 15,000); Tk 1,000 (approximately US$ 13) as a one-time start-up capital to 18 percent of the 6-month AIGA participants (3,200 participants); and no start-up capital to the remaining 82 percent (14,800) AIGA participants. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 57 Promotional Graduation participants were provided significantly more resources as the goal was to sustainably lift them out of poverty. AIGA participants were provided significantly less resources primarily due to a lack of resources, where only 3,200 participants could be provided a minimal input, and the remaining 14,800 were provided no inputs asides from training and capacity building. NJP is currently reviewing this issue where Savings and Loans associations may be expanded to AIGA participants and become a source for loans. Although the project has taken great effort to target the poorest and ensuring equal opportunity, only 14,000 of the identified 62,137 Ultra Poor could be accommodated as Promotional Graduation participants. The remaining Ultra Poor participants were then included in the other components of the project, including the AIGA component. Hence, it can be inferred, that the socio-economic characteristics of the Promotional Graduation and the AIGA participants were similar. This has raised questions among the AIGA members why Promotional Graduation participants receive such higher levels of resource, considering they are both from the Ultra Poor category. This has been further complicated by the split within the AIGA group where only 18 percent received Tk 1,000, and the remainder received no inputs asides from the training. Front line staff repeatedly expressed this as a challenge, where participants were questioning the logic of this approach. This indicated that an effective communication strategy needed to be developed and unrolled explaining to participants that Promotional Graduation participants were part of a larger programmatic effort to pull them sustainably out of poverty, and hence were provided significantly more resources to reach this goal. NJP has recognized the need for a more effective communication strategy and is developing a catchphrase that will be used repeatedly at different forums to bring across the message that the Promotional Graduation intervention is to lift these ultra-poor participants sustainably out of poverty. To meet this objective, these participants are hence provided considerably more resources. This message will be communicated repeatedly in the NJP monthly Upazila meetings such that all staff and partners are reinforced with this message, and the need for them to likewise reinforce the message to their communities and partners. This is planned to take place by November 2018. Similar messages will also be used in the monthly NJP Union level meetings. NJP facilitators will then introduce this message in their scheduled meetings with Producer Groups, Savings and Lending Groups, AIGA groups, Promotional Graduation groups, VDCs and in ELT sessions. This is planned to be introduced by December 2018. Similarly, the messaging will be reinforced in Union Parishad meetings in the presence of the Chairman and Union members – again by December 2018. The NJP graduation component adopts a different approach to that of the traditional BRAC model. The NJP approach waits a full 7-months before the start-up capital is provided to participants, whereas the traditional BRAC approach provides the seed capital much earlier. The BRAC model of providing resources earlier in the timeline allows for participants to receive support over a greater length of time, than the NJP approach. Considering its early stage, it is not possible at the time of this MTE, to assess how successful the NJP approach will be, compared to that of the proven BRAC model. Given the minimal differences between the Promotional Graduation and AIGA participants, questions arise as to why the former benefit from a 9-month ELT, and the AIGA participants receive three months less training – i.e. 6-months. AIGA participants interviewed expressed their appreciation of the 6-month ELT and considered this to be adequate for their needs. If so, the NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 58 same can be considered for the Promotional Graduation component, where field staff shared that they considered 9-months to be too extensive for the ELT. Savings Groups within the Promotional Graduation groups were found to be working well. However, the groups visited in the sample frame were not found to have any savings account in formal financial institutions. The project plans to develop savings accounts over the remaining LOA, primarily exploring the savings feature of bKash, as this cohort till now were in the development stages where savings was still being generated. This can be risky as savings grow, should these be kept within the community and not deposited in a recognized bank – hence it is recommended this is prioritized and formal savings accounts are formed quickly. Objective 2: Evidence of Change Key Evaluation Question: Community perceptions on changes brought about by NJP Community members were appreciative of the SBCC-WASH sessions, where the MTE team directly observed that the project has made progress in changing behaviors as beneficiaries were adopting better health and hygiene practices, especially in areas where improved latrines had been installed. Evidence of these behavior changes were seen when MTE team members visited latrines, with no prior warning, regardless of whether or not these were provided by NJP, and found the majority of latrines were clean and contained soap, sandals, and water nearby. The SBCC-WASH sessions combined with the construction/rehabilitation of latrines, the establishment of water points, and the strengthening of WATSAN committees have all contributed to improving the hygiene and sanitation status of project beneficiaries. However, it was evident that the needs of the broader NJP participants were not being met as the project had only a limited number of beneficiaries who received improved latrines, and the evaluators were constantly requested by community members to inform the project of their need for a latrine. This indicated that participants were not fully conversant on how to be included for getting support on improved sanitation structures, and also indicated that the project does not have the resources to satisfy the needs for all its participants in this intervention. Similarly, SBCC-MCHN sessions were highly spoken of by beneficiaries. These sessions were usually scheduled to take place straight after the SBCC-MCHN sessions, and hence the group benefited from messages on both WASH and MCHN. MTE evaluators observed that the majority of these sessions were conducted professionally by CNFs, engaging mothers and caretakers that were present through verbal communication, as well as practical demonstrations. GMP sessions were also well considered, where mothers appreciated the opportunity to monitor their child’s growth, interact with other mothers, and gain valuable advice and insights on how to improve child care. Mothers specifically highlighted the value of being able to observe their child’s progress through color coded charts, being able to discuss issues such as breastfeeding and other child feeding practices within the GMP sessions, and in follow up household visits by CNFs and MCHN Promoters. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 59 The CCT intervention by far was one of the most valued interventions. This was not surprising considering the large sums of money transferred each month to participating mothers. However, there were risks associated with this intervention as approximately 20 percent of mothers informed the MTE team that the cash was used for other purposes than that of solely purchasing food. Also, the NJP cash transfers, per month, were over four times more than that provided by the GoB CCT program. This has led to tension where the MTE team came across mothers who were not included in the NJP CCT intervention, but were members of the GoB CCT program, who were highly critical of the discrepancy, and felt they had been unfairly treated. Along with the cash, the improved nutritional status of children and PLW was recognized by community members and most local leaders as being partially linked to the NJP intervention. Women who were receiving the MCHN sessions and the CCT also agreed that due to their participation in the project, they were now more knowledgeable about women and child health issues and that they were using techniques they had learned from the project within their households. The evaluators found that the promotion sessions had made mothers aware of the importance of taking iron and calcium tablets during pregnancy, and the importance of Ante-Natal Care (ANC) and Post-Natal Care (PNC) visits. They also appreciated the cooking and feeding demonstrations. Through these activities, mother’s shared that they now felt more confident and were able to ask more critical questions in the GMP and SBCC-MCHN sessions. Community members agreed that one of the most important results of the project was the change it had brought in the role and capacity of women in their communities. By training women and giving them options to generate revenue and by promoting their inclusion in leadership positions of different groups, NJP was found to be elevating the status of women significantly. However, given some of the extreme conservative areas the project operates in, there may be negative perceptions of the project promoting mixed groups. The MTE team was, for example, in one instance told by a community member that, while the actions of the project were beneficial to the community, the project should not mix men and women in the same activities because that was forbidden by religion. Although this was an isolated comment, the MTE team took note of this considering some of the extreme religious and cultural sentiments that may exist in some of the areas the project works in. The ELT was well received by beneficiaries. The MTE team were pleased to observe the pride that women took in their improved literacy skills, frequently wanting to show their reading and writing skills to the evaluators. This has undoubtedly contributed significantly to the confidence of these women, their overall empowerment, and elevated their status within their households and communities. The MTE team considered Homestead Producer groups to be less effective. Although the Homestead Producers interviewed were appreciative of the intervention, the training and inputs received by these subsistence farmers were much too minimal to effectuate any meaningful and sustainable change. The Lead Farmer approach was not found to be an effective mechanism to build capacities of Homestead Producers as Lead Farmers lacked the time and enthusiasm to reach up to 200 farmers through these Producer Groups. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 60 Objective 3: Collaborative Learning and Action Key Evaluation Question – Use of data to inform program decisions, referral and follow-up NJP is doing well in unpacking the M&E system, which is largely driven by FFP M&E and reporting requirements. NJP primarily uses the M&E system for monitoring purposes to respond to management and donor reporting requirements. The system centers on quantitative data collection at an activity level, with minimal qualitative data being collected, and focused entirely on direct beneficiaries of the project. Secondary adoption, and replication data is not collected. Winrock and WFP are following the NJP M&E plan and reporting requirements, with particular focus on the interventions more relevant to them. NJP (WVB M&E team) collects and accumulates the achievement data of Winrock and WFP as part of monitoring the work done by these two organizations. Both institutions submit progress data and report to WVB periodically and provides updates in scheduled coordination meetings. Throughout the field visits, NJP staff across the different partners constantly mentioned the MIS system, which indicated that staff were using the system for various purposes. The M&E specialist within the MTE team performed sporadic checks of the M&E system and found that better “validation” systems could be incorporated to ensure that unrealistic data cannot be entered beyond given ranges. Greater effort also needed to be invested on internal quality checks and more frequent Data Quality and Assurance (DQA) were needed to ensure that the data was accurate and reliable. The PRA conducted at the start of the project, utilized social mapping and WBA (Well-being analysis) tools, where facilitators were trained for improving the quality of the PRA sessions. However, the outputs of the PRA sessions were strongly disputed by many government counterparts, and the MTE team directly observed several poor households from the sample villages that were not included in the NJP project. NJP has already identified this as a major short￾coming and have taken extensive steps to redress this through a “2-3 step physical verification and screening process” to ensure the targeting of the poor and extreme poor households is more reliable and accurate. The “learning” agenda, in the initial design of NJP, did not sufficiently consider how knowledge would be disseminated. NJP management identified this gap early after project initiation and created the Knowledge Management and Learning team. This unit is fully staffed, led by the Knowledge Management and Communications Manager, and the team consists of the Knowledge Management Coordinator, the Learning Coordinator, Program Officer and a Graphics Designer. The MTE evaluators found this team to be knowledgeable, and proactively pursuing learning agendas including a formative research on MEGs, and contracting HDRC to conduct the baseline assessment of the first cohort of Promotional Graduation participants. The team also engaged in positioning and influencing initiatives, where it attended the 2017 SEEP forum in Washington DC to promote NJP. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 61 Objective 4: Sustaining Project Impact Key Evaluation Question: Sustainability strategies A key shortcoming the MTE came across was that NJP lacked a detailed Exit Strategy, developed based on the experiences of the project. At the time of project design, a summary Exit Strategy was attached as an annex to the proposal, however none except one NJP staff during the MTE was able to refer to this strategy, making apparent that project staff were not considering the longer￾term view of how the project would eventually exit and ensure interventions sustained. Further, the Exit Strategy developed at the start of the project was based on initial assumptions and had not been updated based on the experiences of the project after implementation started, and also lacked detail of how the project would responsibly exit out of communities. The MTE finds this to be problematic which will impact on the sustainability of interventions, platforms and processes that the project has introduced. Early signs of this issue were apparent to the MTE team when interviewing beneficiaries and government counterparts, where the universal hope was that NJP would be extended and continue to provide support over the next several years. This was concerning to hear at only the mid-point of the project and was symptomatic that the project was not sufficiently addressing sustainability factors among beneficiaries and support structures. The MTE team also could not find any strategies that promoted secondary adoption, within and beyond the communities in which they worked. This was a shortcoming considering that the project’s success will be based on a “population-based” impact. As a minimum, the project needed to build in mechanisms to ensure that all households within the communities, where the project worked, benefited from its interventions. The MTE team observed that the following progress have been made to date concerning sustainability issues. WASH: Access to safe drinking water is the existential threat across the south-west of Bangladesh, due to the high levels of salinity across this region. NJP was attempting to ease access to water through constructing/rehabilitating PSFs and DTWs. Most of the community-based water options provided by the project have managed to develop bank accounts as maintenance funds. However, there has been mixed progress in collecting maintenance funds from users, which if not strengthened and formalized may impact on the future sustainability of these water points. The rehabilitation and construction of latrines similarly did not have sufficient buy-in from communities, as the MTE team found that most households, although aware of the importance of hygiene and sanitation, were reluctant to invest their own resources to these structures. Communities lack of interest to contribute indicated the prevailing dependency in these areas that donors would continue to pick up on development initiatives, a feeling which potentially has been further fueled by NJP’s intervention of subsidizing latrine construction. Given this reality, it is unlikely that households across project sites will invest to scale-up hygienic latrines through their own resources. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 62 It is essential that the project advocate with government support structures for greater resources to be mobilized to develop sufficient provisions for safe water and hygienic latrines, which are subsidized based on a household’s socio-economic means. At the same time, the project needs to develop the capacity of communities to develop maintenance funds for community water points, and encourage households to maintain the upkeep of their own private latrines. NJP is aware of this issue and accordingly has been using the CVA approach with Union WATSAN committees to make them more accountable and responsive to the needs of their communities. To date, the project has completed interface meetings with 37 Unions. The project should consider this as a key advocacy issue, to mobilize coalitions to pool resources and scale-up effort. NJP supported WASH initiatives by developing the technical capacity of WASH Entrepreneurs and linking them to WASH fairs and promoting WASH products in the SBCC-WASH sessions. However, the project needed to place greater effort in developing WASH Entrepreneurs to a level where youth involved with this initiative could consider this as a potential livelihood. The importance of participants contributing resources for rehabilitating and constructing latrines needed to be better promoted, which is made easier considering the progress NJP has made in raising awareness on hygiene and sanitation issues. With greater demand, WASH Entrepreneurs can grow into viable micro-enterprises, and WASH LSPs can become more active as a result. MCHN: NJP has helped to connect PLW with the community health centers. Through the CCT intervention, mothers were now actively engaging in GMP sessions. However, the discrepancy between the monthly stipends provided by NJP (Tk 2,200) and the GoB (Tk 500), continued to raise frustration among mother’s registered under the GoB program. The MTE team repeatedly heard how these mothers considered themselves to be unfairly treated, and why NJP had not registered them. It is recommended that the project actively engages with the Ministry of Health to seek ways of developing middle ground where the NJP approach compliments the GoB approach, and not be seen to be in direct competition. With regards to GMP, this component of NJP directly compliments the government’s effort in scaling-up GMP across the country. Accordingly, the government health centers fully support NJP to use their facilities to conduct GMP sessions. SBCC-MCHN sessions were also highly appreciated by mothers and caretakers, where they welcomed the opportunity to learn how best to take care of children and mothers, and the space where they could converse of how others were adopting practices and overcoming issues they were encountering. These were positive indications that the project was successfully motivating mothers to understand the importance of such meetings. The MTE found that Community Clinics were responding to the needs of the community, where during visits to the clinics, evaluators observed patients coming to the clinic for treatment. On interviewing these patients, there was a universal positive reflection of the clinic where the patients shared they were happy with the services the clinic was providing, indicating these relationships would continue to sustain after project end. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 63 Gender: The gender integration approaches of NJP were commendable, and the MTE team were impressed on the obvious successes already evident during the field data collection. Platforms such as the MEG, as well as greater attention to developing mixed groups of VDCs, and conducting courtyard sessions such as the SBCC-WASH sessions, have all contributed to the elevated status of women. However, the project lacked strategies of how these platforms would continue, and scale-up across all its working areas. It is recommended that suitable strategies are considered, and updated as part of the TOC, to develop replication and sustainability strategies for these groups. Agriculture and Alternative Livelihoods: NJP correctly strategized the importance of developing market systems with which beneficiaries could effectively connect with. The MTE team commends the local market assessments to identify commodities (both on-farm and off-farm) that were in demand, such that the project could develop the capacity of beneficiaries in these areas. However, limited progress has been made in the formal employment sector, and the project needs to scope local industries for employment opportunities and develop capacity of youth from NJP working areas in these sectors. Greater collaboration with government structures, including the Ministry of Youth, Ministry of Women Affairs, Bangladesh Small and Cottage Industries Corporation (BSCIC) and Technical Colleges, should be explored as they all offer basic training on youth development. The development of LSPs was another good approach. A total of eight LSPs were interviewed where the MTE team found that good progress has been made with Livestock/Poultry LSPs as beneficiaries within these communities universally shared they have excellent access to their services which includes vaccination for their livestock and poultry. However, Agriculture and Aquaculture LSPs were less visible. It is recommended that the project invests greater effort to develop LSPs spanning all these technical sectors. This is especially recommended for the remoter areas where the project functions, where government support services have limited resources to visit. Developing local LSPs in these areas will be especially important so that these marginalized communities can access support from LSPs located within, or around, their communities. The Lead Farmer approach was found to be weak and an approach the MTE team does not consider will sustain. Interviews with Lead Farmers showed that these farmers did not have the time or inclination to build the capacity of their Homestead Producer groups, which was made evident to the MTE team during home visits where evaluators observed only small garden patches. These were too minimal to effectively contribute to the overall household nutrition and were likely to cease after project support ends. These were all indications of the poor levels of attention Lead Farmers were paying to the development of Homestead Producers. Many of the Value-Chain Producers, on the other hand, were likely to better sustain, as 62 of the 124 trained Lead Farmers were part of these groups. Farmers in these groups were of a higher wealth category, which enabled them to engage in commercial production, where as collectives, these groups could bargain for optimum prices for their commodities from surrounding markets. Lead Farmers, as a result, were paying closer attention to these groups, given their direct interest in making larger profits. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 64 The graduation approach adopted by NJP differs from the traditional BRAC model and is discussed at length under Evaluation Objective 1.4. The graduation approach requires greater attention, and more resources than other components of the project, and if these are to be sustained and replicated, require greater mobilization of resources. The MTE team did not find any sustainability or replication strategy that is considered by NJP to promote the graduation approach. Disaster Risk Reduction: The DRR interventions of NJP directly supported the government SOD, and hence UDMCs and DRR committees formed had greater likelihood of being supported and sustaining. Considering there has not been any disasters in the NJP working areas since its inception, the MTE could not validate how these structures would respond to disasters. However, the MTE team were satisfied that appropriate preparedness activities were being undertaken by the project. Good Governance and Social Accountability: With regard to local government officials, and local leaders, NJP was involving them in most activities that were undertaken in their area. When interviewed, these respondents underscored the need for activities to be sustained and their willingness to participate. However, the common issue of resources was repeatedly brought up as a constraining factor, as both the local government and local leadership had insufficient resources to implement their own development plans. Taking on additional commitments without the resources needed to implement them would likely not be an effective undertaking. The CVA approach was repeatedly cited by staff as being a key tool in mobilizing participants. This was evident where the CVA approach was successfully used to strengthen the linkages between communities for better responsive Community Clinics, and where the CVA approach was used across 40 Unions to promote participation of communities to produce Village Development Plans and bring these to the attention of the Union and Upazila for budgetary and technical support. Objective 5: Cross-Cutting Themes Key Evaluation Question: Effectiveness in addressing cross-cutting issues The gender component of NJP is a set of interrelated activities implemented in different components through various groups of actors. Three of those groups include the MEG, MCHN and Life skills SBCC groups. The objectives of the Gender component are to address gender biases that exist in the target community when it comes to access to and control over resources and leadership positions. While the MTE recognized that change takes time, especially on issues regarding social beliefs and behaviors, at midterm, it was clear that the gender component is working well. Anecdotal evidence was gathered to show the change that was taking place. Women were represented in all the groups of influence supported by the project, were speaking with confidence, and speaking from positions of influence in front of their male counterparts. NJP has several interventions that engage youth. These include the LSBE groups, and engaging youth in DRR related activities. LSBE youth members interviewed by the MTE team shared their appreciation of the life skills training they were receiving. However, participation was limited as NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 65 the majority of school children were reluctant to participate in these sessions after a full day of school. The other significant area where youth were engaged in was as DRR volunteers. Youth volunteers interviewed by the MTE team were found to be enthusiastic to perform their functions and appreciated the significance of their roles and how this could protect their communities. When asked what motivated them, the universal response was their sense of pride as other community members recognized the important roles entrusted to them – hence elevating their status within communities. NJP’s strategy of requiring women to have their own bKash accounts to be eligible to receive cash transfers for the Promotional Graduation and MCHN CCT components, was a mechanism to empower women and ensure that money received remained with women. Although the strategy was developed with the best of intentions, this had become an obstacle to many women as they were not eligible to register for bKash accounts considering they did not possess the necessary National Identification Card (NID). The NID is an absolute requirement from the GoB to obtain mobile SIM cards, which is needed to open a bKash account. To date, of the 15,025 MCHN beneficiaries eligible to receive cash through CCT, 64 percent (9,598) beneficiaries do not have their own SIM cards. Likewise, of the 10,146 Promotional Graduation beneficiaries, 21 percent (2,122) beneficiaries do not have their own SIM cards. The project recognized this as a critical issue and consequently offered flexibility allowing women to use their spouses SIM accounts to open bKash accounts. This approach was not universally accepted across NJP, as staff opinions varied that this approach did not assist in empowering women, and that there was no guarantee that money intended for these women remained in their hands. However, the MTE team considered this flexibility as an essential alternative, to reduce the risk of exclusion of mothers without their own SIM cards. Environmental compliance with regard to latrine construction and water testing was a major challenge. As per SUAP, NJP had to ensure that no water source was within a radius of 10 meters from any latrine to be constructed or rehabilitated. Given the limited availability of land in the NJP working areas, many households were unable to comply to this requirement. With regard to water testing, USAID standards required that in every six months several chemical tests were to be conducted, including tests to detect fecal coliforms, and tests to detect the presence of arsenic. In cases where fecal coliforms were detected, chlorination had to be undertaken and in case of the presence of arsenic beyond acceptable thresholds, water sources needed to be painted in red and closed. These tests were expensive and while NJP had acquired a limited set of testing kits, the project had not formalized regular testing. NJP is currently discussing this issue with DPHE and plans to formalize an agreement with DPHE by August 2018. Further, indications were that communities were not yet sufficiently sensitized to encourage their greater contribution to these activities. NJP has made progress on governance and social accountability, where the CVA has been an instrumental tool. The project works with five UPSCs, and evidence of progress was determined through interviews with Standing Committee members where they shared that the committees were now holding meetings regularly, and through these meetings were identifying issues, and developing strategies to overcome these. The CVA approach complemented this effort, by organizing and raising the voices of communities, and ensuring they were heard at the Union and Upazila levels. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 66 In the area of “adaptive” capacity strengthening, NJP has had mixed results. Although the project has worked to develop the capacities of farmers to adopt CSA techniques, the MTE observed that Homestead Producers had made poor progress. However, the significant challenges of salinity were somewhat broached through PSFs and DTWs which provided access to much needed potable water and was seen to be a good adaptive practice. Likewise, NJP has made progress in strengthening the “absorptive” capacity of its participants through its AIGA interventions which has diversified income sources for beneficiaries and afforded a greater safety net at times of shocks and stress. The Promotional Graduation component has also been designed to strengthen the overall resilience of its participants, with the intention of sustainably graduating participants out of poverty. Less progress has been made in developing “transformative” capacity, as institutional structures continue to lack resources to be fully supportive of its communities. Initial progress has been made where government counterparts at the local level are engaging with different NJP initiatives – however, these localized co-operations are not bringing change at scale at the policy and broader implementation levels. Objective 6: Project Framework Key Evaluation Question: How can activities be modified The TOC references that NJP will work with “vulnerable” populations – however, there is no concrete definition of how the project defines vulnerability. It was evident from the MTE that target beneficiaries are registered based on differing criteria for different interventions. The TOC should make clear reference on how “vulnerability” is defined, and the standard be followed throughout the remaining LOA. Purpose 1: NJP interventions to promote healthy and hygienic behavior has been successful through its SBCC-WASH sessions. However, hardware interventions of rehabilitating and building latrines was less visible. As part of the sustainability strategy, the project required that beneficiaries provide “in-kind” support in the form of labor to construct latrines, and the evidence from the primary data collection suggested this was taking place to the credit of the project. Even so, NJP needs to reassess its objectives with regard to this component and make a final determination regarding the cost and quality of the latrines. Does it need the financial participation of beneficiaries to build high quality latrines, or not? If the financial participation is needed, then NJP should start to put in place adequate strategies to mobilize that. This should include building partnerships with other agencies and the private sector involved with latrine building. Likewise, the fee-for-service model and the development of viable WASH business models needs to be strengthened. It is important when revising the TOC to clarify WASH business models to be promoted by NJP and to start working with the target communities from that perspective. All water points that were reviewed by the MTE team, did not have a structured “fee-for-use” of water, indicating that the Water Management Committees had failed to put in place these systems. The lack of financial participation will in the long run jeopardize the sustainability of these water points as insufficient funds are being generated for their upkeep and maintenance. The TOC assumes that the actions of several actors, including the DPHE and NGOs (such as Sushilan, Uttaran, Islamic Relief, Water Aid, and Dhaka Ahsania Mission), collectively will help solve NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 67 WASH problems in its target area. At mid-term it was clear that, apart from the DPHE, none of the other organizations targeted the same villages as NJP. NJP, with its limited resources will be unable to deal with the considerable WASH issues within these areas, and hence the TOC needs to be updated to include a more rigorous advocacy platform to mobilize greater resources. Water quality testing was rightly recognized by NJP as a key activity and the project had plans to establish linkages between community Water Management Committees and water testing service providers. While NJP had bought a number of water testing kits and was performing tests periodically, no formal linkage as yet had been established between Water Management Committees and water testing service providers. Further, NJP had no formal strategy of how to react should water quality test results be negative. NJP has recognized these shortcomings and is currently negotiating with the DPHE to conduct these tests and plans to have a formal MoU by August 2018. With regards to gender programming, the MTE team found that good progress has been made to reduce adolescent pregnancy and child marriage via the use of CPCs. This was evidenced from the interviews held with CPCs, where the team was told about four adolescent marriages that had been prevented due to the efforts from CPCs. The CPC were doing important work, but it was not clear how they would continue to operate in the absence of NJP support. Also, not linking the action of the CPC to the judiciary will likely result in very modest results as the CPC had no authority to stop child marriages. Identifying the conditions on which the CPC could have stronger linkages with the judiciary would certainly result in more sustainable achievements on that front. The project SBCC-MCHN sessions have played an important role in sensitizing communities on the importance of mother and child care, and the GMP sessions provided critical on-going messaging, monitoring and support to PLW with children under the age of two years. However, NJP needs to address the significantly different packages offered by NJP and the GoB, where the project offers Tk 2,200 per month, compared to the government’s Tk 500 per month to PLW with CU2. From interviews with mothers who are part of the GoB CCT program, it was clear that these mothers felt unfairly treated considering NJP CCT mothers were receiving over four times the resources than what they were receiving. To improve gender-equitable norms within households, the project provides life-skills training to adolescent school-going girls and boys, as well out-of-school girls and boys, conducts leadership training for women and youth, and arranges dialogue with the Men Care groups. In addition, the project also planned to create linkages with the existing gender networks such as the Nari Unnyan Forum, the Girls Not Bride – Bangladesh Chapter, and the Bangladesh National Gender Working Group, to ensure sustainability and reinforce messaging – however, this effort is yet to be activated. The TOC needs to consider how linkages can be established with gender networks for systemic support to continue after the close of NJP. The TOC should also consider gender and cultural sensitivities, to articulate when to adopt mixed groups, and when to adopt female only groups, especially when relaying more sensitive issues pertaining to women and girls. Purpose 2: NJP’s female centered approach is sound, considering the imbalance where men are the predominant earners and power holders within households. However, the inputs provided by the project were too minimal to make sustainable shifts out of poverty for these poorest beneficiaries. The TOC needs to determine the minimal thresholds required to break the cycle of NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 68 poverty, and sensibly lead beneficiaries on a graduation trajectory which eventually leads to sustainable livelihoods. The TOC also needs to highlight the intra-household power dynamics, and work towards female empowerment over resources. NJP currently does not have a clear strategy for secondary adoption, which immediately limits the population-based impact required of all Title II projects. The TOC needs to develop clear plans of how to scale-up activities to reach all households within a community, regardless of their socio￾economic status. The strategy should also consider secondary adoption to reach beyond the community and how project activities link with broader initiatives such as Feed the Future (FtF) and other development programs. The promotion of Collection Points was a good strategy to link poor farmers to local markets. However, the project itself, as well as the MTE review, found that limited progress has been made with Collection Points. Although Collection Points were open to both Homestead Producers and Value-Chain Producers, the project strategy of only promoting subsistence farming for Homestead Producers renders this group at a disadvantage as they do not produce sufficient surpluses to take advantage of Collection Points. The TOC needs to detail out how Collection Points can be operationalized and be accessible to all members of the community such that critical thresholds of commodities can be collected to negotiate best terms and prices with surrounding markets. Purpose 3: NJP’s effort in developing the capacity of government institutions and communities to build resilience was a sound approach. The project appeared to be taking the right steps to prepare communities to potential disasters, however there has been no disasters that have hit these areas since the project began. As a result, there was little data to evidence how the capacity of government institutions and of communities have been strengthened as a result of NJP’s interventions. The set of activities that the MTE team were shown are all in line with the initial plan to prepare communities and increase their resilience. NJP could take advantage of the revision of its TOC to explore the conditions needed to make the groups working on DRR more sustainable. The funding of their activities will continue to be an issue if no clear linkages are established with government structures. Purpose 4: NJP has had limited progress on advocacy and governance related issues. WVB has now taken over this component from WFP, recruited a Head of Policy, Advocacy and Research, developed a strategy, have held two policy advocacy workshops and developed two policy briefs. These are all steps in the right direction, and the project needs to continue to identify critical emerging advocacy pieces and put in place strategies that will help to achieve its goal. Issues such as the diverging CCT approaches between the project and GoB, the contribution of the judiciary in fighting against child marriage, the increased participation of government structures in project interventions, the continued commitment of LSPs to work in target communities, are all issues that need to be addressed. NJP needs to do a detailed analysis of its environment, taking into account these emerging issues and craft a forward-looking strategy to address these matters. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 69 6. Recommendations To improve NJP’s effectiveness, efficiency and eventual impact, the MTE recommends the following. Overall 1. The MTE team strongly recommends that a clear Exit and Sustainability Strategy is developed. The strategy should clearly relay how the project intends to build in sustainability parameters against different activities from the onset, and how the project intends to gradually hand-over activities to be taken up by communities and surrounding support structures. It is recommended that findings from the Tufts study on exit strategies be referenced for this work. Notable findings from the Tufts assessment were that  Project success at the end of the project did not necessarily translate to longer-term sustainable impact. The study found that although some projects appeared to be successful during the project life, gains were quickly eroded when project support ceased as communities and support structures had not been sufficiently strengthened to continue making progress on their own.  To achieve long-term sustainable impact, developing resources, capacity and motivation were critical.  Linkages, especially vertical linkages, such as those between community-based organizations or individuals and existing public or private sector institutions, were usually critical for successful phase-over of responsibilities.  In the exit process, a gradual transition from project-supported activities to independent operation was important for sustainability.  Providing free resources threatened sustainability, unless replacement of those resources both as project inputs and as incentives had been addressed. 2. The MTE team recommends that NJP develop clear strategies on how to include households which are not directly participating in project activities, so that they too benefit from the project. It is recommended that the current practice of sensitizing the entire community through platforms such as SBCC-WASH be continued and expanded into other technical areas. This could include encouraging non-participating households to observe and learn from demonstration plots and take an active interest in technical groups operating in their village such as the VDC, MEG, and DRR groups. 3. The MTE team recommends a management review be undertaken to consider staff placements, workloads, and other issues facing staff. Although staff turnover is common across any organization/project, however, NJP’s current turnover of 36.7 percent is unusually high and needs to be brought within manageable levels. 4. Given that multiple partners are involved in data collection, the MTE team recommends that the project increase the frequency of internal DQAs, in addition to its scheduled external DQAs, to ensure data integrity and reliability. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 70 5. Given the 1-year delay in start-up, the MTE recommends that a possible extension be considered to enable the project meet its goal and objectives. This may be taken up as a possible option in year four of the project, by which time there will be a better understanding of how well the project has progressed, and what remains to be achieved. WASH 6. Access to safe drinking water is one of the biggest challenges that target communities are facing. NJP is exploring a number of technologies including reverse osmosis. While that technology is adequate in certain areas, several communities have said that they will not be able to pay for its maintenance. Certain Upazilas have nonetheless found it adequate for their situation. From discussions held with the Additional Chief Engineer (Works) DPHE, and the Project Director and Deputy Project Director of the Bangladesh Rural Water Supply and Sanitation Project of DPHE, several possible alternative sources of water were discussed. It is recommended that NJP makes a careful selection of water technology to install and get the concurrence of the beneficiary community before any final determination is made of the following options: a. scale-up Rain Water Harvesting options. b. scale-up the construction and rehabilitation of PSFs and DTWs. It is to be noted that PSFs may not be operational throughout the year, especially during dry seasons when water levels are depleted. During these times, it is recommended that the project share the locations of nearest DTWs, or other alternative sources of clean water, which can be accessed by communities where PSFs are non-functional, such that these communities can continue to access water. c. during periods of water scarcity, introduce water purifying tablets that can be used to clean pond water as a source of drinking water. Although these are at a household level, and not at a community level, the MTE recommends this option to reduce the possibility of disease from contaminated water, at times of extreme water scarcity. 7. The MTE found that water quality testing is a major issue across NJP sites. NJP is currently negotiating with DPHE and plans to have a formalized agreement by August 2018 where the DPHE begins to conduct routine water quality tests across the different NJP water points. As the DPHE becomes increasingly engaged, it is recommended that NJP introduces Water Safety Plans and Water Quality Testing Protocols at each water point, together with introducing an effective reporting system where relevant documents are maintained at the NJP Upazila and Union offices, and with the Water Management Committees. This will enable the project to ensure that the DPHE are held accountable and that water quality tests are being performed. 8. Putting in place provisions for the maintenance of water points is lacking across project areas – including the lack of maintenance funds, lack of tools, and a lack of guidance on what maintenance schedules are to be followed. The MTE team recommends that Water Management Committees are trained more rigorously on maintenance processes of water points as follows: a. Introduce a mandatory “fee-for-use” approach where households are required to pay a small fee for accessing water, which goes towards the maintenance fund for that water point. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 71 b. Develop a detailed checklist of the schedule, and the components that need to be maintained / replaced to keep the water point functioning effectively. c. Train the Water Management Committee on how to perform basic maintenance functions of the water point. d. Provide the required tools to the committee, immediately after training, such that they are able to perform maintenance functions. 9. The MTE did not come across any WASH Entrepreneurs, and there was a general finding that these entrepreneurs were lacking presence as people were not sufficiently motivated to consider this as a viable IGA. WASH Entrepreneurs can play a vital role in the long-term sustainability of WASH improvements across communities and accordingly it is recommended that NJP prioritize developing WASH Entrepreneurs through targeted training on different WASH products, and the construction/rehabilitation of WASH structures. It is recommended that local scoping surveys are conducted to identify local businesses dealing in WASH products, and that WASH Entrepreneurs are effectively linked to these businesses to supply WASH material needed within communities. This can lead to sustainable business models where local businesses are motivated through the sale of their products, and WASH Entrepreneurs having access to material for them to proficiently use these products for WASH related activities and maintenance within the communities they serve. As a further measure, it is recommended that WASH Entrepreneurs, once trained, are certified by the project, provided a picture Identification Card, and then actively promoted across the locality as qualified individuals who can perform WASH related activities. 10. The need for improved latrines is dire across the region. NJP does not possess the resources needed to provide such latrines at scale across project sites. However, the MTE came across several organizations that were focused on providing improved latrines. It is recommended that NJP actively links with these agencies to develop a viable partnership. This could include NJP taking on the role of developing soft skills of communities through the SBCC-WASH platform, to compliment the installation/rehabilitation of improved latrines by the partner agency. At the same time, NJP is recommended to continue working with WATSAN committees and advocate for government to plan for appropriate budgets and support for the scale-up of improved latrines. MCHN 11. From the MTE, it was evident that CCT mothers were using their cash transfers, not only for purchasing food, but also for other household needs. From the primary data collection, the MTE found that approximately 20 percent of CCT respondents said they saved Tk 1,000 (approximately US$ 13) per month, to contribute in building other assets within their households. This indicated that there remains a lapse in understanding of the purpose of the cash transfer where this is intended to enable the purchase of nutritious food to supplement the diets of mothers and CU2. Accordingly, it is recommended that NJP strengthens its messaging as to why the entire sum of Tk 2,200 (approximately US$ 28) needs to be invested in purchasing nutritious food for the mother and the child, as this can be life-changing for the child in terms of their healthy growth into adulthood. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 72 12. NJP needs to consider how the NJP CCT component aligns with the broader government CCT approach. Interviews with government counterparts revealed that the government health experts largely consider the sums provided by NJP to be more effective to ensure that mothers are able to purchase the food they require during pregnancy and early child care stages. However, there appears to be a need for dialogue between NJP and the Health Ministry to reduce the considerable gap between the NJP and government CCT amounts. The project also needs to consider strategies to reduce the ill-feeling between government and NJP CCT beneficiaries where the former understandably feels unhappy between the differing amounts received from the two programs. Accordingly, it is recommended that a communication strategy be developed, which can be used by both the government and NJP to explain why the difference exists between the two programs and look for avenues to reduce the tension among GoB mothers. The communication content can include explanations such as NJP being a localized project, which can only include a certain volume of mothers, whereas the GoB CCT program is nationwide, and hence limited in the amount of resources that can be spent on each mother. However, ALL mothers are included in programs such as the EPI and the GMP sessions that take place nationwide. 13. The MTE team supports the pragmatic approach taken by NJP, where the project continues to register mothers, even if they do not have personal SIM cards. The project has demonstrated flexibility to allow these mothers to be included through their spouses’ SIM cards, and at the same time continues to support these women to obtain their own SIM cards. Given the complexity of securing these cards, the MTE team recommends that the guidelines be updated to reflect this current practice. 14. The MTE recommends that the project update their guidelines to recognize the risks associated with graphical demonstrations used in SBCC-MCHN sessions and lays out criteria where these demonstrations need to be adjusted in conservative areas. The implications of this can be considerable, especially in known religious areas where NJP operates, as vested power structures may raise objections that can impede project implementation, and even put staff at risk. Gender 15. Although the project has made commendable progress in its women empowerment effort, there remains a gap where relevant gender related oversight from the government is largely missing. Accordingly, the MTE team recommends that NJP advocates for greater engagement of the government Gender Monitoring Committees engaging District and Upazila Women Affairs Officers who are based in NJP working areas. Agriculture and Alternative Livelihoods – Alternative IGA 16. The MTE team considers the engagement of Winrock and WFP, together with their national partners, for the Promotional Graduation component to be a duplication of effort and leading to confusion between partners. The MTE team also considers the differences in definition between the IGA and AIGA components does not justify why two different agencies should be involved in these components. It is recommended that given Winrock’s expertise in this area, and the fact that they already have a full technical team based in Khulna focused on AIGA, NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 73 the project considers Winrock to be wholly responsible for all IGA and AIGA components. It is also recommended that the project use one terminology – AIGA, to describe this activity, rather than using two terms which can be misleading. The term AIGA seems most appropriate as the aim of the project is to diversify the income base (both on-farm and off-farm) of its participants, which directly impacts on developing the resilience of these households. Adopting this recommendation would essentially lead to Winrock conducting the 9-month ELT, identifying suitable AIGAs and development of business plans from month six. Winrock would continue to support this activity for the remaining duration, thereby offering continuous support all through the development of the AIGA activity. This will enable dedicated support to be provided by Winrock throughout the 24-months, building on their expertise and technical and management specialists already available. This will also lead to greater efficiency removing coordination complications that are currently being experienced. It is recommended however, that WFP continue with the transfer of money through bKash, given their significant expertise in this area, and WVB continuing with the development and support to the Savings and Lending Groups. Agriculture and Alternative Livelihoods – Agriculture 17. The MTE team recommends that the Lead Farmer approach be reconsidered. The current approach is too ambitious for one Lead Farmer to develop the capacity of up to 200 other farmers through demonstration plots, Farmer Field Days and pre-season planning. A more substantial effort is required to develop the capacity of mainly subsistence farmers (12,000 Homestead Producers), as well as the 8,000 Value-Chain Producers. Approaches such as the Farmer Field and Business School (FFBS), and the Farmer to Farmer (F2F) approach can be considered by NJP as a more effective means of reaching out to its 20,000 Agriculture participants. This would enable a greater number of FFBS leaders to be developed, who could work with their groups of 20-25 farmers in the range of agricultural practices being promoted by NJP. The F2F approach can also be an effective means of reaching out to further sets of participants, such that the entire community benefits. This is important to ensure that a “population-based” impact is achieved. 18. It is recommended that NJP’s current strategy with its 12,000 Homestead Producers be reconsidered. The MTE revealed that the gains made thus far are too minimal for these activities to be sustainable. The inputs currently provided are insignificant to bring about transformational change, and likely will not be sustained after the project ends. It is recommended that the current strategy of only increasing subsistence production to contribute to household meals needs to be re-strategized where these participants are provided sufficient support to break their cycle of poverty. This may entail looking beyond what these subsistence farmers are able to produce in their homesteads, as in many cases these are too small to effectuate larger change. An interesting proposition by the Agriculture Officer based in Dacope was that NJP considers mobilizing these groups and assisting them to lease government and private land to engage in commercial production. This is an interesting proposition, and one that the MTE team recommends be explored by the project across the four Upazilas in which it operates. This can be piloted, and if successful, can be gradually scaled￾up as feasible over the remaining LOA. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 74 19. Collection Points was found to be an effective platform for participants to negotiate prices, and to access buyers. The MTE team recommends that this not only benefit Value-Chain Producers, but also includes the smaller Homestead Producers who would benefit from such Collection Points as well. Given that the current strategy primarily aims to develop Homestead Producers to be able to grow subsistence food for their household consumption – few Homestead Producers will ever reach sufficient growth to actively engage in these Collection Points for commercial benefits. Recommendation 18 above offers suggestions as to how Homestead Producers can be developed to not only provide sufficient nutrition for their household consumption, but also how they can potentially grow additional produce to actively engage in commercial value-chains. 20. Although Tower Gardens were piloted by the USAID funded CREL project in the south-west of Bangladesh, there remains a lack of evidence of how effective these have been, especially when considering the expense and labor intensity required to establish these gardens. Accordingly, it is recommended that the project reconsider its expansion strategy as the majority of poorer NJP beneficiaries will not have the means to adopt this expensive and labor￾intensive option, and accordingly reduce its target number of Tower Gardens. 21. The aquaculture component appears to be most lagging in the project. Given the challenges of increasing salinity, and the lack of options for the poorest to access water resources, it is recommended that NJP reconsiders its targets for this component and reduce these to more realistic numbers. At the same time, it is also recommended that NJP seeks partnerships with other organizations known to be leaders in aquaculture extension systems, such as World Fish, to develop joint strategies of how best aquaculture can be promoted in areas which are suffering from salinity. 22. NJP largely works in remote areas and needs to devise strategies to develop support mechanisms which are available within and around these communities. Government extension services simply do not have the resources to effectively support these areas, and hence it is recommended that NJP increases its effort to identify, train and develop local LSPs to cover agriculture, livestock/poultry, and aquaculture support needs of these communities. These LSPs should be considered as business enterprises, where they provide quality services to their catchment areas on a service-for-fee basis. It is recommended the project develop the capacities of these LSPs, and link them to government extension services, where they can make periodic visits to obtain advice and materials such as seed, vaccines and fingerlings. Should this be effectively done, local LSPs are likely to sustain, given the incomes generated by providing essential services in high demand within their catchment areas. 23. NJP’s approach of observing turkey chicks in secured poultry pens is commendable and should be continued. It is recommended that the project maintain detailed logs of when these chicks are vaccinated and perform regular monitoring checks to ensure adequate standards are being followed. Before releasing these chicks to NJP beneficiaries, the government Livestock Officer should certify these as being healthy for distribution. NJP needs to ensure that children are separated from turkeys and continue to ensure that turkeys are kept in secure netted areas, thereby reducing the risk of Environmental Enteropathy. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 75 Agriculture and Alternative Livelihoods – Promotional Graduation 24. The MTE team recommends that NJP develops an effective communication strategy to explain to participants why it offers three different types of start-up capital – Tk 1,000 per month for a total of 9-months during ELT sessions, together with a start-up capital of Tk 15,000 to Promotional Graduation participants; Tk 1,000 to 3,200 AIGA participants; and no start-up capital to 14,800 AIGA participants. NJP can approach this in a staged manner as follows: a. The first issue to resolve is to bring consistency among all the AIGA participants where it is recommended the project provides similar start-up capital to all AIGA participants. This will invariably have budget implications. Budget caps may make it unfeasible for all 18,000 AIGA participants to receive Tk 1,000 as start-up capital, in which case the project needs to assess what a more realistic start-up capital may be. At the same time, NJP needs to be careful not to reduce the start-up capital to an amount that is too insignificant to start a business. Another alternative is to promote the Savings and Lending Group concept, expanding this to AIGA groups. The Savings and Lending Group approach is a tested and proven model globally, which enables members to access start-up funds, in addition to funds provided by a project. This approach also builds on sustainability, where these groups can gain experience and confidence over the remaining LOA, and more likely to continue Savings and Lending Groups and AIGAs after project closure. b. The second issue is to explain to all beneficiaries why Promotional Graduation participants receive considerably more resources, and that this is intended to sustainably lift these people out of poverty. It is recommended that an effective communication strategy be developed in this regard. 25. Considering the early stages of the NJP Promotional Graduation intervention, it is difficult to gauge how the different approach of NJP to that of the traditional BRAC model will impact the eventual outcome. It is recommended that NJP carefully monitors this, and actively engages their learning partner to track progress, and adjust approaches through a “data-driven” management process. 26. Given that the Promotional Graduation participants comprise of 14,000 of the overall 62,137 Ultra Poor participants, the MTE team reasons that the remaining Ultra Poor participants are part of the 6-month AIGA group. Using this logic, it stands to reason that the differences between the majority of the AIGA participants and the graduation participants are minimal, as they comprise of similar socio-economic characteristics. Should this be the case, it is reasonable to assume that the literacy rates among the two groups are similar, and hence require similar ELT material and schedules. Respondents from AIGA groups were satisfied and appreciative of the 6-month ELT, which leads the MTE team to believe this is a sufficient period of time for this course. This then raises questions on the merit of conducting a 9-month ELT for the Promotional Graduation participants, when there is a proven model that a 6-month course is just as effective. It is recommended that NJP review the duration of these two courses and consider arriving at a similar timeframe of 6-months ELT for both of these groups, should this not be too disruptive to the project. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 76 Disaster Risk Reduction 27. The MTE team recommends that NJP advocates, at the National level Steering Committee, for greater resources to be allocated for the development of RRAPs and CDMAPs. Union Parishad budgets supporting RRAPs and CDMAPs should be collated, and any funding gaps be identified and presented to the Steering Committee, and convincing arguments be prepared as to the importance of why these finding gaps need to be addressed for the long-term development of these communities. It is important to have these discussions at this strategic management level as funds can only be allocated from the national budget to support these initiatives. If successful, funds can then be made available to local budgets to support these activities. 28. It is recommended that NJP works with the Disaster Ministry to fill all vacant position within DRR departments at the District and Upazila levels. This can then be discussed as a priority agenda at the NJP National Steering Committee meeting, to hold accountable the Disaster Ministry to ensure that all DRR departments are functioning, or to discuss the obstacles which are prohibiting the recruitment of a full set of staff. Identifying key issues will enable the Steering Committee to discuss remedial measures to support the Disaster Ministry, especially when considering that these structures are GoB priorities as per the SOD. 29. The Acceleration Fund, although a good initiative, is vastly under-resourced for many of the Unions in which NJP operates. The MTE recommends that this be discussed by the National Steering Committee to seek additional funding sources for Unions which are under greater threat of river erosion, water-logging and salinity. 30. Private sector engagement was found to be lacking in DRR related initiatives. The MTE recommends that the project conducts a messaging campaign in their different localities to share how private sector can be affected by disasters when support structures and communities are unprepared. Messages can focus on loss of business, and the longer-term losses recovering from disasters. These messages can serve as a motivation for local businesses to be more supportive and volunteer resources to DRR activities. Good Governance and Social Accountability 31. One of the greatest challenges to projects such as NJP, is the deliberate expansion of “unregulated” saline gher farms across the south-west of Bangladesh. Gains made by development projects will be short-lived unless active measures are taken to advocate against local power structures, who include powerful businessmen and elected officials, abusing their influence to illegally expand gher farms. It is recommended that NJP seek greater coalitions, together with the support of the World Vision country office. The MTE team does not consider NJP as an “activist” project, however, it wields considerable influence given the large number of participants it reaches, in some of the remotest and hardest to reach areas of the country. NJP and WVB are in a unique position to invite other credible organizations, actively engaged in “rights-based programming,” to form coalitions at multiple levels to influence government to take greater action to mitigate the deliberate salination of agricultural land. It is recommended that an Expert Consultant be engaged to investigate this component further, and to work with the Advocacy specialists of NJP and WVB to identify potential partners, and to develop platforms and strategies on how best to work in this advocacy arena. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 77 32. As part of the sustainability effort, government extension and support structures need to be effective and responsive to the communities they serve. To strengthen these systemic structures, it is recommended that the National Steering Committee meet twice a year. It is recommended that these meetings review action plans and budgets developed at the Upazila and Union levels, and obtain commitments from the different ministries and departments to support these local level efforts through technical and budgetary support as needed. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 78 Annexes Annex A. NJP theory of change The Nobo Jatra/“New Beginning” project (NJP) seeks to improve gender-equitable food security, nutrition, and resilience of vulnerable people within Khulna and Satkhira districts in Bangladesh using an integrated and evidence-based approach. NJP’s theory of change (ToC) is based on eight building blocks:  Increasing access to clean water and improved sanitation  Improving maternal and child health and nutrition (MCHN) practices, including infant and young child-feeding  Increasing use of climate-smart agricultural (CSA) techniques and natural resources management (NRM)  Improving income diversity, asset maintenance, and savings  Developing more alternative livelihood opportunities for youth  Increasing mobility and voice for women  Helping communities become more engaged and influencing change at national policy level  Reducing chronic poverty, with households gaining greater resilience for a truly transformative change in the lives of individuals of Khulna and Satkhira districts in Bangladesh. NJP will deliver a comprehensive support package for context and needs assessment, training, ongoing technical assistance, and accompaniment across multiple complementary sectors. Interventions will include protecting household and community assets; improving production and employment opportunities (and therefore increasing incomes); facilitating change in household nutrition and consumption habits; increasing acceptance and practice of gender-equitable norms; increasing use of MCHN practices and services; and strengthening water, sanitation, and hygiene (WASH) infrastructure and behaviors. World Vision (WV) and partners will support improvements in the abilities of households and communities to absorb shocks, adapt to change, and transform local systems both through direct implementation of activities and in coordination with state and non-state actors. Under Purpose 1 through improving utilization of WASH practices, reducing adolescent pregnancy, increasing equitable nutritious food intake, and increasing the acceptance of gender￾equitable norms, activities of the NJP will assist in improving nutritional status of children under 5 years, pregnant and lactating women, and adolescent girls. To improve utilization of WASH practices, the project will improve hygiene-related knowledge and practice through training and promotion, plus social and behavioral change communication (SBCC). NJP believes that by building knowledge and encouraging positive and healthy hygiene behavior, community members will utilize better WASH practices and will understand more fully the importance and value those behaviors bring to their health and nutrition. At the same time, NJP will support improvements in WASH infrastructure through direct construction and rehabilitation using co-investment schemes with households. To ensure ongoing maintenance of new or rehabilitated WASH infrastructure and sustainability of WASH knowledge, the project will strengthen existing local water and sanitation (WATSAN) committees as well as to form and strengthen local water management committees, using fee-for-service and viable WASH business models. NJP assumes building in community stakeholders will lead to greater sustainability of the improved NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 79 infrastructure, along with the increased hygiene-related knowledge and greater emphasis on the importance of healthy WASH behaviors. The Department of Public Health Engineering (DPHE), along with nongovernmental organizations (NGOs) (e.g., Sushishilan, Uttaran, Islamic Relief, Water Aid, Dhaka Ahsania Mission, etc.), is also working to improve the current WASH situation. It is assumed that along with the NJP they will continue their WASH interventions to achieve the purpose of improved WASH practice utilization. The project will establish linkage between community water management committees and water testing service providers that have existing water quality testing facilities, including the DPHE, the government of Bangladesh (GoB), NGO fora, and universities. It is assumed that they will continue water quality testing service for the community. Reducing adolescent pregnancy will require a shift in attitudes around child marriage in Bangladesh. The NJP believes that by strengthening child protection committees and engaging influential people and groups, all while raising awareness through a media campaign about sharing health risks inherent to adolescent pregnancy for girls and their babies, attitudes around child marriage will begin to change and girls will wait until they are older to have their first pregnancy. The assumption is with improved knowledge of the consequences of adolescent pregnancy, an external outcome will emerge, namely an increase in the utilization of family planning services directly provided by the Ministry of Health and Family Welfare (MoHFW), specifically the family planning department through field-level staff (FWA/FWV). NJP will facilitate SBCC through a mass awareness campaign on the consequences of early marriage and adolescent pregnancy, which will enable communities to understand the need of relevant existing family-planning services and its best utility. NJP will work closely with MoHFW field staff and beneficiaries will be linked with these services. Closely linked to reducing adolescent pregnancy is the increased practice of gender-equitable norms (including decision-making). The focus here is to increase the acceptance of gender-equitable norms through leadership and conflict-resolution training for women and youth and the implementation of MenCare groups in Bangladesh. Globally, WV has experienced success in supporting communities to shift gender norms through piloting MenCare groups in several countries, including India, Sri Lanka, and Armenia. These programs have developed relevant documentation outlining their approach and success, including guidelines, manuals, impact studies, and others. The NJP is working with these current projects and will leverage their documentation and support to contextualize its approach to implement MenCare groups in Bangladesh. Involvement of women, men, girls, and boys is required to shift attitudes around gender equity. In addition, the NJP will complement the activities of other service providers and actors, such as “Nari Unnyan Forum” Women Development Forum, working to reduce adolescent pregnancy. To increase equitable nutritious food intake, the NJP will focus on increasing the adoption of MCHN practices through increasing the health- and nutrition-related knowledge of caregivers. Interventions include direct education and messaging, as well as outreach to households by trained community nutrition facilitators. Simultaneously, the project will increase access to and use of health and nutrition services. NJP will be providing micro-nutrient powder (MNP) to children aged 6–23 months. Additional MNP support is available from MoHFW (policy in place) as well as other NGO actors in the area, such as through the Mother and Young Child Nutrition Security Initiative to distribute through community clinics. The NJP will not overlap with those beneficiaries; however, the assumption is that once the NJP MNP interventions end, communities will have greater knowledge of the benefits of MNP and, through their adoption of this very important feeding practice, will seek out MNP services through the MoHFW. Increased knowledge, outreach, and referrals will increase overall use of MCHN services, leading to greater knowledge of healthy practice, followed by an NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 80 increased adoption of MCHN practices. Also contributing to this change pathway are the increased production of safe, diverse, and nutritious foods and the increased diversification of livelihoods. Improved knowledge and training in positive health and nutrition behaviors will be useless if the ability to practice that knowledge (through availability and access to nutritious food) is not there. The NJP will directly link eligible vulnerable pregnant women with the conditional cash transfer (CCT) program to facilitate immediate access to nutritious food. Through a set of selection criteria, vulnerable pregnant women will be enrolled for CCT interventions, and the NJP will provide a SIM card to each of the CCT participants to withdraw cash using their mobile phone. NJP’s assumption for the CCT is that it will be used to purchase nutritious food from the local market. Influential groups, such as community clinic–support group (CC–SG), will be engaged to attend growth monitoring promotions and the administering of MNP to sensitize them on gender issues. The project will train CC–SG members on how to conduct growth monitoring throughout communities. The training is expected to build CC–SG members’ knowledge, assuming they will continue to provide growth monitoring after NJP interventions end. To increase practice of gender-equitable norms in the household, the NJP will provide life-skills training to adolescent girls in schools, conduct leadership training for women and youth, and arrange dialogue with the MenCare groups. In addition, the project will create linkages with the existing gender network, “Nari Unnyan Forum,” to ensure sustainability and reinforce messaging. The NJP will form MenCare groups in order to (1) promote an equal division of caregiving and domestic tasks between fathers, mothers, and other family members, and (2) improve and support healthy relationships between couples who support gender equality and nonviolence. Thus, childcare and feeding will be better managed. Under Purpose 2 project interventions focus on increased income and equitable access to nutritious food for both males and females. The NJP believes that by increasing the diversification of livelihoods and increasing production of safe, diverse, nutritious, and high-value foods, households will increase their income and therefore have greater opportunity to purchase and intake nutritious food. WV and partners will work to increase the diversification of livelihoods via a pathway that includes strengthened linkages to the private sector to identify and select market-based livelihoods opportunities to improve employment opportunities and ultimately increase incomes. A key activity underpinning employment and income generation is the completion of market assessments, the results of which will be used to (1) train participants in order to support entrance into viable income￾generating activities (IGAs); (2) prioritize vocational training and apprenticeships that will increase employment opportunities for youth and women; and (3) identify both high-value crops and opportunities for value addition that will raise incomes for producers, including women planting household vegetable gardens. All NJP participants will receive entrepreneurial literacy training as an underpinning intervention to increase skills on entrepreneurial development. For the extremely poor, a modified promotional graduation program will focus on first building basic skills and then serve as a feeder program into one of two pathways: increased adoption of alternative livelihoods or improved use of sustainable production practices. Graduation participants will receive a CCT to provide for their immediate needs. Conditions may include (1) attendance in entrepreneurial literacy training or (2) a business plan to start their enterprise using the CCT as a cash grant. The business will assist to adopt the alternative livelihood option as well as increase their income sources. Graduation participants will receive technical skills training with the agriculture and alternative IGA producer groups, where they will learn CSA production practices and NRM and will create service and market linkages for sustainable production practices. Through provision of a CCT, the NJP NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 81 believes graduation program participants will meet their immediate needs and therefore be allowed to graduate into participating NJP production or alternative livelihood interventions, creating sustainable options for their futures. The project also seeks to increase production of safe, diverse, nutritious, and high-value foods. The NJP will provide farmers with training in improved and CSA practices via a hands-on training modality and the use of demonstration plots. In addition, it will facilitate NRM training, agro￾production, and farm management skills to producer groups. To support adoption of new practices, the NJP will increase access to high-quality inputs via facilitating access to inputs for project participants. The NJP will also support increased and sustainable access to extension services by building the capacity of local service providers (LSPs). Improved production practices, combined with increased access and linkages to markets (described above), will ultimately provide households with the income required to sustain new practices. The livelihoods component of the NJP will be linked with different actors (private and public). The project will seek technical support from the GoB through maintaining regular linkages with the Department of Agriculture Extension (DAE), Department of Fisheries (DoF), Department of Livestock Services (DLS), and Department of Youth Development (DYD). NJP staff will facilitate building linkages between project participants and union-level sub-assistant agriculture officers (SAAOs), LSPs, and local farmers. CSA demo plot sites will finalize in collaboration with the DAE. DoF: upazilla-level fishery officer will be one of the key technical persons for selecting fish variety and technical support. GoB fish nursery and hatchery will be the main sources of quality fingerlings. DLS: upazilla-level livestock department will link to NJP staff and LSPs for technical issues and sources of poultry and livestock vaccination. The DYD will help to identify market-based trade for alternative livelihoods option. The NJP will customize DYD existing training materials and use their resources. To be sustainable, communities need direct linkages with government-line departments, private agro-input companies, local market actors, and other institutions to ensure ongoing support and a positive operating environment, as well as to facilitate access to alternative sources of technical assistance. To participate effectively in markets, smallholder farmers must overcome certain constraints. The NJP will assist smallholders to overcome these barriers by helping to integrate farmers into agricultural markets—both as consumers of products and services and as producers of saleable crops—with the end result being increased incomes and improved livelihoods. The NJP will utilize the three-part market framework for the implementation of the agriculture and livelihoods component. The framework combines the three integral components of the smallholder market environment: input market, small farm, and output market. LSPs pertaining to these three parts will be identified, including agro-input (seed/fertilizer, fingerlings, feed, chemicals) sellers, para-vets, lead farmers, and aggregators. They will be trained on how to provide timely quality goods and services to small, poor producers. These trained LSPs will then be linked with producer groups to participate in the market as consumers of inputs in order to produce crops for home consumption and sale to output market. The output market will provide market information and credit required to move the small farm production from field to market at economically rewarding prices. The NJP will develop 300 LSPs/local market actors (e.g., seed retailer, nursery owner, para￾vet, aggregators, etc.) who will conduct business with embedded services in the community. Some 150 trained lead farmers will remain in the community and act as community-based service providers and foster horizontal linkages. After it is phased out, the NJP assumes trained lead farmers, service providers (fee-based services), and seed/fingerling suppliers will take over the roles of NJP staff and continue running their businesses on their own. It is also assumed they will maintain relationships NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 82 with government departments as well as with large companies to continue growing. This assumption evolves from project beneficiaries learning how to ask for required services, how to apply the services for better production and marketing, and continuing to expand their businesses as a result of project interventions. Supply and demand will therefore continue within the market, leading to increased production and sales and thereby increasing incomes for beneficiary households. As a mechanism for integrating the poor/small producers in the mainstream market, 75 collection points will be established and production clusters will be developed surrounding them to enhance the production and supply of vegetables. Besides these, a good number of wholesalers (paikers) and brokers (forias) will be linked with the collection points, thus ensuring product sales. The NJP will work to strengthen a “firm-to-farm/firm-to-farmer” approach on the ground. This relationship will be emphasized further to stimulate effective interactions between farmers (lead farmers and beneficiary as well as non-beneficiary farmers) and private companies to strengthen the businesses of both parties. As a result of NJP interventions, private companies marketing agricultural products will penetrate into new customer bases through the mechanism developed by the NJP, such as lead farmers, para-vets, and trained seed/feed/fingerling sellers. Large fast-moving consumer goods companies will extend their supply chain to reach the bottom of the pyramid market by developing “last-mile vendors” (very local village-level vendors). Interested beneficiaries will be trained in skills required to become effective local vendors and linked with the network of the companies, resulting in income generation for their families. Vocational training to youth on their self-selected trades through government and private sector institutes and linkages with potentials employers will also help to generate income. Under Purpose 3 through improving the capacity of government institutions who are mandated for delivering disaster risk reduction (DRR) services, as well as improving the capacity of the targeted communities, the NJP will support the strengthening of gender-equitable ability of people, households, communities, and systems to mitigate, adapt to, and recover from natural shocks and stresses. To strengthen government institutions for better services, emphasis will be placed on the capacity building of disaster management committees (DMCs) for their respective functions to plan and implement RRAP and the Comprehensive Disaster Management Action Plan. The project will strategize training and technical assistance interventions for efficient and effective implementation of DRR services. To that end, the NJP intends to promote collaborative efforts and joint responses of public and private actors, and will provide block grants to accelerate the implementation of DRR services. The NJP will facilitate coordinated and focused support of multiple actors—most importantly households and communities themselves. The pivotal role of local populations will be enhanced by:  Increasing DRR knowledge of the community in targeted areas  Engaging local volunteers in the community to prepare for and respond to disasters  Mobilizing village development committees (VDCs) (male and female) to participate and oversee RRAP development and implementation by DMCs Through these measures the NJP will improve the functionality of local-level governance structures that can effectively plan and coordinate resilience-building activities to effectively target and benefit all segments of society, support modification and adaptation of community-based plans, and facilitate meaningful coordination with district and regional governance structures. Under Purpose 4 (crosscutting), the NJP aims to improve social accountability and national policy engagement of service provision for vulnerable men and women. It will do this through increasing NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 83 the capacity of the government to implement pro-poor and gender-equitable policy and increasing the capacity of poor and extremely poor to influence local decision-making bodies. To increase responsiveness of market-based local service providers (WASH and agriculture), the NJP will link LSPs with the government service institutions so that they can meet the needs of vulnerable people. To increase responsiveness of government service providers (health and nutrition, DRR, ag. extension, social protection), the NJP will engage in evidence-based, national￾level policy dialogue to create an enabling environment for sustainable impact by demonstrating and influencing changes to critical GoB DRR, resilience, climate change, and safety net programs and policies. A good number of LSPs in the WASH and agriculture sectors will be associated with the project, providing goods/inputs and technical services to meet the local community needs and demands in project locations. The NJP assumes that upazilas-level WASH and agricultural sectoral government departments will ensure regulatory control for maintaining quality services of LSPs and create an enabling environment for them to access the local market and serve community needs. The NJP also assumes that VDCs will remain within their community after the project ends and that community members will be empowered and informed about their service entitlements through application of the community voice and action (CVA) process. In addition, five relevant Union Parishad Standing Committees (UPSCs) would be reactivated/ reformed through NJP interventions. The five UPSCs are: 1. Education, Health, and Family Planning 2. Agriculture, Fisheries, and Livestock 3. Water Supply, Sanitation, and Sewerage/Drainage System 4. Social Welfare and Disaster Management 5. Family Arbitration, Women. and Child Welfare In doing so, communities would be equipped with Union Parishad groups who will work to ensure that communities receive their service entitlements. Well-trained VDC members would remain in the community and be linked with government service providers. Through strengthened inclusive VDCs, via capacity building and training in gender and gender-based violence, CVA, good governance and leadership, and others, the NJP will ensure the existence of a local-level governance body that can effectively plan and coordinate resilience-building activities that appropriately target and benefit all segments of society. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 84 Annex B. Evaluation Matrix The evaluation matrix will help the MTE team address each of the key evaluation questions in the TOR. It provides an overview of how the evaluation questions will be addressed, including: Sub￾questions; A set of indicators to measure performance, Links to the relevant parts of the methodology that will contribute to answering the sub questions. The matrix also describes how the findings from each of these will be triangulated. Finally, the matrix provides sources of information (specifying whether secondary data will be used and where primary data is needed). AREAS OF FOCUS ASPECTS THAT WILL BE EVALUATED INFORMATION SOURCES METHODS OF DATA COLLECTION DATA ANALYSIS METHODS 1. How well have the activity’s interventions met planned schedules, beneficiary numbers, and Outputs? What factors promoted or inhibited adherence to schedules? How were problems and challenges managed? Adherence to planned schedules • Start dates and rates of expansion of coverage for each intervention • Numbers and timeliness of planned beneficiaries and Outputs (e.g.,): - Formative research, barrier analyses, gender analysis - Various types of direct trainings - Indirect training through trainees, e.g., farmer to farmer or cascade training - Formation of or connections with community groups - Construction or rehabilitation of assets - Development and progress of community action plans - Distribution of cash and goods • Logframe • Indicator Performance Tracking Table (IPTT) • ToC • Baseline • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff • Govt. officials • M&E Coordinator • CoP/DCoP • Interview members of activity staff at various levels about factors that delayed or interrupted interventions and Outputs, and how problems were identified and managed. • Ask groups and individuals from different stakeholder groups at locations of greater and lesser achievements about factors they believed inhibited or promoted efficiency and efforts have been made to overcome barriers. • Literature review • FGDs • KIIs • Use secondary data from routine monitoring, annual results reports, and other reports to compare planned and actual start dates, numbers of outputs, and other targets, noting differences in achievements according to location, implementing partner, or sector. • Compare across locations, beneficiary groups, activity administrative units, etc. to identify factors associated with differing degrees of achievement. 2. What are the strengths of and challenges to the overall activity design, implementation, management, communication, and collaboration so far? What factors appear to promote or challenge the activity operations or effective collaboration and cooperation among the various stakeholders? Activity management • Strengths and weaknesses of the activity work plan and schedule • Evidence that management has explored and implemented new and/or innovative ideas and approaches • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff • Interview members of management about outcomes of work plan reviews and how they handled changes and challenges that presented. • Ask implementing staff in different roles how feedback and ideas are solicited and • Review the activity work plan and schedule to assess how completely and clearly they define the work needed to meet objectives, when, and by whom. Is the schedule feasible? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 85 AREAS OF FOCUS ASPECTS THAT WILL BE EVALUATED INFORMATION SOURCES METHODS OF DATA COLLECTION DATA ANALYSIS METHODS • Changes and challenges in the operating context and how management responded shared within and among partners, especially among field, country office, & headquarters. • Literature review • FGDs • KIIs • Examine the roles of the different implementing partners and how the plan promotes good collaboration among them and leverages partners’ relative advantages. Staffing • Adequacy of numbers and capacities (knowledge, experience) from beginning of the activity until present • Strengths and weaknesses of supervision and support to ensure accountability, performance, and confidence among implementing staff • Adequacy or inadequacy of resources (tools, work space, transportation, communication, information, work aids) to support interventions’ efficient performance at all times from start to current time • Gender sensitivity and balance at various levels of staff • Conflict sensitivity • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff Interview a sample of field staff and supervisors in different sectors and interventions and at different levels about: • Factors that affect their performance and motivation • Sources of satisfaction and dissatisfaction with resources, training, supervision, and support from technical experts • Their roles in decisions about intervention design and implementation • Training they received • Literature review • FGDs • KIIs • Review the characteristics and capacities of staff at all levels in all sectors, and assess their confidence and capacities to perform assigned tasks. • Review training and supervision schedules for monitoring and supporting implementing staff, including an assessment of the numbers of people and sites per supervisor. Community engagement and participation • Strengths and weaknesses of the ways the activity has included community members, including vulnerable or marginalized members, in decisions about intervention choices, design, implementation, and monitoring • Community members’ perceptions about their participation in the activity and the • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff Interview groups and individuals from different stakeholder groups about: • Who has been involved in the activity and how? • Who else would have liked to be involved, and how? • Satisfaction and dissatisfaction with the way activity staff and community members engaged • Reports of or potential for exploitation or NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 86 AREAS OF FOCUS ASPECTS THAT WILL BE EVALUATED INFORMATION SOURCES METHODS OF DATA COLLECTION DATA ANALYSIS METHODS degree and nature of their engagement with activity staff • Use of incentives of different types • Safeguards against exploitation and discrimination • Application of “do no harm” principles discrimination by implementing staff or with activity resources • Interview volunteers who contribute significant amounts of unpaid time to activity-supported interventions to learn what motivates their participation and performance. • Literature review • FGDs • KIIs Communications • Quality and timeliness of communications of vision, objectives, plans, implementation guidelines, and other activity information among activity staff, partners, government counterparts, and communities • Knowledge in various stakeholder groups about the activity • Strengths and weaknesses of the ways the activity encourages and handles feedback from community members, staff, and partners • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff Interview members of implementing partners, communities, government counterparts, and other stakeholder groups to assess: • Knowledge of objectives, interventions and implementation, intervention duration, eligibility, outputs, and entitlement transfers • How and when they learned about activity objectives and interventions • Frequency and content of communications with other types of stakeholders • Satisfaction and dissatisfaction with ways feedback is received and responded to • Literature review • FGDs • KIIs Partnerships and linkages • Consider collaboration and links with: • Other USAID activities • Government activities • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members Interview implementing staff, government counterparts, members of CBOs, and staff of linked or collaborating activities about: NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 87 AREAS OF FOCUS ASPECTS THAT WILL BE EVALUATED INFORMATION SOURCES METHODS OF DATA COLLECTION DATA ANALYSIS METHODS • Community based organizations • Other complementary activities in the activity area • Strengths and weaknesses of coordination within the activity and between the activity and other activities and agencies • Factors that make partnerships more or less beneficial to activity implementation • Project staff • PNGO staff • The nature and sources of satisfaction and dissatisfaction with their collaboration and ways that it could be improved • How they feel their cooperation benefits the implementation and results on both sides • Other activities, agencies, and groups that are doing similar or complementary work to which the activity is not linked • Review samples of activity’s memorandums of understanding with collaborators • Literature review • FGDs • IDIs • KIIs Financial management • Financial accountability • Sufficiency of finances to ensure good activity implementation • Flexibility of the budget to respond to changing conditions • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff • Review financial records • Interview managers about: - The adequacy of finances and effects of financial constraints on activity implementation - Perceived limits of financial flexibility to respond to change • Literature review • FGDs • KIIs Branding • Compliance with USAID policy • Knowledge and attitudes toward donor and implementers within target communities • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff • Interview beneficiaries and community leaders about their knowledge of and attitudes toward USAID and implementing partner agencies • Literature review • FGDs Assess how well planned and actual actions do or do not comply with USAID branding requirements. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 88 AREAS OF FOCUS ASPECTS THAT WILL BE EVALUATED INFORMATION SOURCES METHODS OF DATA COLLECTION DATA ANALYSIS METHODS • KIIs M&E • Completeness and clarity of the documented M&E Plan • Ways the system: - Collects data useful to monitor the quality and outputs of processes - Solicits and reports opinions, ideas, and concerns from field staff - Provides constructive feedback to implementing staff to inform, assist, and ensure accountability and motivate good performance - Ensures accurate reporting to USAID - Supports timely problem solving and decision making for all stakeholders - Ensures data quality: validity, reliability, timeliness, integrity, and precision - Has been used to adjust implementation - Is or is not supported by the institutional structures - Monitors environmental impact - Monitors gender equity - Monitors context - Monitors unintentional results (positive and negative) - Strengths and weaknesses of data collection methods - Design; management; and roles in monitoring, • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff • Interview staff in various roles in the collection, analysis, and reporting of routine monitoring about their activities and roles, to determine their understanding and confidence in the data collected, and challenges they face getting or using the data. • Interview recipients of reports and other outputs about how they use the information they receive, which information is most useful, the timeliness of the information, and any other information they would like to have. • Literature review • FGDs • IDIs • KIIs • Review the M&E Plan and systems: staffing, processes, and outputs. • Interview key decision makers about the timeliness and usefulness of the data from the M&E system. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 89 AREAS OF FOCUS ASPECTS THAT WILL BE EVALUATED INFORMATION SOURCES METHODS OF DATA COLLECTION DATA ANALYSIS METHODS analysis, and report generation of data bases. - Challenges the M&E team faces Environmental safeguards and compliance • Adequacy of the Environmental monitoring and mitigation plan (EMMP) • Adherence to the details of the EMMP through specific environmental monitoring systems • Incorporation of the EMMP into the IPTT and annual monitoring processes • Recognition or avoidance of unforeseen environmental damage and climate stressors • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff • Interview technical experts, implementing staff, and other key informants about activity interventions’ apparent or potential threats to the environment and identify those not addressed by the EMMP and how well the activity implementation has addressed these threats. • Literature review • FGDs • KIIs Examine how well planned and actual actions and outputs do or do not comply with the activity’s EMMP. Commodity management • Division of roles and cooperation among activity staff, external partners, and community members • Common causes of delivery delays and commodity losses • Adequacy of mechanisms to safeguard against loss or abuse • Adequacy of mechanisms to ensure adherence to “do no harm” principles • Quality and cleanliness of storage facilities • Completeness and clarity of commodity records and reports • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff • Interview people managing the resources about what is and is not working well, why, and how problems have been addressed. • Literature review • FGDs • KIIs • Review processes and records related to tracking, management, and delivery of commodities to points of use or distribution: identify challenges and measures taken to address them. • Inspect warehouses and storerooms of various sizes and managed by different entities. Exit and sustainability strategies • Comprehensiveness of the exit and sustainability strategies • Factors that threaten the continuation of targeted practices and services and maintenance of new infrastructure • Logframe • IPTT • ToC • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff • Interview key informants and beneficiaries about threats and promoters of targeted practices and infrastructure. • Literature review • KIIs Review the exit and sustainability strategies and progress in its implementation in light of the findings related to the challenges to practices promoted by and threats to infrastructure NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 90 AREAS OF FOCUS ASPECTS THAT WILL BE EVALUATED INFORMATION SOURCES METHODS OF DATA COLLECTION DATA ANALYSIS METHODS • Progress in implementing the strategy • Ways the activity is strengthening or establishing links between communities and private or public financial or technical resources developed by the activity. 3. In each technical sector, what are the strengths of and challenges to the efficiency of interventions’ implementation and their acceptance in the target communities? How well do implementation processes adhere to underlying principles and activity protocols? What factors in the implementation and context are associated with greater or lesser efficiency in producing Outputs of higher or lower quality? Which interventions and implementation processes are more or less acceptable to members of the target communities and why? • Behavior change communication • Direct and indirect training • Health and nutrition • Water, sanitation, and hygiene • Agricultural production • Application of findings from formative research and gender analyses to implementation • Technical quality of activity Inputs and Outputs • Strengths and weaknesses of how the various interventions engage target groups and protect against unintentional harm • Annual Reports • Lessons Learned docs • M&E Reports • Project staff • PNGO staff • Project beneficiaries • Observe interventions (training sessions, distributions, construction, community meetings, FSS or care group sessions) and talk with implementing staff and direct beneficiaries about: • What interventions are more and less effective • What and how could interventions be improved • Literature review • Direct observations • FGDs • KIIs Review formative research and evaluate how well implementation has applied the findings. • Income generation • Savings and loans • NRM and environmental protection • Women’s empowerment and promotion of gender equity • Prevention of gender-based violence • Early warning systems and DRR • Selection of direct beneficiaries; coverage of target groups • Perceptions of quality, appropriateness, and use of distributed goods and promoted services • Composition, activities, and governance of groups created or promoted by the activity • Networks and connections facilitated by the activity • Collaboration with and support to relevant • Annual Reports • Lessons Learned docs • M&E Reports • Project staff • PNGO staff • Interview beneficiaries and assess the extent to which interventions are more or less interesting or useful • Who benefits; who should benefit; how are beneficiaries selected • Opportunity costs of participation in interventions • Knowledge and understanding of key activity messages • Talk with non￾beneficiaries from • Determine the perceptions about the benefits they could gain with participation in interventions • Compare and contrast men’s and women’s participation and perceptions. • Review messages on the same topic transmitted through different pathways for consistency and clarity. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 91 AREAS OF FOCUS ASPECTS THAT WILL BE EVALUATED INFORMATION SOURCES METHODS OF DATA COLLECTION DATA ANALYSIS METHODS • Community governance • Food/cash for work government service providers • Cultural acceptability and relevance of intervention methods and messages • Consistency of content and recipients’ understanding of similar messages received via different pathways • Strengths and weaknesses of measures taken to ensure gender equity with regard to access to, participation in, and benefit from activity interventions • Strengths and weaknesses of linkages, coordination, and integration among the different sectoral and crosscutting technical areas • Validity and comprehensiveness of assumptions in the activity’s ToC that are critical to intervention implementation and outputs the same communities about: • Which interventions are more or less interesting or seem more or less useful • Who benefits; who should benefit; how are beneficiaries selected • Talk with members of groups formed or promoted by the activity about: • How the group was formed; level of satisfaction with group composition • Challenges and successes working as a group • Nature and adequacy of support from program • How and why the group chooses interventions • How members’ other roles in the community affect their participation in the group • Examine beneficiary records to assess the proportion of households and communities that benefit from multiple sectors in different combinations: • Talk with program staff to understand who was targeted for multiple sectors and why. • Interview members of households benefiting from interventions in single and multiple program sectors about their participation; compare characteristics of those who benefit from one vs. multiple sectors. • Compare the understanding of the key messages of trainers and direct and indirect trainees. • Inspect the technical quality of community and household infrastructure and natural resources to which the activity contributed. • Examine the composition of the various groups created or supported by the activity: Who in the community did or did not join. Why or why not? How is the gender balance? Are marginalized groups represented? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 92 AREAS OF FOCUS ASPECTS THAT WILL BE EVALUATED INFORMATION SOURCES METHODS OF DATA COLLECTION DATA ANALYSIS METHODS • Interview staff members about their interactions with staff working in other sectors, especially regarding site and beneficiary selection and developing messages to beneficiaries. • Interview members of various types of groups initiated by the activity about, e.g., making decisions, managing joint resources, and sharing information and experiences. • Literature review • KIIs • FGDs 4. What changes—expected and unexpected, positive and negative—do community members and other stakeholders associate with the activity’s interventions? What factors appear to promote and deter the changes? How do the changes correspond to those hypothesized by the activity’s TOC or RF? Changes observed or reported • The activity’s ToC or RF • Intended and unintended change • Positive and negative change • Differential change among beneficiaries (individual, community) of one sector, beneficiaries of multiple sectors, and non-beneficiaries • Differential change among beneficiaries representing different population subgroups • Perceived benefits of participation in interventions from multiple sectors vs. a single sector • Perceived trajectory of change and conditions that threaten or promote sustained change • Changes in conditions related to assumptions • Project documents • Lessons learned docs • Annual Reports • Community members • Project staff • PNGO staff • Govt. officials • Interview community members (beneficiaries of one or more sectors and non-beneficiaries) and activity staff to gain perspectives about: • Changes they have made themselves, observed in others, or observe in the social, economic, or physical environment • Factors that promoted the changes • Barriers to changes intended by the activity • Conditions that promote or threaten sustained change • Technically evaluate how strategically selected infrastructural outputs affect or can affect livelihoods, well-being, or NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 93 AREAS OF FOCUS ASPECTS THAT WILL BE EVALUATED INFORMATION SOURCES METHODS OF DATA COLLECTION DATA ANALYSIS METHODS environmental conditions. • Literature review • FGDs • IDIs • KIIs 5. Based on the findings from Questions 1–4, how could the activity be modified to improve its acceptability to targeted communities or the efficiency and effectiveness of its implementation? How should the activity’s TOC or RF be refined or modified? Based on findings from 1–4 above • Observed and perceived strengths and weaknesses of the implementation so far • Factors in the design, implementation, and context that affect the efficiency or acceptability of the processes, outputs, and intermediate outcomes • Targeted communities’ and individuals’ perceptions and priorities • Relative cost and feasibility and anticipated value of acting and benefiting within the life of the activity • Potential to advance the activity’s ultimate objectives and goal • Annual Reports • M&E Reports • Community members • Project staff • PNGO staff • Govt. officials • Literature review • FGDs • KIIs • Contribution analysis • Use the results of inquiries to the questions above to form conclusions and recommend concrete actions to help improve activity performance and final results. • Prioritize the recommendations and identify the actor(s), the purpose for change, and anticipated benefits. All recommendations should be directly related to stated conclusions and based on evidence presented as findings NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 94 annex C. Data collection tools 1. Interview Guide for Service Providers and Local NGOs (Nutrition, WASH, Health) Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Jatra Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date: Name of interviewee Title Name of organization Union Parishad: District: Discussion guide for various service providers providing various type of services in the program intervention zones. 1- Can you tell us a bit about your organization? - When was it formed? - By who? - How are you organized? - What are your domains of intervention? - Who owns it now? 2- Can you tell us about [thematic area 1] that is one of your focus area. What is the situation like in the intervention zone? 3- What are the most important issues and challenges that people face , with regard to [thematic area 1] in this zone? 4- What solutions has your organization been offering regarding those issues? 5- What is the uptake like? 6- What is preventing more people from adopting those solutions? 7- What about [thematic area 2]? 8- What are the most important issues and challenges that people face , with regard to [thematic area 2] in this zone? 9- What solutions has your organization been offering regarding those issues? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 95 10- What is the uptake like? 11- What is preventing more people from adopting those solutions? 12- Have you been in contact with the NJ staff? When did that start? 13- What are the activities you are involved in with the NJ program? 14- Do you have an idea of the number of people/households who have directly benefitted from your interventions in the NJ program? 15- Have you received any training or support from the NJ program? 16- If Yes what did you receive? 17- How did those items help you reach more people? 18- How did those items help you offer better services? 19- Are there anything that you have received (trainings or equipment) and that you feel you do not need at this time for some reason? 20- If Yes What are they? 21- What have been your major successes with that collaboration? What has worked really well? 22- What improvements would you like to see in the collaboration with the NJ program? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 96 2. Interview Guide for EPI (Extended Program for Immunization) Volunteer Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Complete the list of attendees Union Parishad: District: 1. Since when have you started to work as an EPI volunteer? 2. What activities have you done so far with the NJ staff in immunization? - What training have you received - When was that? - What follow-up contact (supervision) did you have? How often does that happen? By who? 3. As a EPI volunteer, what activities have you done (or do you normally perform in your community? - Activity 1 - Activity 2 - Activity 3 - Activity 4 4. For each of these activities, what is the expected objective? What are you trying to achieve? - Objective for Activity 1 - Objective for activity 2 - Objective for Activity 3 - Objective for activity 4 5. Since you started these activities can you tell me about three successes you have had so far (what have you achieved) - Success 1 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 97 - Success 2 - Success 3 6. What were the key drivers to those successes? What made it possible? 7. What are the top three issues that prevent mothers from immunizing their children? 7.1 issue 1 7.2 issue 2 7.3 issue 3 8. What are you doing about that? 9. What strategies have been put in place with NJ suport for which you can see some changes taking place? 10. What are the strategies that are not working as expected? 11. How satisfied are you with your collaboration with NJ and the service you give in your community? 12. How long do you plan to be an EPI volunteer? 12.1 Any reason for that? 13. What would you recommend the NJ program to focus on over the next years to increase immunization coverage in your zone? 13.1 Recommendation 1 13.2 Recommendation 2 13.3 Recommendation 3 14. Do you have any other comments about the NJ program in your community? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 98 3. Interview Guide for Mothers and Caretakers Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Complete the list of attendees Union Parishad: District: Attention: Mothers and caretakers may not know that the goods or services they have received are coming from NJ. In that case NJ program should be replaced with the name of the local NGO which has been in contact with them. 1- Since when are you in contact with the NJ program/local NGO? 2- What services or goods have you directly received from the NJ program/local NGO?? 3- How were you selected to benefit from those goods/services? 4- What do you think was the purpose of the goods or services? 5- How useful were the goods or services you received? 6- Have you received any training in nutrition (formal or informal ) from the NJ program/local NGO? staff?  If Yes list them here  5.1 training 1  5.2 training 2  5.3 training 3 7- What were the objectives of each of those trainings? 8- What do you think about those trainings? 9- What have you learned from the trainings and that you currently apply? 10- What did you choose not to apply? Why? 11- Have you received any training in wash (formal or informal ) from the NJ program/local NGO staff?  If Yes list them here  10.1 training 1 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 99  10.2 training 2  10.3 training 3 12- What were the objectives of each of those trainings? 13- What did you think about those trainings? 14- What have you learned from the trainings and that you currently apply? 15- What did you choose not to apply? Why? 16- Have you received any training in health (formal or informal ) from the NJ program/local NGO staff?  If Yes list them here  15.1 training 1  15.2 training 2  15.3 training 3 17- What were the objectives of each of those trainings? 18- What did you think about those trainings? 19- What have you learned from the trainings and that you currently apply? 20- What did you choose not to apply? Why? 21- Have you received any training in agriculture production or processing (formal or informal ) from the NJ program staff?  If Yes list them here  21.1 training 1  21.2 training 2  21.3 training 3 22- What were the objectives of each of those trainings? 23- What did you think about those trainings? 24- What have you learned from the trainings and that you currently apply? 25- What did you choose not to apply? Why? 26- What is your satisfaction level with the services that you have received from the NJ program/local NGO so far? 27- Why do some women are not receiving the same goods/services in your community? 28- Have you received any cash or grant from the NJ program/local NGO? 29- If yes what was it for? 30- How were you selected? 31- What did you use the cash for? 32- What change did that cash/grant make in your live and the life of your household? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 100 33- What can you suggest that NJ program to do differently in order to reach more women in your community? 4. Focus Group Discussions with Members of WATSAN Committees Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date: List of attendees Union Parishad: District: 1- Tell me a little bit about your group? When was it formed? Membership? Objectives? 2- What are the main problems faced by your community members when it comes to water and sanitation?  2.1 Problem 1  2.2 Problem 2  2.3 Problem 3- When was the first time you have been in contact with the NJ program? 4- What training have you received from the program staff? When how long? 4.1 training 1 4.2 training 2 4.3 training 3 5- What did you choose to apply from the training?  5.1 Item 1  5.2 Item 2  5.3 Item 3 6- Is it working? 7- Why or Why not?  7.1 reason 1  7.2 Reason 2  7.3 Reason 3 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 101 8- What did you choose not to apply from the training?  8.1 Item 1  8.2 Item 2  8.3 Item 3 9- Why? 10- How do community members access the water/sanitation point? Do they have to pay? If yes how much? 11- Are they using it as expected? 12- What changes do you think the water/sanitation points put in place with the help of the NJ program are making in the lives of the community members?  12.1 Change 1  12.2 Change 2  12.3 Change 3 13- Now how often do you meet with the program staff? 14- What linkage do you have with other WATSAN committees? 15- What do you think could be done by the NJ program or the government to improve community access to water/sanitation?  15.1 Suggestion 1  15.2 Suggestion 2  15.3 Suggestion 3 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 102 5. Interview Guide for Caretaker of Water Points/ Sanitation Points Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date: Name of interviewee Gender: Contact (not mandatory): Union Parishad: District: 1- Since when are you taking care of this water/sanitation point? 2- What did you have to do to be granted the management of this point? 3- Who put in place this water/sanitation point? 4- What are the main problems faced by your community members when it comes to water and sanitation?  4.1 Problem 1  4.2 Problem 2  4.3 Problem 5- When was the first time you have been in contact with the NJ program? 6- What training have you received from the program staff? When how long? • 6.1 training 1 • 6.2 training 2 • 6.3 training 3 7- What did you choose to apply from the training?  7.1 Item 1  7.2 Item 2  7.3 Item 3 8- Is it working? 9- Why or Why not? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 103  9.1 reason 1  9.2 Reason 2  9.3 Reason 3 10- What did you choose not to apply from the training?  10.1 Item 1  10.2 Item 2  10.3 Item 3 11- Why? 12- How do community members access the water/sanitation point? Do they have to pay? If yes how much? 13- Are they using it as expected? 14- What changes do you think this point is making in the lives of the community members?  14.1 Change 1  14.2 Change 2  14.3 Change 3 15- What costs/expenses do you have to bear when you manage a sanitation/water point?  15.1 Cost 1  15.2 Cost 2  15.3 Cost 3 16- Is the activity profitable enough? 17- What could be done to increase demand from community members?  17.1 Suggestion 1  17.2 Suggestion 2  17.3 Suggestion 3 18- Now how often do you meet with the program staff? 19- What linkage do you have with other managers of water/sanitation points in your district? 20- What do you think could be done by the NJ program or the government to improve community access to water/sanitation?  20.1 Suggestion 1  20.2 Suggestion 2  20.3 Suggestion 3 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 104 6. Interview Guide for UPSC—Union Parishad Standing Committee Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : List of participants Union Parishad: District: 1. Can you tell us a little bit about your UPSC? 1.1 Membership: male/female 1.2 Objectives 1.3 Role 2. How often do you meet? What do you discuss during those meetings? 3. Are you aware of the Nobo Jatra activities in your UP? 4. What have been the activities implemented by the NJ program in your UP? 5. What communication channel does the NJ program has with your committee? 6. Do you know how the program interventions have been identified in your UP? If Yes how was it? 7. Do you know how the program individual beneficiaries have been selected? Were there any shared criteria? Was the management of the UP involved in the selection? Were the process fair? 8. How were the most vulnerable people in the UP included in the program activities? 9. Do you have an agreed complaint mechanism regarding the NJ intervention in your UP? 10. If yes, has the UP ever used it? What were the results? Were you satisfied? 11. What about the individuals living in the UP? 12. What do you see as the main success of the NJ program in your UP? 13. Why? 14. What do you see as the main challenges/problems with the NJ program in your UP? 15. Why? 16. Is there any change of your Union Parishad that is attributable to the NJ program? i. (Wash ii. MCHN iii. Food security iv. Resilience and Disaster reduction v. Financial literacy? vi. …) NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 105 17. What are the challenges you often face being a being Union Parishad member? 18. What would you recommend the NJ program to focus on in your UP for the remaining of the program? Where are the most pressing needs? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 106 7. Interview Guide for VDC/Community Groups/Elite Groups and Community Support Groups Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Complete the list of attendees Union Parishad: District: 1. Since when has your VDC been formed? 2. How many members do you have? Who are those members? 3. How do you become a member of the VDC? Any criteria to fulfill? 4. How are you organized? 5. What are the main development issues your community members are facing? 6. What have you started to do to address those issues? 7. Who are the main development actors who are (or who have) helped you address those issues? How? 8. When did you start working with NJ program? 9. What activities have you done so far with the NJ staff? 10. What training have they received 11. When was that? 12. How many people were trained? 13. What follow-up contact (supervision) have they had? How often does that happen? By who? 14. As members of VDC, what activities have you done (or do you normally perform in your community? 15. Activity 1 16. Activity 2 17. Activity 3 18. Activity 4 19. For each of these activities, what is the expected objective? What are you trying to achieve? 20. Objective for Activity 1 21. Objective for activity 2 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 107 22. Objective for Activity 3 23. Objective for activity 4 24. Since you started these activities can you tell me about three successes you have had so far (what have you achieved) 25. Success 1 26. Success 2 27. Success 3 28. What were the key drivers for success per success? What made it possible? 29. What are the most important development challenges members of your community are facing? 30. Challenge 1 31. Challenge 2 32. Challenge 3 33. Challenge 4 34. For each challenge, ask the following five questions:  Why do you think [cite a challenge] is still prevailing in your community?  What are you doing about that?  Is it working?  Do you have any examples of noticeable change that you can share about that? 34. What would you recommend the NJ program to focus on over the next years to increase your engagement/interest in the program activities? 35. Recommendation 1 36. Recommendation 2 37. Recommendation 3 38. Do you have any other comments about the NJ program in your community? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 108 8. Key Informant Interviews for Community Leaders Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Name of interviewee Contact (not mandatory): Union Parishad: District: 1. Are you aware of the Nobo Jatra activities in your community? 2. What have been the activities implemented by the NJ program in your community? 3. What communication channel does the NJ program has with you and your peers? 4. Do you know how the program interventions have been identified in your community? If Yes how was it? 5. Do you know how the program individual beneficiaries have been selected? Were there any shared criteria? Was the leaders of the community involved in the selection? Were the process fair? 6. How were the most vulnerable people in the community included in the program activities? 7. Do you have an agreed complain mechanism regarding the NJ intervention in your community? 8. If yes, have you ever used it? What were the results? Were you satisfied? 9. What about the individuals living in the community? 10. What do you see as the main success of the NJ program in your community? 11. Why? 12. What do you see as the main challenges/problems with the NJ program in your community? 13. Why? 14. Is there any change of your community that is attributable to the NJ program? i. (Wash ii. MCHN iii. Food security iv. Resilience and Disaster reduction v. Financial literacy? vi. Livelihood? vii. …)economic empowerment? viii. Gender? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 109 15. Any suggestion that we can relay to the NJ program in order to increase its benefits for your community members? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 110 9. Key Informant Interviews with Government Officials KIIs with UP Administrator (Chairman/Secretary/Women Member) Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Name of interviewee Gender: Contact (not mandatory): Union Parishad: District: 1. Since when are you administering this UP? 2. Are you aware of the Nobo Jatra activities in your UP? 3. What have been the activities implemented by the NJ program in your UP? 4. What communication channel does the NJ program has with the management of your UP? 5. Do you know how the program interventions have been identified in your UP? If Yes how was it? 6. Do you know how the program individual beneficiaries have been selected? Were there any shared criteria? Was the management of the UP involved in the selection? Were the process fair? 7. How were the most vulnerable people in the UP included in the program activities? 8. Do you have an agreed complain mechanism regarding the NJ intervention in your UP? 9. If yes, has the UP ever used it? What were the results? Were you satisfied? 10. What about the individuals living in the UP? 11. What do you see as the main success of the NJ program in your UP? 12. Why? 13. What do you see as the main challenges/problems with the NJ program in your UP? 14. Why? 15. Is there any change of your Union Parishad that is attributable to the NJ program? i. WASH ii. MCHN iii. Food security iv. Resilience and Disaster reduction v. Financial literacy? vi. …) NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 111 16. What are the challenges you often face being a being Union Parishad member? KIIs with Government Officials (MoLGRD&C, MoDMR, MoWCA, MoH&FW) Introduction Hello. My name is.......... I work for............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this funding is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other households in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree ? General data Date : Name of interviewee Position of the interviewee: Gender: Contact (not mandatory): Department: 1. Since when are you at your current position? 2. How was your department involved in the design of the NJ program? 3. How is your department involved in the implementation of the NJ program? 4. What is the institutional linkage between the NJ staff and your department? Is there a formal agreement? A joint work plan? 5. How are you informed about the program activities and performance? 6. How do you learn about the NJP performance? 7. What is the value added of the NJ program for your ministry? 8. How different are the activities implemented by the NJ program different from the activities that your institutions used to implement? 9. On a scale of 1 to 5 (1 being the least and 5 the maximum), how would you rate your satisfaction level with regard to the NJ program activities? 10. Why? 11. In the absence of the program, do you have the capacity (human and financial) resources to do the same activities and get to the same (or better) results)? 12. Are you aware of a sustainability plan for NJ program, has anything been discussed and put in place so far? 13. If yes what are the main building blocks of that plan? 14. Overall, how effective do you think are the approaches adopted by Nobo Jatra in:  (just ask the question related to the thematic areas the interviewee work) a. Reducing malnutrition and improving mother and child health (for ministry of health) b. Improving wash conditions in target communities, educating people on better Wash behaviors (health) c. Improving the production and productivity of farmers , increasing the profitability of value chains investments (ministry of ag) d. Increasing household and community resilience for future potential disasters (ministry of DRR) NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 112 15. What would be your top three recommendations to NJ for the remaining time? What could they do to improve the program performance? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 113 10. Interview Guide for Male Engagement Group Member Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Complete the list of attendees Union Parishad: District: 1. Since when has your group been formed? 2. How many members do you have? Who are those members? 3. How do you become a member of the male engagement group? Any criteria to fulfill? 4. How are you organized? 5. When did you start working with NJ program? 6. What activities have you done so far with the NJ staff? 7. What training have they received 8. When was that? 9. How many people were trained? 10. What follow-up contact (supervision) have they had? How often does that happen? By who? 11. As members of the men engagement group, what activities have you done (or do you normally perform in your community? 12. Activity 1 13. Activity 2 14. Activity 3 15. Activity 4 16. For each of these activities, what is the expected objective? What are you trying to achieve? 17. Objective for Activity 1 18. Objective for activity 2 Objective for Activity 3 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 114 19. Objective for activity 4 20. Since you started these activities can you tell me about three successes you have had so far (what have you achieved) 21. Success 1 22. Success 23. Success 3 24. What were the key drivers to those successes? What made it possible? 25. As change agent at community level, you must have faced, at different occasion resistance (or antagonism). Can you tell me what are the most important challenges you face in your work?  [pick up each challenge and ask the following five questions] 26. Why do you think [cite a challenge] is still prevailing in your community? 27. What are you doing about that? 28. Is it working? 29. Do you have any example of noticeable changes that you can share about that? 30. [if It is not working ] What needs to be done? 31. What would you recommend the NJ program to focus on over the next years to increase male engagement/interest in the program activities? 32. Recommendation 1 33. Recommendation 2 34. Recommendation 3 35. Do you have any other comments about the NJ program in your community? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 115 11. Interview Guide for Adolescent Group Member/Youth Volunteer Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Complete the list of attendees Union Parishad: District: 1. Since when has your group been formed? 2. How many members do you have? Who are those members? 3. How do you become a member of the adolescent group? Any criteria to fulfill? 4. How are you organized? 5. When did you start working with NJ program? 6. What activities have you done so far with the NJ staff? 7. What training have they received 8. When was that? 9. How many people were trained? 10. What follow-up contact (supervision) have they had? How often does that happen? By who? 11. As members of the adolescent group, what activities have you done (or do you normally perform in your community? 12. Activity 1 13. Activity 2 14. Activity 3 15. Activity 4 16. For each of these activities, what is the expected objective? What are you trying to achieve? 17. Objective for Activity 1 18. Objective for activity 2 19. Objective for Activity 3 20. Objective for activity 4 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 116 21. Since you started these activities can you tell me about three successes you have had so far (what have you achieved) 22. Success 1 23. Success 2 24. Success 3 25. What were the key drivers to those successes? What made it possible? 26. What are the most important challenges adolescents face in your community? 27. Challenge 1 28. Challenge 2 29. Challenge 3 30. Challenge 4  [pick up each challenge and ask the following five questions] 31. Why do you think [cite a challenge] is still prevailing in your community? 32. What are you doing about that? 33. Is it working? 34. Do you have any examples of noticeable change that you can share about that? 35. What would you recommend the NJ program to focus on over the next years to increase adolescent engagement/interest in the program activities? 36. Recommendation 1 37. Recommendation 2 38. Recommendation 3 39. Do you have any other comments about the NJ program in your community? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 117 12. Interview Guide for Members of the Child Protection Committees Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Complete the list of attendees Union Parishad: District: 1. Since when has your group been formed? 2. How many members do you have? Who are those members? 3. How do you become a member of the Child protection committee? Any criteria to fulfill? 4. How are you organized? 5. When did you start working with NJ program? 6. What activities have you done so far with the NJ staff? 7. What training have they received 8. When was that? 9. How many people were trained? 10. What follow-up contact (supervision) have they had? How often does that happen? By who? 11. As members of the Child protection committee, what activities have you done (or do you normally perform in your community to change that? 12. Activity 1 13. Activity 2 14. Activity 3 15. Activity 4 16. For each of these activities, what is the expected objective? What are you trying to achieve? 17. Objective for Activity 1 18. Objective for activity 2 19. Objective for Activity 3 20. Objective for activity 4 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 118 21. Since you started these activities can you tell me about three successes you have had so far (what have you achieved) 22. Success 1 23. Success 2 24. Success 3 25. What were the key drivers to those successes? What made it possible? 26. What has not worked as expected? Why? 27. What are the most important challenges for children in your community? 28. Challenge 1 29. Challenge 2 30. Challenge 3 31. Challenge 4 32. [pick up each challenge and ask the following five questions] 33. Why do you think [cite a challenge] is still prevailing in your community? 34. What are you doing about that? 35. Is it working? Why? 36. Do you have any examples of noticeable change that you can share about that? 37. What would you recommend the NJ program to focus on over the next years to increase your engagement/interest in the program activities? 38. Recommendation 1 39. Recommendation 2 40. Recommendation 3 41. Do you have any other comments about the NJ program in your community? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 119 13. Interview Guide for Producers Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date: Complete the list of attendees Union Parishad: District: 1- What is the name of your group? 2- When was this group formed? 3- How many members do you have? 4- What are the criteria to become a member of your group? 5- What are the main challenges producers are currently facing in your community?  [check the boxes that apply here 5.1 Not enough rain 5.2 Too much rain 5.3 Lack of inputs (seeds, fertilizers) 5.4 Lack of knowledge (techniques) 5.5 No access to markets 5.6 Postharvest losses 5.7 Other challenge 5.8 Other challenge 5.9 Other challenge 6- Over the past what has the NJ program done to help you reduce these challenges?  Activity 1  Activity 2  Activity 3  Activity 4 7- [If any training has been cited, ask the following questions] 7.1 When were you trained? 7.2 Tell me how the training was organized? 7.3 Who did the training? 7.4 How many other people attended the training? 7.5 What changed for you after the training? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 120 7.6 Is it working? Why or why not? 8- What have you done with the NJ program over the past two years? 9- What training have you received? By who? 10- What assets/services have you received? By who? 11- Do you have demo plots in your community? 12- [if yes] who manages the demo plots? 13- How often do you go to the demo plots? 14- What have you learn from the demo plots so far? 15- What have you applied so far?  15.1 technology 1  15.2 technology 2  15.3 technology 3  15.4 Technology 4 [pick up each technology that has been applied and ask the following questions] 16- For [technology aaa that you have applied] Do you think it is working? 16.1Why or why not? 16.2Do you think you will continue to apply that technology? 16.3What are you doing/what could be done to improve the performance of that technology? [pick up each technology that has not been applied and ask the following questions] 17 For technology BBB 17.1 Why didn’t you opt to apply it? 17.2 What should be done to improve it? 18 If Y, how satisfied are you with the quality of the inputs you have received? 19 Have you tested them? 20 For the next cropping season, you may not receive the same input from the program. Can you find those inputs in the market? 21 Are you ready to buy them if they are available in the market? Why or Why not? 22 Do you have problems selling your produce? Y. N 23 What are those problems? 24 What are you doing to solve them and get more cash from your production? 25 How is the NJ helping with that? 26 Is it working? 27 What could be done to help you get more cash from your production? 28 Is your group linked to any collection center? 29 Do your members have access to the collection centers? 30 What are your impressions about the collection centers that are accessible to your members? 31 What needs to be done to improve the effectiveness of the collection centers? 32 Do you have any suggestion to the NJ to help you produce more, sell more and improve your livelihoods? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 121 14. Interview Guide for Processors Introduction Hello. My name is.......... I work for............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Complete the list of attendees Union Parishad: District: 1- Since when have you started processing ag products? 2- What are the main products that you process? 3- Where do you source your inputs? 4- What are the main challenges that you face to access to the products you need for your activity? 5- What support have you received so far from the NJ program? 6- How is that support helping you process more products? 7- Have you received any training from the program? 8- If yes when, by who? How often? 9- What has changed for you since you got the training?  9.1. change 1  9.2. change 2  9.3. change 3 10- If nothing, has changed, why? 11- What have you learned and applied from the training?  11.1 item 1  11.2 item 2  11.3 item 4 12- Is it working? 13- What have you learned and decided not to apply? • item 1 • item 2 • item 3 14- Why did you choose not to apply them? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 122 15- Who are your suppliers? 16- What more can be done to make you adopt them? 17- How often do you have supply problems? 18- Do you have any formal linkages with suppliers? 19- How do you sell your products? Where? Who are your clients? 20- Are you able to satisfy all the demand, at all time during the year? 21- What could be done to help you gain more markets and clients? 22- Any other comments? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 123 15. Interview Guide for Participants of the graduation program Introduction Hello. My name is.......... I work for............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Complete the list of attendees Union Parishad: District: Entrepreneurial literacy training: 1. Have you taken any literacy training so far? 2. How many training have you had? 3. How often? 4. What process was followed to designate you as a beneficiary of the graduation program? 5. How useful did you find the training? 6. What do you suggest to improve the trainings and their results? (2) Monthly cash transfer 7. Since When are you enrolled in the graduation program? 8. How much money are you receiving from the Program since you started to participate in the graduation program? 9. Do you know for how long that will last? 10. What happens if you don’t participate in the trainings? 11. What did you mainly use the cash for? 12. Going forward, what do you think should be done with the management of the cash transfer activity to improve its impact on you and your household? (3) IGA selection and development 13. Have you been trained on how to start a business? 14. Have you already developed your business plan? 15. Have you already launched your business? 16. Where did you take the funds from? 17. What do you suggest to improve the identification process and development for future beneficiaries? (4) Productive asset development 18. After the training, have you launched your business? 19. Did you receive a startup fund from the NJP? 20. If Yes how much? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 124 21. If No, is it in the plan to receive that fund? 22. Was it enough to launch the business? 23. If the business has been launched, what are the challenges that the person is facing now? 24. What are the solutions that will be implemented to address those challenges? (5) Participation in savings groups: 25. Do you have saving groups in your village or around? 26. Are you a member of a saving group? 27. Why or why not? 28. Since when are you a member of a saving group? 29. What services are you expecting from the group? Loan? Saving? Transfert?... 30. Do you have mobile money options with your saving account? 31. What suggestions do you have to increase the potential benefits from your participation in the saving groups? (6) Ongoing supervision, mentoring and follow-up 32. How often have you received a personalized supervision visit since you are part of the graduation program? 33. When was the last time you received a supervision visit from the NJP staff? 34. What was recommended during that visit? 35. Have you implemented the recommendations? 36. Why or why not? 37. What suggestions do you have to maximize the impacts/results from the supervision visits? 16. Focus Group Discussions with DMC Members Introduction Hello. My name is.......... I work for............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this project is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other individuals, households and groups in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree to participate in this interview? General data Date : Complete the list of attendees Union Parishad: District: 1- Tell me a little bit about your group: When was it formed? How many members? Male/female? Objectives? 1.1 Date of creation 1.2 membership 1.3 objectives NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 125 2- How do you become a member of your group?  2.1 criteria 1  2.2 criteria 2 2.3 criteria 3 3- What activities do you do at community level?  3.1 Activity 1  3.2 Activity 2  3.3 Activity 3 4- When did you start working with the NJ program? 5- What training have you received from the NJ program? When and for how long?  5.1 training 1  5.2 training 2  5.3 training 3 6- What have you applied (or are you applying from those training)? • Item 1 • Item 2 • Item 3 7- Is it working? What has it changed in your community?  7.1 Change 1  7.2 Change 2  7.3 Change 3 8- If nothing has changed, why is that? What did not worked well? 9- What are the main disasters your community has undergone over the past five years?  9.1 disaster 1  9.2 disaster 2  9.3 disaster 3 10- Before the collaboration with the NJ program how were you addressing those disasters? 11- Now how are you addressing those disasters? 12- For each disaster what are the trigger indicators? How do you know you have to trigger the plan to address the disaster?  12.1 trigger indicators for disaster 1  12.2 trigger indicator for disaster 2  12.3 trigger indicator for disaster 3 13- What other goods or services have you received from the NJ program? 14- How are they making change in the way you address the potential disasters?  14.1 change 1  14.2 change 2  14.3 change 3 15- Have you ever had to test them? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 126 16- Are they working? 17- What could be done to improve them? 18- Overall, how do you feel about the collaboration of your group with the NJ program? 19- What else needs to be done to strengthen the effectiveness of your group? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 127 17. IDI with MCHN Specialists Date: Name: Position: Organization: 1. How long have you been working with the NJP? 2. What are the main three MCHN issues that the NJP is focusing on? 3. For each issue, what is the NJP strategy? 4. How different is that strategy from what has been done before by the government and other development actors? 5. Is the level of effort among different interventions and activities adequate to solve those MCHN problems facing the community? 6. At community level, do you think there are clear linkages and coordination with public and private health, nutrition and social services? 7. Do you think the community members have a clear understanding about the services offered by the NJP and who are eligible to receive them? 8. How successful has the activity been in increasing access to government preventative and curative health and related social services? 9. How successful has the activity been in getting the community linked with potential government financial resources (i.e., construction of WatSan related infrastructures)? 10. Do you think the field staff are qualified and knowledgeable of the purpose and methods used in the NJP? 11. What is being done to improve the capabilities of the staff and local partners to respond to community needs in MCHN and meet the objectives of the NJP? 12. Are structured supervision plans and tools in place to monitor those responsible for providing health and nutrition assessment/care/treatment? 13. Is there a problem with the attrition of community health or nutrition volunteers? If so, why do they drop out and what can be done to reduce the rate of attrition? 14. What is the quality of growth promotion sessions? Has there been follow up discussions with the mothers and/or caregivers? 15. Do you think the front line staff are staff aware of common micronutrient deficiencies? Do they know how to link people to available services? 16. Do you think the staff and community volunteers clear on recommended behaviors and the rationale for these? 17. Are the NJP MCHN eligibility criteria appropriate given the objectives and assumptions of the project? If not, how should it be modified? 18. Which MCHN strategies have been more effective so far and why? 19. What new knowledge have beneficiaries acquired and what knowledge is lacking? 20. Has new knowledge translated into new practices and recommended behaviors? If yes, what are the contributing factors to this achievement? If no, does the NJP understand the obstacles to optimal behavior and have a plan to address? 21. Do those responsible for promoting health and nutrition behavior change have SBC materials that are appropriate – tailored to the user, actionable, accurate and linked to growth promotion messages? Which materials need strengthening, if any, and how? Are users of these materials able to select the appropriate messages and provide effective counseling? If not, what skills need strengthening? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 128 22. Through what process were the health and nutrition behavior change and communication (BCC) materials developed, tested and applied? 23. Is there an exit or graduation strategy developed for MCHN? 24. Are the NJP MCHN graduation criteria appropriate given the objectives and assumptions of the project? If not, how should it be modified? 25. What are your main three suggestions on how to improve the NJP MCHN activities delivery and maximize their impact going forward? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 129 18. IDI with Disaster Risk Reduction Specialists Date: Name: Position: Organization: 1. How long have you been working with the NJP? 2. What are the main three potential disasters that the target NJP communities are facing? 3. What is the overall NJP DRR strategy? 4. How different is that strategy from what has been done before by the government and other development actors? 5. In that strategy, which activities have been most effective as perceived by the community? Which activities were not successful and why? How might these activities be improved? 6. How well did the DRR activities integrated with interventions to promote food access, availability and utilization? 7. What linkages do you have between your DRR activities and the government promoted emergency preparedness or disaster risk reduction activities? 8. What roles did the communities and individuals have in the development of the DRR activities? 9. Are government and community stakeholders aware of the NJP DRR activities and outputs? How successful has the program been in assuring government buy-in/support? 10. What evidence, if any, exists to indicate improved community preparedness for response to and recovery from disasters? 11. Are there additional direct or indirect benefits derived from infrastructure construction or rehabilitation that are not currently being captured? 12. Are there any unintended negative environmental impacts stemming from infrastructure activities? If so, are there sustainable mitigation measures being implemented? What additional measures can be implemented? 13. Is there an exit or graduation strategy developed for communities benefiting from the DRR package? 14. Are the NJP DRR graduation criteria appropriate given the objectives and assumptions of the project? If not, how should it be modified? 15. What are your main three suggestions on how to improve the NJP DRR activities delivery and maximize their impact going forward? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 130 19. IDI with Water, Sanitation and Hygiene (WASH) specialists Date: Name: Position: Organization: 1. How long have you been working with the NJP? 2. What are the main three WASH issues that the NJP is focusing on? 3. For each issue, what is the NJP strategy? 4. How different is that strategy from what has been done before by the government and other development actors in the WASH sector? 5. Is the level of effort among different interventions and activities adequate to solve those WASH problems facing the community? 6. What strategies are employed to influence water, sanitation and hygiene (WASH) behaviors of men, women and children? 7. What is the level of successes of these strategies in influencing WASH behaviors of target population? 8. What is the quality (i.e. interactive sessions, length of the session, topic selection, quality and use of visual aids, quality and use of flipcharts, participation, and demonstrated knowledge of the facilitator on the topic) of behavior change sessions? How it can be improved? 9. Which WASH activities are proving to be most effective in addressing assessed needs? 10. Which activities are not successful and why? How might these activities be improved? 11. Is the NJP building or distributing or rehabilitating potable water system (s) (borehole pump or tube-well) in the community? Are all of the pumps working? How they are maintained? Who are benefiting from the water systems? Who were excluded? And why? 12. How has the community been involved in deciding the type and location of water sources? How the sources have been maintained? Are there examples of community maintained water sources? What are the key factors behind the success of this model? What are the challenges in community led management of water points? How it can be addressed? 13. As far as you know, what is the percentage of functioning water committees? Do the water committees have savings accounts? 14. Are there households in the NJP target community that do not receive water supply? If yes how do those households get water? Why don’t those household receive water? 15. Do the water committees collect monthly fees from the households to maintain the water system? Are the monthly fees collected from the households enough to pay for the operation and maintenance of the water system? Where do the necessary funds come from for maintenance or repair? 16. What are men and women’s preferences for location of water points? What preferences are there for design and technology? 17. What communication methods are preferred by women and men to provide them with the information they need about the WASH activities? Are these formal or informal? Are there differences between women and men in willingness and capacity to pay for water services? 18. What is the level of training of staff and volunteers to promote health and hygiene interventions? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 131 19. What additional training, if any, would be recommended for project staff and volunteers? 20. Is there an exit or graduation strategy developed for communities benefiting from the WASH package? 21. Are the NJP WASH graduation criteria appropriate given the objectives and assumptions of the project? If not, how should it be modified? 22. What are your main three suggestions on how to improve the NJP WASH activities delivery and maximize their impact going forward? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 132 20. IDI with Monitoring and Evaluation Specialists Date: Name: Position: Organization: 1. How long have you been working with the NJP? 2. Does the IPTT have robust indicators to measure a specific result stated in the Results Framework (RF)? Does the IPTT have all of the FFP required standard indicators? What is the balance of indicators? Does the project have too many indicators for each results stated in the Results Framework? 3. Does the IPTT include gender-sensitive indicators appropriate to the RF? 4. Whether the targets are reasonable? Are they ambitious enough to make a difference to the project beneficiaries? Are the targets justifiable -- based on or agency’s past experience with a Title II project or another similar project? Comparable to the targets for similar interventions in the country? Are targets clearly defined as cumulative or not? Have the targets been revised using the baseline survey data? 5. Does the project have a clearly defined comprehensive M&E plan that detailed out all of the monitoring activities, tools, methodologies, frequency, analysis plan, data quality assurance plan, communication strategy, data usage strategies, and feedback mechanism? 6. How functional the routine output monitoring system is? Does it have well developed data collection forms, data flow diagram, defined responsibilities for each partner and/or sector, aggregation levels, and a clear link showing how the information will be aggregated and reported onto the Annual Results Report? What percent of staff time is devoted to routine monitoring? 7. To what extent is the monitoring system operational? How this could be improved? 8. Does the system produce reports on time? Do the users find the reports helpful? Do the reports present data in an easy to understand format? Does it help the technicians, managers, front line staff and partners to track the output level progress against targets? Do the reports offer interpretation of the data presented? Perception of the users about the quality of the effectiveness of the system? How this can be improved? 9. Does the monitoring system measures outcome level changes? Does the system use appropriate methods and tools? How often it measures outcome? Is the system too burdensome to the M&E team? 10. What is the quality of data collected and reported through the NJP M&E system? How does the project manage data quality? 11. Does the project have a data quality insurance mechanism? How effective the system is? How this can be improved? 12. Does the system insure all five dimensions of data quality – validity, reliability, timeliness, integrity and precision? How this system has been working? How this can be improved? 13. Does the project have an internal data quality assessment system? What is the quality of the tools used for DQA? How has the system been working? 14. Is there any evidence suggesting that the project management and technicians used M&E data to make programmatic decisions? How effective the communication NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 133 strategy of the monitoring results? Which communications strategies are useful and what needs to be improved? How it could be improved? 15. What are the challenges faced by the M&E team in influencing project decisions through M&E data? How they could be overcome? 16. What is the M&E capacity (staff, budget, core competencies of staff) of the project vis￾à-vis the M&E roles and responsibilities? What can be done to improve the capacity? 17. Does the organizational structure supportive of M&E? What are the barriers and how could this be eliminated? 18. What are your main three suggestions on how to improve the NJP M&E and make it more effective? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 134 21. IDI with COP/DCOP/Project managers Date: Name: Position: Organization: 1. How long have you been working with the NJP? 2. Who is responsible for the vision for the program? How well has the vision for the program been articulated? How effectively has this vision been imparted to program staff within the different consortium members? 3. How are operational plans developed for the project? What has worked well in this process? What has not worked well? 4. How have the implementation related problems identified, analyzed and solved? What has worked well in terms of problem solving? What has not worked well? 5. How well does information get communicated throughout the project, including within the project, with implementing partners at different levels, with external contacts and FFP? What has worked well? What has not worked well? 6. How well has the project communicated with stakeholders outside of the project? What has worked well? What has not worked well? 7. What are the most importance lessons learned from the development NJP relative to management of the program? 8. How does the project track environmental indicators? Does the project have capacity to measure environmental indicators? What are the challenges in monitoring environmental indicators? Recommendations to address the challenges. 9. How would the NJP direct beneficiaries be described on the poverty/food insecurity/vulnerability continuum? For example, are they from the "poorest" end of the continuum, from somewhere in the middle, or from the "top" end of the continuum? 10. How does the achievement on target numbers of people compare to proposed targets? Is the program on-track to achieve its target beneficiary numbers? 11. Who else benefits from program activities who may not have been an intended beneficiary? How are they benefiting? 12. Who has been adversely affected by program activities? How have they been affected? 13. How effective14 the partnerships in planning, developing implementation strategies, reviewing monitoring reports, documenting and sharing lessons learned and taking programmatic policy decisions. Provide specific recommendations to improve the quality of partnership. 14 Attributes of effective partnership includes: a) Open and transparent communication and decision making processes among the partners; b) Common understanding of both individual and joint responsibilities of partners; c) Involvement of key stakeholders in all stages of partnership and program development; d) Strong leadership role taken by WV ; e) Flexibility in terms of scheduling meetings, adjusting roles and responsibilities, being willing to adapt to changes in planning and implementation, and accepting differences in philosophy, management style, and ability to contribute to the project; and f) Mutual respect in the areas of organizational expertise and accepting others’ judgment. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 135 14. What other programs are being implemented by the partner in the same geographic areas as the NJP? How does the project interface with these? 15. What other programs are being implemented by the partner in the same sectors as the NJP but in different geographic areas? How does the NJP interface with these? How do experiences get exchanged across different projects in the same sector? 16. Did the partners or consortium members periodically meet to review project progress, and implementation challenges? 17. Which strategies have worked in encouraging partner participation and involvement? Which strategies did not work? 18. Is the program effectively developing the capacity of counterparts and/or partners? If so, how? 19. Have there been unexpected developments relating to the external environment that called for a change in strategy or plans? Was the process used to manage this change effective at all levels of implementation? How could this process be strengthened? 20. Does the consortium have an open and transparent process for decision making and arbitration? Can it be improved? 21. Has the NJP developed an exit/ sustainability strategy? If yes does it clearly define the approaches, criteria for exiting, measurable benchmarks, timeline, actions steps, responsibilities of different stakeholders and staff, indicators to measure progress, and a mechanism to assess progress? If No when will that be done? 22. What services are being established or strengthened by the NJP that are intended to be continued beyond the life of the project? What are the main threats to enabling these services to survive? What is likely to happen to these services after the NJP ends? What strategies are put in place to address those threats. 23. What physical structures are being constructed or rehabilitated by the NJP that are intended to be continued beyond the life of the project? What are the main threats to maintaining these structures? What is likely to happen to these structures after the project ends? What strategies are being put in place to address those threats. 24. What relationships are being established or strengthened by the NJP that are intended to be continued beyond the life of the project? What are the main threats to enabling these relationships to survive? What is likely to happen to these relationships after the project ends? How is the program addressing these threats? What strategies are being put in place to address those threats. 25. Do you have any participant who have graduated from the program? If Yes Where are they? And how far are they from the benchmarks15 26. What are your main three suggestions on how to improve the NJP activity delivery and maximize its impact going forward? 15 (1) multiple sources of income; (2) no self-reported food deficit in the last year; (3) ownership of at least 10 poultry or other productive assets of equal value; (3) kitchen gardens with vegetables and lemons or chili plants; and (4) cash savings in an active savings account with a formal financial institution. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 136 22. IDI with Gender Specialists Date: Name: Position: Organization: 1. How long have you been working with the NJP? 2. Has there been a gender strategy either developed or adapted by the NJP? How well the strategy has been implemented so far? 3. Did the NJP offer some type of gender training at project start-up? If so, have annual refresher courses as well as training for staff arriving to the project after the initial training been offered? 4. What is the mix of male and female staff in the organization, and in the project by level. Why proportion of female staff is low (if it is)? What has been done to improve the sex ratio of male and female staff? Please provide realistic suggestions in improving the situation. 5. Are the intended targeted beneficiaries (men, women, boys, girls) for specific program interventions being reached? What are some of the benefits of the interventions targeting specific gender groups? Have negative impacts of NJP interventions on gender been observed? If so, what are they and what is being done to mitigate these impacts?” 6. What types of benefits do female beneficiaries receive in light of additions to workload NJP interventions may bring? Is there any evidence of women empowerment16 because of the project? 7. Does the NJP have accurate records around beneficiaries. Is the project demanding more time from women? 8. Has the work burden of women increased because of the project? How does the community see women’s participation? What do the men and other household members think about women’s participation in the project? 9. Does the M&E system collect and process gender-sensitive, sex and age disaggregated data to track outputs and outcome level changes? 10. What are your main three suggestions on how to improve the NJP activity delivery and maximize its impact going forward for female and male beneficiaries? 23. Direct Observation Checklist Training/Supervision Date: Site: Contact of a person who may have additional information: 16 Female empowerment is achieved when women and girls acquire the power to act freely, exercise their rights, and fulfill their potential as full and equal members of society. While empowerment often comes from within, and individuals empower themselves, cultures, societies, and institutions create conditions that facilitate or undermine the possibilities for empowerment. Dimensions of empowerment may include decision making power over agricultural production activities including crop, livestock, poultry and fisheries; decision making power over micro-enterprises; ownership, access to, and decision making power over productive resources such as land, livestock, agricultural equipment, consumer durables, and credit; decision making power over heath, nutrition and family planning choices; control over income and expenditures; allocation of time to productive and domestic tasks; satisfaction with the available time for leisure activities and increased participation in decisions around disaster preparedness. NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 137 1. What did you come across? Training Supervision 2. How many people attended? # female # male 3. Was the setting adequate? Yes No 4. Did the supervisor/trainer communicate clearly? Yes No 5. Did the participants try to implement the advice they received? Yes No 6. If yes did they succeed? Yes No 7. What was the success rate? 8. The last time similar activity took place, was it documented? Yes No 9. Did the trainer/supervisor review the report of the last training/supervision? Yes No 10. What is the percentage of recommendations that have been put in practice? 11. Additional comments Infrastructure/asset Date: Site: Contact of a person who may have additional information: 1. What infrastructure did you come across? Infra 1 Infra 2 2. Does it have a known- designated person to take care of it? (owner or manager) Yes No NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 138 3. Is it functional? Yes No 4. Does it have USAID brand on it? Yes No 5. Did the NJP put in place any Environmental Management Plan? Yes No 6. Has any measure been Implemented? Yes No 7. If yes which ones? If No why? Measure 1 Measure 2 8. Does the owner/manager know the mitigation measures that need to be implemented? Yes No 9. Who is paying for the implementation of the mitigation measures? NJP Owner Community Other 10. Any plan or clarity on who is going to pay for the implementation of those measure after the project ends? Yes No 11. Comments Branding 1. IP : _______________________________________________________________________ 2. Date of visit :______________________________________________________ 3. Person interviewed :___________________________________________________________ 4. Was the IP or its representative briefed on USAID marking requirement after the award? Yes No NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 139 5. Did the IP receive any Branding or marking waiver? Yes No 6. If Yes, which one? Waiver 1: ______________________________________________________ Waiver 2: _____________________________________________________ Does it have a Branding Strategy? Yes No 7. If No, Why? _________________________________________________________ 8. If Yes, Has the IP started to implement its branding Strategy? Yes No 9. Are all the program/project commodities/equipment that you have come across during the visit marked with USAID logo? Yes No 10. Are all public communication materials/events that you have come across during this visit, marked with USAID identity? Yes No 11. Was the USAID logo that you have seen during this visit accurately designed? Yes No 12. If No, give an example Example 1:___________________________________________________________________ 13. Were the size, placement and proportion of USAID logos thoroughly respected? Yes No 14. If No, give an example: Example 1:_______________________________________________________ NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 140 24. Focus Group Discussions with Community Support Groups (CSGs) Guiding Questions for the CSG FGDs Check list for FGDs with CSGs Date of FGD: ______ (date): April/ May, 2018 FGD Group Name: Community Support Group Area: Village: ______ Union: ______ Upazilla: ____ District: ______ Total Participated: _____ Category: Male: ___ Female: ___ xx ____ xx___ Facilitators: ________ Key Questions/ Check list for facilitators use 1. How have you been involved in Nobo Jatra Program (NJP)? How long? 2. What activities, function, role and responsibilities do you perform as CSG member? 3. What are the processes you do apply? Where? How? 4. How the CSG have been formed? 5. Can you please tell us about your satisfaction or dissatisfaction on group formation? What went well and not well, and why? 6. How to improve this in during the remainder of NJ program? 7. Any other comments/suggestion? 8. How do you see your voluntary services? Strengths and weakness? How to improve this in during the remainder of NJP? 9. How do you locate pregnant women and promote ANC, safe delivery and PNC, essential newborn care and immunizations? 10. Which of the process do you consider as effective and ineffective? Why? 11. How to improve this? 12. How to replicate and sustain them better? 13. Any other comments/ suggestion? 14. How do you promote Child care services and bring them door to door. 15. Which process (s) do you considered as most effective and least effective? Why? 16. Which of the process do you consider as much effective and less effective? Why? 17. Of the services pertained to nutrition, monitoring of adolescent eve-teasing (sexual harassment), and services for the disabled, which went well and not? 18. Why? 19. What are the key challenges and limitations? How to overcome? Any specific suggestion(s)? 20. How to increase CC visits and functioning of the community support groups? 21. How to replicate and sustain them better? 22. Any other comments/ suggestion? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 141 23. Overall, what are the challenges for the CSG to facilitate the process and provide better / quality service delivery? 24. Can you please tell whether this is more focused on the demand driven or service delivery? 25. What are the differences/ gaps you have observed between the groups (service recipients and delivery? What are these? How to improve/ minimize? 26. Think back over all the past years/ period that you've worked as CSG member and tell us your fondest functions and most effective process. 27. Why do you think these functions and processes are most effective? Any evidence? 28. How well have the activity’s interventions met CC - CSFs planned schedules, beneficiary numbers, and Outputs? What factors promoted or inhibited adherence to schedules? How were problems and challenges managed? 29. What are the strengths of and challenges to the CSGs overall activity’s implementation, management, communication, and collaboration so far? What factors appear to promote or challenge the activity operations or effective collaboration and cooperation among the various stakeholders and CSGs functions? 30. Think back over the past years of the things that NJ did to revitalize the CCC-CSGs. 31. What (which process) went particularly well? Why? Evidence? 32. Do they differ from type of activities? What? Why? 33. Do they vary from various socio-economic groups, occupation, gender, literacy, geo context? What are these? 34. Which process did not work well and ineffective? Why you think so? 35. How to improve the effectiveness and efficiency of NJ in future? 36. What is your engagement with other service providers? Who are they? (categorize by importance/ effectiveness and efficiency) 37. What is your experience by engaging with them? Which process do you find effective and went well? Why? Evidence? 38. Which process did not went well? Why? Any bad example? 39. How to improve? 40. Suppose that you were in charge and could make one change in the program process that would make the program better. What would you do? What can each one of you do to make the program better? 41. What else (Is there anything else) you’d like to say about CC-CSGs functions, role and responsibilities to improve the quality of service delivery process? 42. What else (Is there anything else) you’d like to say about the CC-CSGs service delivery process and beneficiaries demand more effective and sustainable? 43. What else (Is there anything else) you’d like to say about NJ to bring the program process more effective and sustainable? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 142 NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 143 25. Interview Guide for Alternative IGAs Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this funding is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other households in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree ? General data Date : Complete the list of attendees Union Parishad: District: 1- What is the name of your group? 2- When was this group formed? 3- How many members do you have? 4- What are the criteria to become a member of your group? 5- What are the main challenges people, who wish to enter into Alternative IGAs, are currently facing in your community?  Challenge 1  Challenge 2  Challenge 3  Challenge 4  Challenge 5 6- Over the past what has the NJ program done to help you solve these challenges?  Activity 1  Activity 2  Activity 3  Activity 4 7- [For each training that has been cited, ask the following questions] 7.1 When were you trained? 7.2 Tell me how the training was organized? 7.3 Who did the training? 7.4 How many other people attended the training? 7.5 What changed for you after the training? 7.6 Is it working? Why or why not? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 144 8 Have you received any input from the NJ program? Y, N 9 If Y, what types of inputs? 10 If Y, how satisfied are you with the quality of the inputs you have received? 11 Do you still require the project provide these inputs? 12 If Y, at which point do you feel you will no longer be dependent on the project? 13 How can the project get you to that state quickly? 14 What are the most promising AIGAs in your community? • IGA 1 • IGA 2 • IGA 3 • IGA 4 • IGA 5 15 Do you have problems selling your product? Y. N 16. What are those problems? 17. What are you doing to solve them and get more cash from your AIGA? 18. How is the NJ helping with that? 19. Is it working? 20. What could be done to help you get better sales of your product? 21. Is your group linked to any collection center? 22. Do your members have access to the collection centers? 23. What are your impressions about the collection centers that are accessible to your members? 24. What needs to be done to improve the effectiveness of the collection centers? 25. Do you have any suggestion to the NJ to help you produce more, sell more and improve your livelihoods? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 145 26. Interview Guide for Employment Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this funding is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other households in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree ? General data Date : Complete the list of attendees Union Parishad: District: 1. What is the name of your group? 2. When was this group formed? 3. How many members do you have? 4. What are the criteria to become a member of your group? 5. What are the main challenges people, who wish to enter into the Employment, are currently facing in your community? • Challenge 1 • Challenge 2 • Challenge 3 • Challenge 4 • Challenge 5 6. Over the past what has the NJ program done to help you solve these challenges?  Activity 1  Activity 2  Activity 3  Activity 4 7. [For each training that has been cited, ask the following questions] 7.1 When were you trained? 7.2 Tell me how the training was organized? 7.3 Who did the training? 7.4 How many other people attended the training? 7.5 What changed for you after the training? 7.6 Is it working? Why or why not? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 146 8. Have you received any input from the NJ program? Y, N 9. If Y, what types of inputs? 10. If Y, how satisfied are you with the quality of the inputs you have received? 11. Do you still require the project provide these inputs? 12. If Y, at which point do you feel you will no longer be dependent on the project? 13. How can the project get you to that state quickly? 14. What are the most promising Employment sectors in your community? a. b. c. d. e. 15. Which government structures provides support to you to secure employment? a. b. c. d. e. 16. What kind of training did they provide? 17. What kind of inputs did they provide? 18. Were these effective / useful? 19. If not, why were these not useful? 20. How can the project strengthen its support such that you are more employable? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 147 27. Interview Guide for Value Chains (a Mixed Group Involved in Bitter Gourd, Bottle Gourd, GIFT, and Duck) Introduction Hello. My name is .......... I work for ............................... We have been selected to assist in the midterm evaluation of the Nobo Jatra program, funded by the US Government in Bangladesh and involved in improving food security, nutrition and resilience of vulnerable people in Khulna and Satkhira . We seek to understand how this funding is improving livelihoods in the target zones and what could be done to improve its performance for the benefits of more people. We want to ask you some questions, to you as to many other households in this locality ................... to gauge the level of achievement of the results of this project and see if necessary how to improve its functioning. Answers are confidential and will not be disclosed individually. Do you agree ? General data Date : Complete the list of attendees Union Parishad: District: 1- What is the name of your group /s? 2- When was this group/s formed? 3- How many members do you have? 4- What are the criteria to become a member of your group? 5- What are the main challenges producers are currently facing in your community? a. C1 b. C2 c. C3 d. C4 e. C5 6- Over the past what has the NJ program done to help you solve these challenges?  Activity 1  Activity 2  Activity 3  Activity 4 7- [For each training that has been cited, ask the following questions] 7.1 When were you trained? 7.2 Tell me how the training was organized? 7.3 Who did the training? 7.4 How many other people attended the training? 7.5 What changed for you after the training? 7.6 Is it working? Why or why not? 8- What have you done with the NJ program over the past two years? 9. Have you received any input from the NJ program? Y, N NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 148 10. If Y, how satisfied are you with the quality of the inputs you have received? 11. Do you have problems selling your produce? Y. N 12. What are those problems? 13. What are you doing to solve them and get more cash from your production? 14. How is the NJ helping with that? 15. Is it working? 16. What could be done to help you get more cash from your production? 17. Is your group linked to any collection center? 18. Do your members have access to the collection centers? 19. What are your impressions about the collection centers that are accessible to your members? 20. What needs to be done to improve the effectiveness of the collection centers? 21. Do you have any suggestion to the NJ to help you produce more, sell more and improve your livelihoods? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 149 28. Focus Group Discussion CVA Working Group  What is CVA working Group?  Why is it formed and what is its working mechanism? What are the steps to enable citizens’ engagement through CVA approach at first level?  Who are the member of CVA working group?  Through this CVA initiative what are the strategic objectives Nobo Jatra wants to achieve?  What are the monitoring standards for Union Community Clinic Committee?  What are the monitoring standards for Union WATSAN Committee?  What are the monitoring standards for Union Agri Service Committee?  What are the major achievements of this CVA working Group till now?  What are the Challenges to keep active of the CVA working Group? Common Qs for Governance as a crosscutting issue • Group forming process • how the community representation is done through considering different strata of the community (ethnicity/religious minority/disability/different income group/different profession etc.) • how the man - woman ratio is being maintained • is there any constitution developed / or in process to be developed • Group meeting resolution book is maintained • how this group will be sustained after the end of NJ, whether those process has been initiated • is there any fund accumulation process within the group, if yes how the fund is being generated • how the group savings transparency is being maintained like- loan processing, profit sharing, balance sheet maintenance process through sharing with the group members • frequency of group regular meetings / always on need based / ad hoc basis Department of the ministries need to consider following departments:  Ministry of Health & Family Welfare  DPHE  DGFP  IPHN  Ministry of Women & Children Affair  Department of Women & Children  Ministry of Local Government for Rural Development & Cooperative  Department of Local Government for Rural Development  Department of Co-operative NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 150 29. In-depth Interview with WASH Beneficiaries 1. Would you get the essential hygiene messages? What are the messages? Would you feel to require improving and what are the limitations / challenges to place in practice these messages? 2. Are the constructed WPs / latrines useful to you, do you feel any modification requires, had any discussion with you in regards environmental standard? Would you feel any environmental impact evolves? 3. Are there any community base monitoring tools for using safe water and latrine, hygiene practices? Do you any plan for O & M of your latrine? 4. Will you go to shelter when you get warning for evacuation, if yes, how get the message? 5. Do you feel completion of construction / rehabilitation work took excessive time, if yes, what reasons behind? 6. Have you any participation in installation of WP, latrine etc., if yes, what type and what extent? Would you feel it have been achieved at reasonable cost? 7. Do you feel after project intervention, will you have also usual practice of WASH? 8. How do you collect, preserve and use collected water? 9. How do you understand that the quality of water is good? 10. During the last 6 months if there was an outbreak of any diarrhea / intestinal diseases, if yes, how many people were affected, who got affected more, what is your suggestions to overcome that? 11. Before using this water have you suffered from any intestinal diseases and after using new one? 12. Do you received any DRR training, is these you perceived and practice? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 151 30. Key Informant Interview with WMC (including Caretaker of Water Points/Sanitation Points) 1. Was there any WMC in earlier? If not, how run the used one, how WMC was formed? Have you got any training earlier, if yes, what’s the difference that adds to run your activities currently? 2. Since when are you taking care of this installation/s and what you do for the management of these installations? 3. Who select the location of these installations? 4. What trainings you have received from the program staff? When and how long? 5. What did you choose to apply from the training? Is it working? Why or why not? 6. What do you choose not to apply from the training? Why? 7. Have any contribution from the community members, what kind and extent / amount? Is it covering the O & M cost, How many surpluses? 8. What linkage do you have with the other managers of water / sanitation points in your district? 9. What you think could be done by the NJ project or the GoB to improve the community access to water / sanitation? 10. Would you feel after project intervention it will operate /run? 11. Is the existing Water Point (WP) is rehabilitated or new one? How many people getting service from the WP? Is it sufficient to provide water supply? Is the WP is rightly located? If not, what is your suggestion? Have you get any complain from any corner (e.g. women), what type of participation you have in installing the WP? 12. Have you any idea about WSP, how you understand that the quality of water is good? 13. During the last 6 months if there is outbreak any diarrhea / intestinal diseases, if yes, how many people affected, who affected more, what is your suggestions to overcome that? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 152 31. Key Informant Interviews with Government Department and Ministries Some of the KIIs with the relevant Government Departments / Ministries (that expected jointly with other members) DMB / Ministry of Disaster Management 1. What is you’re feeling about less number of DMCs training received than of target? Is there any impediment for formation of Ward level DMC? Have you been process / released SoD for formation of Ward DMC? 2. Would you feel any kind of reorganizing the project support that will enhance the project outcomes in terms of quantity and quality? What is your suggestions in this regard? 3. Would you feel there have any climate change impact upon project interventions, would you feel it requires reconsideration in project for getting better result? 4. Would you feel for development of road connecting to the shelter requires to consider in DRR that may be implemented by the LGIs, how you feel to collaborate with them to get better project output? DPHE / LGD of Ministry of Local Government, Rural Development & Cooperatives 5. What is your feelings about private entrepreneur’s role in producing latrine products including disable / elderly people? Is Government can support or to link with the big company like RFL can play pivotal role in latrine products and its marketing; 6. Is single pit latrine well run more than one year, what you feel if pit is filled up, what support the project expects from the Government / project? 7. When tidal surge will occur or inundation of water remain more than say 2 days what type assistance / technology can be extended by the Government? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 153 32. Focus Group Discussions with WATSAN Committee Topics for discussion 1. Before project starting (a) was there any WATSAN committee; (b) if not when and how it was formed (c) would you feel participation of women will be effective and how? 2. When was the first time you have been in contact with the NJ program? What training have you received from the project staff? When and how long? 3. After getting training what type of knowledge you attained and what do you choose to apply from the training and what not and why? 4. Have any contribution from the community members, what kind and extent / amount? Is it covering the O&M cost, How many surpluses? Is there any basket fund or plan for routing O&M? 5. What linkage do you have with the other WATSAN committees? 6. What you think could be done by the NJ project or the GoB to improve the community access to water / sanitation? 7. Is WATSAN committee is functional, how you feel it operates well, without project intervention will it operate and how, would you feel its operational function will impact upon surroundings and how many people getting WASH SBCC message; 8. What do you feel about the challenges / limitations to operational function WATSAN committees? 9. Is there any open defecation or cultural practice rather than of sanitary latrine use? 10. Is there any more latrine / WPs demand beyond of project outlined? 11. Have any idea of WSP, how clean and maintain WPs /latrine? 12. Is the WP/ latrine rightly located for women use? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 154 33. Focus Group Discussions with DMC Members Topics for discussion 1. Brief when the group was formed, how many members (number of male and female) and what is your objectives? 2. How do you become a member of your group? Is any criterion? Tell about the criterion? 3. What activities do you at the community level and tell about at least three? 4. When did you start working with the NJ program? 5. What training have you received from the NJ program? When and for how long? 6. What have you applied or are applying from those training? 7. Is it working? What has changed in your community? 8. If nothing is changed, why is that? What did not work well? 9. Before collaboration with NJ program how were you addressing those disasters? 10. How presently addressing those disasters? 11. What are trigger indicators for each disaster? How do you know you must trigger the plan to address the disaster? 12. What other goods or services have you received from the NJ program? 13. How are they making change in the way you address the potential disasters? Have you ever had to test them? 14. What is your role that NJ project expects? Are you able to provide that support? If not, what is your suggestion to overcome? 15. Overall, how do you feel about the collaboration of your group with the NJ project and what else needs to be done to strengthen the effectiveness of your group? NOBO JATRA PROJECT/“NEW BEGINNING”: MIDTERM EVALUATION (MTE) REPORT, SEPTEMBER 2018 155 Annex D. List of documents received from WV  NJ proposal document  NJ Indicator Performance Tracking Table  NJ Theory of change narrative  FY 16 annual report  FY 17 Q1, Q2, Q3 reports  Agriculture and LH market assessment  Gender Analysis report  Wash and MCHN strategy  Wash market Assessment report NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-156 Annex E. Terms of reference World Vision Bangladesh USAID’s Development Food Aid Project ‘Nobo Jatra/New Beginning’ Mid-term Evaluation (MTE) Terms of Reference August 29, 2017 NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-157 Table of Contents 1. Introduction 159 2. Project Description 159 3. Theory of Change 162 4. Mid-Term Evaluation (MTE) Purpose 163 5. Mid-Term Evaluation (MTE) Objectives 163 6. Key Evaluation Questions 164 7. Evaluation Methodology 168 8. Audience and key stakeholders 169 9. Roles and responsibilities 169 10. MTE Team Composition, Qualifications, and Roles 171 11. Deliverables/Products of MTE 173 12. Timeline 174 13. Key documents to support the MTE 175 14. Intellectual Property 176 15. Ethical Guidelines 176 16. Expression of Interest 177 17. List of Annex 177 18. Resources 177 NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-158 List of Acronyms BIHS Bangladesh Integrated Household Survey BCC behavior change communication CVA Citizen Voice and Action DFAP Development Food Assistance Project DRR Disaster Risk Reduction EMMP Environmental Mitigation and Monitoring Plan FGD Focus Group Discussion FFP Food for Peace FSFC Food Security Country Framework HH Household HEA Household Economic Analysis IDIs In-depth interviews IPTT Indicator Performance tracking Table IYCF infant and young child feeding IEE Initial Environmental Examination LF Logical framework (LogFrame) MCHN Maternal Child Health and Nutrition MTE Mid-Term Evaluation MoDMR Ministry of Disaster Management and Relief NRM Natural resources management NGO Non-Government Organization PIRS Project Performance Indicators Reference Sheet SUAP Safer Use Action Plans TL Team Leader ToR Terms of Reference ToC Theory of Change USAID United States Agency for International Development VDC Village development committees WASH Water, sanitation and hygiene WI Winrock International WFP World Food Programme WVB World Vision Bangladesh NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-159 1. Introduction This document sets forth the Terms of Reference (ToR) for a Mid-Term Evaluation (MTE) for USAID’s Title II Development Food Assistance Project ‘Nobo Jatra-New Beginning’. As World Vision Bangladesh fast approaches the halfway point of the Nobo Jatra project in April 2018, this is a prime window of opportunity to carry out a MTE to take midcourse corrections in project strategy and address issues that are impeding effective implementation. Key to note, the MTE serves as a process evaluation rather than a summative evaluation and concentrates on reviewing implementation processes to produce planned outputs and the logic reflected in the results framework to determine if the project is likely to achieve intended results, objectives and impact. World Vision Bangladesh (WVB), a Christian humanitarian organization, is dedicated to the wellbeing of children, their families and communities to serve and partner with people in need regardless of religion, ethnicity, gender and ability. ‘Nobo Jatra-New Beginning’ is a five year USAID Food for Peace Title II Development Food Assistance Project that seeks to improve gender equitable food security, nutrition and resilience in southwest Bangladesh. World Vision Bangladesh, together with the World Food Programme, Winrock International and 3 local partner NGOs, undertook the project in September 2015, integrating interventions in MCHN, WASH, agriculture and alternative livelihoods, DRR, good governance and social accountability, and gender to achieve its objectives. Nobo Jatra is being jointly implemented in partnership with the Ministry of Disaster Management and Relief (MoDMR) of the Government of Bangladesh in four upazilas under two districts – Dacope and Koyra upazilas in Khulna and Shyamnagar and Kaliganj upazilas in Satkhira – it aims to reach 856,116 direct beneficiaries. The project seeks to engage an external consultant/agency to undertake the MTE based on the scope of work and specific deliverables outlined in this ToR. Given the scale of the Nobo Jatra project in terms of geographic coverage, number of direct beneficiaries, and households, as well as the integrated nature of the project interventions, specific technical expertise and sound previous experience of working with United States Agency for International Development - USAID projects is sought from the consultant/agency. Additionally, they should possess strong evaluation management skills with prior experience in successfully leading evaluation teams in Bangladesh. 2. Project Description Poverty remains the primary cause of food insecurity in Bangladesh.17 High poverty rates and food insecurity are associated with small landholdings, and 51 percent of households in rural Bangladesh are landless.18 Cultural norms and water, sanitation and hygiene (WASH) conditions heavily influence household (HH) dietary practices and nutrition outcomes.19 Deficiencies in WASH conditions20 result in environmental enteropathy21 contributing to stunting and affecting overall health conditions in the population. Women’s access to and control over assets and decisions on health and finances, personal mobility and speaking in public are severely limited22 . 17 USAID Office of Food for Peace (FFP) Food Security Country Framework (FSFC) for Bangladesh FY 2015- 2019 18 Ahmed, A.U. et al. 2012. The Status of Food Security in the Feed the Future Zone and Other Regions of Bangladesh: Results from the 2011–2012 Bangladesh Integrated Household Survey (BIHS). Dhaka: IFPRI. 19 Bangladesh USAID BEST Analysis, FinTrac, May 2014 20 Khulna Division WASH Assessment, World Vision, December 2014 21 Environmental enteropathy is a subclinical condition caused by constant fecal-oral contamination and resulting in blunting of intestinal villi and intestinal inflammation, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3372657/ 22 WV Gender Assessment, December 2014 NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-160 Further to a comprehensive food security assessment comprised of: Household Economic Analysis (HEA) assessments in Satkhira, Khulna, a Rapid Market Assessment, and a multi￾sector qualitative assessment, World Vision, Inc. (WV) and sub grantees, World Food Programme (WFP) and Winrock International (WI), commenced in 2015 a five year USAID-funded Title II Development Food Assistance Project (DFAP), called Nobo Jatra, which means ‘new beginning’ in Bangla. The project targets 856,116 beneficiaries including pregnant and lactating women, children under 2, youth and adolescents across Dacope, Koyra, Kaliganj and Shyamnagar Upazilas. With the goal to improve gender equitable food security, nutrition and resilience of vulnerable people in Bangladesh and to address the issues described above, interventions are organized around four important themes: WASH and Maternal Child Health and Nutrition (MCHN) (P1), agriculture and economic development (P2), and resilience (P3), as well as the cross-cutting purpose of governance and social accountability, with youth development and gender integrated throughout all activities. A map of Nobo Jatra Geographical Coverage Nobo Jatra Goal, Purposes and Sub-Purposes NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-161 Purpose 1: Project interventions focus on the improved nutritional status of children under five years of age, pregnant and lactating women and adolescent girls through improving utilization of WASH practices, reducing adolescent pregnancy, increasing equitable nutritious food intake, and increasing practice of gender equitable norms in the household, including food distribution, work load, supporting environment and decision making. Purpose 2: Interventions focus on increased income and equitable access to nutritious food for both males and females through increasing the diversification of livelihoods and increasing production of safe, diverse, nutritious and high-value foods. Activities also focus on increasing the diversification of livelihoods, via a pathway including strengthened linkages to private sector to identify and select market-based livelihoods opportunities which improve employment opportunities and ultimately increase incomes. Purpose 3: The project supports strengthened gender equitable ability of people, households, communities and systems to mitigate, adapt to and recover from natural shocks and stresses through improving the capacity of government institutions who are mandated for delivering Disaster Risk Reduction (DRR) services, as well the capacity of the targeted communities. Purpose 4 (cross cutting): the Nobo Jatra project aims to improve social accountability of service provision for vulnerable men and women through increased responsiveness of market￾based local service providers (WASH and agriculture) and increased responsiveness of government service providers (H&N, DRR, AgE, social protection). Through strengthened inclusive village development committees (VDCs), via capacity building and training in gender sensitivity and GBV, Citizen Voice and Action (CVA), good governance and leadership, the project ensures the existence of a local-level governance body that can effectively plan and coordinate resilience-building activities that appropriately target and benefit all segments of society. Engagement in evidence based national level policy dialogue NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-162 aims to create an enabling environment for sustainable impact by demonstrating and influencing changes to critical GoB DRR, resilience, climate change and safety net programs and policies. As prime, World Vision Bangladesh is responsible for overall project and grant management, the MCHN, WASH, graduation promotion, gender and good governance technical components as well as compliance, quality assurance and local partner Non-Government Organization (NGO) management. WFP lead on the cash transfers under both MCHN and graduation promotion components, government relations and policy engagement, and DRR engagement. WI lead on the agriculture and livelihoods portfolio including value chains and alternative livelihoods. 3 local partner NGOs work with each sub grantee to implement specific interventions as shown in the diagram below. 3. Theory of Change Nobo Jatra’s theory of change is that by:  increasing access to clean water and improved sanitation,  improving maternal and child health and nutrition practices, including infant and young child feeding (IYCF),  increasing use of climate smart agricultural techniques and natural resources management (NRM),  improving income diversity, asset maintenance and savings,  developing more alternative livelihood opportunities for youth,  increasing mobility and voice for women, NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-163  and communities becoming engaged and influencing change at the national policy level, chronic poverty will be reduced, households will gain greater resilience and there will be a truly transformative change in the lives of individuals of Khulna and Satkhira districts in Bangladesh. Please refer to Annex-B for the full Theory of Change graphic with details on purpose, sub purpose, outcome, output, assumptions etc. 4. Mid-Term Evaluation (MTE) Purpose The main purpose of a mid-term evaluation is to improve project effectiveness and efficiency to inform specific future decisions. An MTE is a process evaluation that will suggest midcourse corrections in project strategy and recommendations for improving effectiveness. To this end, the MTE focuses particularly on gathering evidence on the quality and effectiveness of service delivery, and the efficiency and acceptability of activity implementation. The MTE further serves to develop recommendations for adjustments that will improve the activity’s final outcomes. The MTE will review the following: resources utilized, management of sub-grantees, internal and external communication and coordination, compliance of USAID policy and regulation, and partnership and relationship with the host government and other stakeholders. 5. Mid-Term Evaluation (MTE) Objectives The MTE will explore how well the activity is following implementation plans and meeting targets, the acceptability of the methods employed to the beneficiary population, and early signs of changes that beneficiaries associate with program interventions. The evaluators will prepare a Key Evaluation Questions Matrix based on the objective, key questions and methods to consider as part of the Inception Report. The matrix should identify more-detailed areas of focus for each question, aspects to consider within each focus area, and methods for investigation. See Monitoring & Evaluation Policy & Guidance for Food for Peace Development Food Security Activities Annex 3 for illustrative examples.23 The main objectives are as follows: 1. Quality & Effectiveness: to evaluate the quality and effectiveness of service delivery, the strengths and weaknesses of activity implementation and management, and the quality of Outputs, in terms of adherence to terms agreed with FFP and their appropriateness and perceived value to target communities, identifying factors that appear to enhance or detract from the quality, acceptability and usefulness of implementation and Outputs. 2. Evidence of Change: to present early evidence of changes (intended and unintended, positive and negative) associated with activity interventions and Outputs, assess how well the observed changes reflect the TOC and Logframe, and identify factors in the implementation or context that impede or promote the observed and intended changes. 3. Collaborative Learning & Action: to review systems for capturing and documenting lessons learned and assess the extent to which they are used in project implementation and refining project design, including feedback from the perspective of stakeholders and participants. This would cover assessing processes to use evidence including baseline results and monitoring data for adjusting project strategies. Also it will assess how well the project is seeking out, testing and adapting new ideas and approaches to enhance projects’ effectiveness or efficiency. http://pdf.usaid.gov/pdf_docs/PBAAF103.pdf NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-164 4. Sustaining Project Impact: to determine the extent to which outcomes, systems and services are designed and being implemented to continue after the project ends and assess progress made on implementing sustainability strategies. 5. Cross-cutting Themes: to determine the appropriateness and effectiveness of support for gender equity in terms of access to, participation in, and benefit from project interventions. Assess the extent to which project interventions target youth, support greater capacities for local governance and address sources of environmental risk. 6. Project Framework: to recommend adjustments to the TOC and Logframe, activity design, resource allocation, activity management, M&E Plan, or implementation that could improve the likelihood of achieving desired results by the activity’s end, based on the evidence collected and conclusions drawn for the evaluation objectives above. 6. Key Evaluation Questions For Objective 1: 1. How well have the activity’s interventions met planned schedules, beneficiary numbers, and Outputs? What factors promoted or inhibited adherence to schedules? How were problems and challenges managed? For these questions, the evaluators may use secondary data from the activity proposal, detailed implementation plans, intervention reports, MIS and other activity resources to compare plans to actual start dates and achievement of Outputs. To understand promoting and inhibiting factors, the evaluators should interview members from various stakeholder groups and compare and contrast achievements across different intervention sites, beneficiary groups, implementing partners, administrative units, geographic regions, etc. 2. What are the strengths of and challenges to the overall activity’s implementation, management, communication, and collaboration so far? What factors appear to promote or challenge the activity operations or effective collaboration and cooperation among the various stakeholders?  What are the strengths and challenges in implementation, management, communication, and partnership thus far in the project cycle? The strengths and challenges should also be streamlined by each technical component, in terms of acceptance by local communities, producing quality outputs.  Evaluation of the linkages established by the project with complementary projects in the Feed the Future zone of influence and beyond as well as relevant government departments. The MTE team members should emphasize to identify the strengths and challenges in the ways the activity is being managed or monitored at different locations or the way information is communicated among different stakeholders, they should investigate why. Investigation may involve review of activity records that describe the design and results of processes; primary data collection at purposively sampled sites implemented by different agencies, supervised by different individuals, collaborating with different partners, or showing greater and lesser performance; and interviews with a variety of stakeholders, including staff at various levels— direct, indirect, and non-beneficiaries in targeted communities; community leaders; activity collaborators; USAID Mission staff; and others. 3. In each technical sector, what are the strengths of and challenges to the efficiency of interventions’ implementation and their acceptance in the target communities? How well do NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-165 implementation processes adhere to underlying principles and activity protocols? What factors in the implementation and context are associated with greater or lesser efficiency in producing Outputs of higher or lower quality? Which interventions and implementation processes are more or less acceptable to members of the target communities and why?  To what extent are the project interventions meeting the needs of the target beneficiaries? Are they aligned with the Bangladesh country strategies and policies and/or with USAID development goals, objectives, and strategies?   Specifically: is the project addressing the most critical problems or constraints to Improve gender equitable food security, nutrition and resilience of vulnerable people? Are the project activities adequate to meet the needs of project beneficiaries? 4. To assess the effectiveness of cash-for-asset interventions (graduation) in achieving the activity's purpose and intermediate outcomes. To assess the targeting approach used for the graduation promotion component, evaluate the process/steps used in the approach to graduate participants out of extreme poverty with analysis of the capacity of the M&E system to accurately monitor and capture this data. To answer these questions, the technical experts should observe interventions to the extent possible and use other methods to further assess: the technical quality of interventions and Outputs; implementing staff members’ accuracy of knowledge and level of confidence in delivering services and messages; how well intervention protocols meet national and international standards of good practice and match the priorities and contexts of the target communities; the level of satisfaction of beneficiaries with implementation methods, timing, location, etc.; environmental compliance; gender integration; level of community engagement; conflict-sensitivity and measures to do no harm; and how well the sector is integrated with the activity’s other sectors. In addition to a review of records and M&E data provided by the activity, answering these questions will require direct observation at a purposively selected sample of intervention sites and Outputs and discussions with members of a variety of stakeholder groups, including direct, indirect, and non-beneficiaries in targeted communities; local sectoral experts; implementing staff and management team. For Objective 2: 1. What changes—expected and unexpected, positive and negative—do community members and other stakeholders associate with the activity’s interventions? What factors appear to promote and deter the changes? Which interventions appear to be more or less potential to influence knowledge or behaviors? How do the changes correspond to those hypothesized by the activity’s TOC and Logframe?  Specifically focus on the ToC and assess any changes, identify gaps, external outcomes and stakeholders.  Are objectives of the project in keeping with locally defined needs and priorities? Should the direction of the project be changed to better reflect those needs and priority?  To what extent have interventions aligned with planned targets of participant, timelines and outputs? Any extenuating elements that impeded or accelerated plans? How were challenges mitigated or addressed? By the time of an MTE, there should be early signs of change apparent, especially among individuals and households who directly benefit from activity interventions. Annual Review Reports and routine monitoring reports will be helpful in assessing quantitative Outputs and NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-166 some Outcomes, but the evaluators must observe and talk with community members, activity staff, local leaders and other stakeholders to gain perspectives about emerging changes and related factors. For example:  Changes in behavior, practices, attitudes, and knowledge that individuals have made themselves or observed in others  Changes that have occurred in the biophysical, social, or household environment (e.g., reduced erosion, better pasture condition, safer pesticide use, improved water quality, change in market prices or terms of trade between livestock and staple goods, reduced flooding of residences) As they investigate, to understand factors affecting change, the evaluators should be alert to:  Characteristics of individuals, communities, and implementation where changes have advanced at different rates  Reported barriers to changes intended by the activity  Evidence of potential harm from activities  Conditions that have promoted or discouraged the observed changes or are likely to promote or discourage the continuation of observed changes For Objective 3: 1. How have NJ management and technical specialists used data to inform programmatic decisions, referral and follow up? What processes have been instituted to improve data collection and data quality? How has the project improved effectiveness or efficiency as a result of new ideas or approaches brought into the activities? How is information generated by the projects used to inform decision-making? How can this be made more effective? Objective four will include taking stock of research completed thus far in the cycle of the project (Gender Analysis, Market Assessment and Product Development on WASH and Livelihoods) with evaluation of how the research and recommendations are being used to inform and adapt programming. An analysis of external dissemination strategies including any national level advocacy should also be the focus of attention. For Objective 4: 1. Has the project developed and implemented sustainability strategies? What organizations, services or relationships are required to sustain the outcome changes observed by the review team? Has the project identified the outcomes to be sustained, and the necessary services required to sustain these outcomes? Have the projects identified the potential service providers? What are the motivations of the service providers to continue service provisioning after the projects end?  What would be the motivation of the beneficiaries to receive these services? Have the projects identified the resources and capacity strengthening supports for the service providers?  What has the project done to ensure that this motivation does not diminish after the project ends? We’d like the MTE to also look at the extent government officials, formal and informal local leaders (whose support and understanding will be critical for continuing program initiatives NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-167 once the project has ended) are involved in project activities and included in ongoing program discussions. Analysis of activities being implemented and how they are implemented will assist in establishing stronger processes to ensure that the service providers will have continuous access to required resources and capacity strengthening support, creating demand and influencing the motivations of the beneficiaries and service providers. This will assist in establishing and strengthening critical linkages necessary to sustain resources, and sustain capacities, and the external factors that may positively or negatively influence sustainability.  Have the projects identified the indicators and planned for a phased transfer of responsibilities yet? For Objective 5: 1. How effective are program design and implementation mechanisms in addressing the cross￾cutting issues of gender, governance, the environment and targeting of youth?  What (if any) challenges have projects encountered in these areas that may not have been anticipated in the project design, and how have the projects responded? To what extent do project interventions and implementation mechanisms reflect integration of these cross-cutting priorities?  What steps have the activities taken to ensure that staff has adequate capacity for addressing these cross-cutting issues?  In what ways is the project changing roles, relationships, communication and decision- making dynamics among women and men, young and old, in relationship to food security at the household and community levels? Findings and recommendations of the Year 1 gender analysis have been incorporated into the program strategy. We’d like the MTE to analyze if these findings have been considered. What specific changes were made? Have gender gaps and related concerns been addressed adequately? Is the project drawing on the potential of women, men, boys and girls as much as possible? For Objective 6: 1. Based on the findings from Questions 1–5, how could the activity be modified to improve its acceptability to targeted communities or the efficiency and effectiveness of its implementation? How should the activity’s TOC and Log Frame be refined or modified?  The extent to which the project resources (inputs) have led to the achieved results. An assessment of efficiency should also consider whether the same results could have been achieved with fewer resources or whether alternative approaches could have been adopted to achieve the same results After analyzing the evidence collected to answer the other evaluation questions, the MTE team should form conclusions and recommend and prioritize concrete actions to help improve activity implementation during the remaining activity years. Conclusions should consider both strengths of and challenges to implementation. When prioritizing the recommended actions, the evaluation team should consider the beneficiaries’ priorities; the cost, feasibility, and benefits of action; and the potential to advance the activity’s ultimate Purposes and Goal during the remaining implementation period. NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-168 7. Evaluation Methodology The MTE will be in form of an external review and will be conducted by an independent external Consultant. The MTE team is expected to comprise expatriate and local experts with expertise in relevant program sectors (e.g., MCHN, food security, WASH, DRR, gender, empowerment and Good Governance) and in Bangladesh. It will use a mixture of qualitative and quantitative methods. The collection of primary data will involve mostly qualitative methods, especially unstructured or semi-structured interviews and observation. Large-scale quantitative survey to collect primary data is not expected. Considering the purpose and objectives, the consultant will be responsible to define the MTE methodology and approach including the tools, techniques, and analysis of data collection. The evaluation approach, tools, techniques and overall methodology will be reviewed and validated by different stakeholders and approved by the donor. To meet the data requirement for objective 2, the consultant may need some quantitative data. The primary sources of the quantitative data would be baseline survey results, existing M&E data (census survey, beneficiary based sample survey results, routine monitoring data) available. However, small scale beneficiary based quantitative survey or other measures, if deemed necessary by the consultant, can be considered with approval from the donor. The Consultant should explain the purpose, sampling methods, and potential biases and representativeness for such quantitative survey. For example, the small scale beneficiary based survey can be used to provide additional information on the changes in behavior, practice, attitudes and knowledge that individuals have made themselves as well as for making conclusions from the extensive qualitative assessments. Contractors should consider the existing quantitative data and potential qualitative data in considering whether or not to conduct a small scale beneficiary based survey on the following focus areas: maternal and child health and nutrition, water supply, sanitation and hygiene, market-based agriculture production with natural resource management and income generation, gender equity, community based disaster preparedness and social accountability of service provision by local government bodies. Results from these surveys should not be extrapolated beyond the local context. This data can be used as supplement for making the conclusions from the extensive qualitative findings. The quantitative data will also provide an indication of the direction and understanding that NJP is heading towards achieving its outcomes. The qualitative and participatory approach will include all key stakeholders, including sub￾grantees, consortium members, frontline extension volunteers and project participants. The process will include:  Desk review of existing literature, including project proposal, reports, base line report, formative research, M&E data, MIS system and other relevant quantitative and qualitative secondary data to establish and assess the theory of change of the project, implementation strategies, allocations of resources, and relevant results.  Qualitative data collection conducted with beneficiaries and stakeholders. Within each community, several subgroups will be selected for focus group discussion to better understand the effectiveness of key interventions in the project. Focus Group Discussion (FGD)s may include: Community Nutrition Facilitators, Community Clinic Support Groups, WATSAN Committees, community leaders, farmers group members, Lead Farmers, Graduation Participants, VDC members, Union Disaster Committees and other project related stakeholder groups. Respondents for the focus group discussions will be randomly selected to minimize selection bias. Key informant interviews will be conducted with the donor, implementing partners, government officials, and Union Parishad administrators. In-depth interviews (IDIs) will be conducted from direct, indirect and non-beneficiaries in targeted communities. NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-169  The evaluation must also include individuals who live in targeted communities but who have not directly benefited from any program intervention, as well as individuals who were excluded from the program interventions (looking for evidences of both inclusion and exclusion errors) and to gather outsiders’ impressions of the interventions’ implementation and relevance. Furthermore, learning about non-beneficiaries’ practices and changes during the activity implementation period will help evaluators distinguish which reported and observed changes might be associated with activity interventions and which are related to general shifts in the context. 8. Audience and key stakeholders There is a broad group of stakeholders who are interested in the achievements of the Nobo Jatra project. Given that an important component of the mid-term evaluation is to provide recommendations to make necessary course corrections to implementation, the primary stakeholders will be direct beneficiaries of the project, project staff and implementing partners (WVI, WFP, WI), Ministry of Disaster Management and Relief and the donor and USAID. Participant Main task USAID  Give inputs, recommendations and approval to the MTE documentation, MTE team and implementation plan.  Advise the MTE team in terms of methodologies and approaches  Be part of key respondents  Review and approve the final MTE report Ministry of Disaster Management and Relief (MoDMR)  Give inputs to the tools  Participate in field data collection, supervising the process Target households Participate as MTE respondents during field data collection VDC, WATSAN Committee, UDMCs, UP Standing Committees and other community groups Participate as MTE respondents during field data collection 9. Roles and responsibilities During data collection and analysis, the primary role of the program staff and staff from any implementing partner with a direct stake in the project are as informants and reviewers. They may review and provide comments on data collection tools and instruments before they are finalized. They must not collect primary data, or participate in translation, analysis, or interpretation of the data. The following will be the key roles and responsibilities for the mid-term evaluation participants: Position Activity Chief of Party  Give orientation for MTR team and provide advice  Provide approvals for any expenses related to MTE  Provide the review team with project documents for review  Coordinate the MTR exercise both to office and field work, in terms of follow up the project MGD NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-170 Position Activity  Oversee results dissemination with key partners  Advise on dissemination results of MTE Design, Monitoring and Evaluation Specialist (US office)  Provide final feedback to the MTE tools  Review the ToR  Provide inputs for external consultants or MTE team  Participate in MTE reporting process  Review the MTE report  Oversee quality assurance of the mid-term evaluation  Comment on the deliverables of the mid-term evaluation  Oversee results dissemination with key stakeholders  Advise on dissemination results of MTE M&E Manager  Ensure that the review team has all logistical support necessary to conduct the process  Review the tools for data collection  Provide selection of enumerators and supervisors for MTE process  Prepare documentations to support the MTE  Monitor the data collection  Monitor the conduction of the project review by the mid-term evaluation team.  Oversee quality assurance of the mid-term evaluation  Ensure that all deliverables of the project review meet the expectations of Nobo Jatra Project  Review the 1st draft report of MTE and submit to Nobo Jatra Management team  Comment on the deliverables of the review  Oversee results dissemination with key stakeholders. External consultants  Develop tools for MTE and submit to the M&E team  Conduct training for enumerators and supervisors  Review project documentation  Undertake data collection using the agreed upon tools.  Provide progress updates to Quality Assurance  Provide data analysis and report writing  Complete deliverables/products  Submit report to M&E team for revision Mid-term evaluation field team (Deputy Chief of Party, Field Office Coordinators, M&E Officers, Technical Managers)  Participate in training as MTE supervisors  Supervise all field data collection  Give necessary support for MTE logistics, as well during the field data collection  Provide any additional support for the consultant when needed  Participate in dissemination process of MTE results Partners (WFP, WI) Govt. and other Project staff  Provide feedback to MTE tools (Agriculture and DRR component)  Participate in all meetings with Mid-term evaluation team  Provide feedback and information on the project as requested for the Mid-term evaluation  Provide feedback on the deliverables of the mid-term evaluation Additional stakeholders (beneficiaries, on the ground partners as UP chairs)  Participate in meetings with mid-term evaluation team, discussions, agreements about the process, etc.  Provide feedback and information as requested for the MTR NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-171 Position Activity  Participate in evaluation debrief and stakeholders meeting USAID  Review the MTE TOR and give recommendations to operationalize the process  Provide relevant information about the project procedures, approaches and operations through MTE questions addressed to the main key informant, prior to the field data collection.  USAID officers may provide comments on, and will give final approval for, the evaluation plan, data collection tools, timeline and logistical plan before the MTE consultant begins field work.  Participate in discussions regarding the evaluation findings and recommendations (evaluation debrief and stakeholders meeting) 10. MTE Team Composition, Qualifications, and Roles The MTE Team Leader (TL) will be the primary implementer of the MTE. S/he will provide oversight to the evaluation throughout the entire evaluation cycle. The MTE TL will work in close collaboration with the COP and USAID at all times, ensuring good communication, receiving and acting on all inputs and comments, and calling meetings as necessary to resolve any problems that arise during the process. To ensure the security of MTE team members, all movement within the country will be made following the country security protocol. The individual team members will manage their own security, World Vision will have no liability to assure it. If the security situation permits, and the MTE team requests, WVB may assist the MTE consultant to rent unmarked vehicles for the team's use. To ensure the MTE is as unbiased as possible and to avoid the disruption of project implementation that could affect the results of the evaluation, the MTE consultants will not use WVB’s marked vehicle and project staff as translators, primary data collectors, or supervisors. The MTE team should present themselves as separate from WVB as much as possible, but more importantly, the program activities must not be compromised by a lack of transportation. The MTE consultant/ team members will be responsible for adhering to national and local formalities and all security protocol while in-country. The MTE consultant will be responsible for obtaining visa(s) and provide to WVB a written proof of health and accident insurance for all foreign personnel working with the MTE consultant in Bangladesh. MTE Team Composition The MTE team will typically consist of a team leader who is an evaluation specialist, plus three to five technical specialists (WASH, MCHN, Income & Agriculture, Gender, DRR) who bring expertise and practical experience in one or more of the activity’s technical sectors and interventions in addition to strong qualitative research skills and experience. The team may include a data analyst experienced in analyzing and relating data across various technical sectors. The MTE should be a mix of international and national members with gender consideration. MTE Team Members’ Qualifications • Every team member’s resume must show substantial application of qualitative research skills in developing countries. • The team leader must have significant formal education in a field relevant to NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-172 evaluation (e.g., program evaluation, statistics, economics, agricultural economics, anthropology, applied research, organizational development, sociology, or organizational change) at a post-graduate or an evaluation professional continuing￾education level. The team leader must have extensive experience in evaluation using mixed methods of investigation (qualitative and quantitative) in developing countries- Bangladesh and/or south Asia is preferable. Knowledge of the conceptual framework of food security and experience evaluating food security programming is highly desirable. • Each technical specialist should have a post-graduate degree in a field related to at least one of the technical sectors of the activity, plus extensive practical experience in developing countries with interventions similar to those implemented by the activity. • At least one member of the team must have substantial demonstrated experience in gender integration. • The MTE team should comprise technical expertise from all activity sectors and activity management. • No member of the MTE team should have had any prior input to the activity’s design or implementation. • Having conflict sensitivity expertise on the MTE team is also desirable, particularly when an MTE will be undertaken in a context that is prone to violent conflict. The team must include expertise from all of the program technical sectors, including but not limited to: logical framework and theory of change, M&E systems including automation, Knowledge Management systems, staff management, partnerships, food security, SMART Agriculture/NRM, alternative livelihoods, extreme poverty graduation, conditional cash distribution, safety nets, entrepreneur literacy, value-chains, savings groups, WASH, nutrition, climate change and resilience, gender, social behavior change communication (BCC) and governance & social accountability. MTE Team Members’ Roles Team Leader: • Organize and lead the overall evaluation • Ensure a thorough review and analysis of activity monitoring data and other available secondary data by the appropriate team members • Lead the selection of a purposively selected sample of implementation sites and Outputs for primary data collection • Ensure an MTE plan that includes adequate triangulation and validation of evidence collected in all sectors • Lead the collection and analyses of primary and secondary data to evaluate the activity’s M&E processes and the integration of activity sectors and interventions • Ensure that final report presentation is logical and presented in a way that clearly separates the evidence collected, conclusions, and recommendations in different sections of the report, and conclusions and recommendations are based only on the evidence presented in the report • Interact, on the part of the MTE team, with the COP and USAID NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-173 • Serve as a technical specialist for specified sector (optional) Technical Specialists: • Lead the collection and analyses of primary and secondary technical data related to his/her field(s) of expertise, document findings, and draw conclusions and form recommendations for the sector(s) • Evaluate the general aspects of the implementation of all interventions related to his/her sector(s). While the team leader will likely be tasked as the primary investigator for the activity management overall, a technical specialist must consider management aspects of the implementation of interventions in his/her technical sector and the interaction between his/her technical sector and other activity sectors by examining:  Staff and material resources  Communication, both internal and external  Community involvement  Beneficiary targeting (especially overlap/consistency with other sectors)  Management of Cash transfers of entitlements  Branding  Partnerships and linkages  Consortium management  Routine monitoring and data quality assurance for all interventions  Exit/sustainability strategies  Gender integration  Environmental protection  Draft the report sections assigned by the team leader in the specified format 11. Deliverables/Products of MTE The MTE consultant must adhere all the requirements including USAID/ FFP M&E Policy and Guidance, relevant ADS Chapters, DEC, etc. The main products of mid-term evaluation will be as follows:  Inception Report: will include the proposed evaluation approach, Key Evaluation Questions matrix, sampling approach and timeline.  MTE Budget: The MTE proposal should include a reasonable detailed budget to cover all costs associated with the MTE. This should be submitted by major activities and line items for review and decision. This would break-down the cost of contracted MTE team members, international and local travel, and in-country lodging and per diem. Other related costs that might be in the budget include expenditures for hiring local personnel (drivers, translators, enumerators and other local technical experts), translating reports, and renting meeting rooms for presentations.  MTE Implementation Plan: will include the final overall mid-term evaluation process, updated timeline with detailed calendar of key activities and milestones, supervisor and enumerator training manual/guidance (English and Bengali24), sampling approach and methods, data collection tools, validation (data quality assurance), data analysis plan, showing how each question will be analyzed from the data collected and roles, responsibilities, anticipated outputs and associated levels of effort of each of the evaluation team members 24 The primary product should be in English, translated into Bengali and back translated into English to ensure correct and quality translation. NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-174  Lists of sites visited with types and numbers of informants at each, list of stakeholders interviewed and consulted  Digital transcripts of interviews and discussions conducted as part of the MTE, such as FGDs, KIIs, and IDIs.  Cleaned primary qualitative and quantitative (if applicable) data set  Draft Report for review  Presentation of findings to the project and possibly stakeholders of the evaluation findings, conclusions, recommendations, lessons learnt and good practices.  Final MTE Report approved by USAID. The MTE Consultant must follow the USAID’s evaluation policy requiring that the report describe the strengths and limitations of the evaluation methods and how and to what degree these factors influenced the process and findings of the evaluation. The report must clearly separate in different sections the evidence (i.e., raw data) collected by the evaluation team, the conclusions and recommendations that are based on the presented evidence. It should also state that sources of all evidence must be identified; conclusions must be based only on evidence presented in the report; and recommendations must directly correspond to the conclusions. The MTE contractor should deliver all files, including, quantitative data sets (raw and refined products, in proprietary and non- proprietary format) if applicable, transcripts of qualitative data and others in an easy to read format; organize them professionally; and maintain naming conventions and labelling for the use of Nobo Jatra program and key stakeholders and ready for posting to DEC and DDL. All documents should be 508 compliant before submission to DEC. 12. Timeline The mid-term evaluation (document review, inception report, implementation plan and instrument development, training, data collection, analysis and report writing, reviews, and documentation) is scheduled to take place within the months of March 2018 and September 2018 and a tentative timeline is indicated, but can be revised by the MTE team with prior consultation with the project management and donor. Additionally, if proposing small scale beneficiary based quantitative survey, MTE team will need to update timeline accordingly as part of the implementation plan.  MTE Task/Output Dates Who is Responsible Terms of Reference finalized, including budget and obtained approval from USAID August 2017 Person commissioning review, in consultation with those funding it. ToR Advertised August 2017 Coordinated by M&E team Submission of proposal bids October 2018 Supply Chain Manager - Nobo Jatra Program Review of proposals October 2017 to November, 2017 Project review team Finalize MTE team and obtain approval from USAID January 2018 COP Provide feedback on proposal to MTE Team February 2018 COP and WV-US team Submission of revised proposal and budget from MTE Team to WVB February 2018 Supply Chain Manager - Nobo Jatra Program Contract agreed and signed by both parties March 2018 COP and WV-US team MTE Team begins assignment in Bangladesh March 2018 M&E Team MTE team holds meetings with Nobo Jatra management, partners and USAID to better understand the Program to help design the methodology, March 2018 Nobo Jatra Senior Management team and Partners NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-175 MTE Task/Output Dates Who is Responsible approach, tools and questionnaires to be applied in the MTE MTE Team submit inception report, including evaluation approach and Key Evaluation Questions matrix March 2018 M&E Team TL submits MTE Implementation Plan, including updated timeline, detailed methodology, questionnaires and tools to be applied in MTE April 2018 COP and M&E Team Question guides and Questionnaires (if propose small scale survey) approved April 2018 COP/USAID Secondary data review period for MTE team March-April 2018 M&E Team Training for enumerators and field data collection conducted as per MTE implementation plan (qualitative and quantitative, if proposed) April-June 2018 MTE team Data analysis and report writing June 2018 MTE team MTE Team provides briefing on preliminary findings July 2018 MTE Team briefs Senior Management team, Partners and M&E team MTE report drafted and circulated to relevant staff July 2018 MTE Team to write and pass to the COP and M&E team Project team review report findings as part of validation workshop July 2018 M&E Manager runs process. MTE report finalized and approved by USAID July 2018 MTE Team finalizes the report based upon comments received. Project Manager reviews, gives final approval of report and submit to donor Presentation of MTE results to project staff, WV projects and programs, and relevant partners August 2018 M&E team MTE Consultants submit digital copies of all evaluation documents and data sets to NJ M&E team August 2018 MTE team 13. Key documents to support the MTE The following documents will be provided to the mid-term team for review:  Nobo Jatra’s Technical Proposal and Budget  Project Results Framework  Assessment and Baseline Reports  Quarterly and Semi-annual Narrative Reports  Performance Monitoring Plan  Performance Indicator  Cash distribution post monitoring reports  Project Monthly, quarterly, semi-annual and annual Reports Mid-term Report Structure (max of 40 pages)  Title Page  Table of Contents  List of Acronyms and Abbreviations  Executive Summary  Introduction  Brief project description  Evaluation purpose and objective NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-176  Summary of the Mid-term evaluation methodology, including limitations of the evaluation methods  Principle findings and discussions  Conclusion and key recommendations  Summary of lessons learned Note: Please note that WV will communicate if the Donor has a specific Template to be used for the Report. The consultant will review the ADS 201 and should comply the requirement of evaluation reports (https://www.usaid.gov/sites/default/files/documents/1870/201.pdf) Pertinent Permissions, Approvals, Insurance, and Other Required Permits The MTE team will be responsible for obtaining all necessary permissions, approvals, insurance, and other required permits and for adhering to national and local formalities. These include required permits related to data collection from human subjects, including necessary internal review board approvals and health and accident insurance for MTE team members. Logistics The MTE team will be responsible for the evaluation logistics including accommodation, transport, foods. To avoid compromising activity implementation during the MTE and to maintain a separation between the MTE team and the implementers, activity vehicles and other vehicles branded to identify them with any of the implementing partners may not be used by the MTE team while they are in the activity area. The MTE team is expected to-  Arrange and manage their own administrative support, communication, photocopying, printing, binding, etc.  Arrange the virtual meetings and set-up for all electronic communication and data entry, processing and overall management  Arrange computer and other electronic devices e.g., audio-visual instruments and authentic software require for the MTE including data collection, entry, processing and reporting The consultant must keep this in their prime consideration how to manage if any unexpected situation occurs, such as a strike, political movement, etc. that may affect the MTE. They should keep options for contingency plan and alternatives without compromising the overall quality, purpose and timeline. 14. Intellectual Property The ownership of the report for publication rests with World Vision Bangladesh. All data and the reports including the findings and recommendations will remain the property of World Vision Bangladesh and must not be published or shared with a third party by the consulting agency. The completed data set and report will be the sole property of World Vision and USAID. The consultant should not have any right to use the data for its own research purposes, nor license the data to be used by others, without the written consent of USAID & World Vision unless otherwise provided in the award’s provisions. World Vision may retain the rights, title, and interest to data that are first acquired or produced under the award. In addition, “USAID reserves a royalty-free, worldwide, nonexclusive, and irrevocable right to use, disclose, reproduce, prepare derivative works, distribute copies to the public, and perform and display publicly in any manner and for any purpose and to have or permit others to do so.” 15. Ethical Guidelines Each member of the evaluation team must adhere to ethical guidelines as outlined in the American Evaluation Association’s Guiding Principles for Evaluators. A summary of these guidelines is provided below. NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-177  Systematic inquiry: Evaluators conduct systematic, data-based inquiries.  Competence: The evaluation team possesses the education, abilities, skills, and experience appropriate to undertake the tasks proposed in the evaluation. Evaluators practice within the limits of their professional training and competence, and decline to conduct evaluations that fall substantially outside those limits. The evaluation team collectively demonstrates cultural competence.  Integrity/honesty: Evaluators display honesty and integrity in their own behavior, and attempt to ensure the honesty and integrity of the entire evaluation process.  Respect for people: Evaluators respect the security, dignity, and self-worth of respondents, activity participants, clients, and other evaluation stakeholders. Evaluators regard informed consent for participation in evaluation and inform participants and clients about the scope and limits of confidentiality.  Responsibilities for general and public welfare: Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation. 16. Expression of Interest Interested consultants may submit a narrative proposal (5 pages maximum) and detailed budget by (date), highlighting information indicating that they are qualified to perform the services. The proposal shall include the following documents:  Consultant’s profile and CV (s)  Past consultant’s performance  Past evaluation or project review reports  Proposed methodology  Proposed detail budget  Cover letter stating why you are interested to conduct mid-term evaluation A consultant will be selected in accordance with the procedures set out in World Visions Bangladesh Procurement guidelines selection and employment of consultants. (Contact names, address, email; cc DME Specialist WVUS) Note: Expressions of interest for this consulting service received after (date) will be rejected. 17. List of Annexes Annex-A: Logical framework (LogFrame) Annex-B: Theory of Change (ToC) Graphic & Narrative Annex-C: Indicator Performance tracking Table (IPTT) Annex-D: Project Performance Indicators Reference Sheet (PIRS) Annex-E: M&E Plan narrative Annex-F: USAID Monitoring and Evaluation Policy 18. Resources i) Baseline survey report ii) Project implementation guideline iii) Different formative research reports iv) Different training module(ToT, ToF, Entrepreneur literacy, WATSAN Committee training, UDMC training, VDC training, MenCare training, Life skill, leadership training for youth etc) v) WASH SBCC module vi) Gender analysis report vii) Initial Environmental Examination (IEE) report NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-178 viii) Environmental Mitigation and Monitoring Plan (EMMP) ix) Safer Use Action Plans (SUAP) NOBO JATRA PROJECT/“NEW BEGINNING”: DRAFT REPORT FOR THE MIDTERM EVALUATION (MTE) A-179 Annex F. Ratanpur Union budget activities