1 April 2017 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Richard B. Daite, Normito R. Zapata Jr., and Divine Krizza B. Padua, all from the University of the Philippines Los Baños. Evaluation of the Activity on “Catalyzing Entrepreneurship Among Iloilo’s Ultrapoor” EVALUATION OF THE ACTIVITY ON “CATALYZING ENTREPRENEURSHIP AMONG ILOILO’S ULTRAPOOR” April 2017 Contract Number: AID-492-O-16-00111 Photo credits: Evaluation Team. Clockwise from upper left: Houses of activity participants in an urban slum area; some products being sold by participants of the activity; houses in a poor rural area; participants in a focus group discussion for this evaluation. DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT This cursory evaluation was done to gather indications of emerging results and outcomes of the USAID-assisted activity entitled “Catalyzing Entrepreneurship Among Iloilo’s Ultrapoor” (CEAIU). In this activity, ultrapoor participants underwent CEAIU’s 16-session training on livelihood, health, and values designed to help them build skills and capacities in putting up micro￾businesses. Results showed positive changes in participants’ household income after the training. However, the evaluation cannot make any conclusion on the contribution of CEAIU to the increases in income. It found out that the activity positively influenced the savings behavior of sample participants. There were also indications that participants sustained awareness of health-related practices that they learned from the activity. CEAIU provided a good opportunity to build entrepreneurial skills among the participants. However, continued use of these skills by the respondents was observed in only three of the ten sample communities. The link of CEAIU with USAID’s Cities Development Initiative (CDI) was apparent at the levels of sub-purposes and outputs of the latter’s logical framework. The connection, however, appears uncertain with the indicators intended to deliver the CDI outputs. The target deliverables of the activity were realized. Activity implementation and monitoring is commendable. Differences in outcomes across communities highlight the need to design the training to consider community characteristics. Sustaining immediate results long after the activity has concluded may be related to community location, involvement of the pastors, and interest among the participants. CONTENTS Acronyms ........................................................................................................................................................................... 1 Executive Summary ........................................................................................................................................................ 2 Evaluation Purpose & Questions ...............................................................................................................................7 Activity Background .......................................................................................................................................................9 Evaluation Methods .................................................................................................................................................... 13 Limitations ...................................................................................................................................................................... 18 Findings, Conclusions and Recommendations ................................................................................................. 19 Annexes............................................................................................................................................................................ 77 - Annex I: Evaluation Statement of Work ............................................................................................................ 78 - Annex II: Evaluation Methods and Limitations............................................................................................... 88 - Annex III: Data Collection Instruments.............................................................................................................. 89 - Annex IV: Sources of Information ..................................................................................................................... 116 o Sites Visited o Respondents Interviewed By Community o Documents Reviewed o Database Reviewed - Annex V: Disclosure of Any Conflicts of Interest……………………………………………………………………..121 1 ACRONYMS BiB Business-in-a-Box CDI Cities Development Initiative CEAIU Catalyzing Entrepreneurship Among Iloilo’s Ultrapoor DCA Development Credit Authority EQ Evaluation Question FGD Focus Group Discussion GiB Garden-in-a-Box GIDA Geographically Isolated and Disadvantaged Areas HBF Home Based Feeding KII Key Informant Interview ICM International Care Ministries Foundation, Inc. M,E,& L Monitoring, Evaluation, and Learning PFG Partnership for Growth PH Philippines PHP Philippine Peso RHU Rural Health Unit SG Savings Group SMEs Small and Medium-sized Enterprises SOW Statement of Work SURGE Strengthening Urban Resiliency for Growth with Equity USAID United States Agency for International Development USD United States Dollar USG United States Government ToC Theory of Change 2 EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS This cursory evaluation was done to gather and validate emerging results and outcomes of the USAID-assisted activity entitled “Catalyzing Entrepreneurship Among Iloilo’s Ultrapoor” (CEAIU). Specifically, the evaluation sought to identify patterns of results among the intended beneficiaries in terms of household well-being and to identify lessons learned in activity implementation. It also analyzed the activity’s possible links with the Cities Development Initiative (CDI), a USAID project. The design of this evaluation was guided by the four evaluation questions identified in the statement of work (SOW), as enumerated below: 1. What major changes in household conditions and well‐being are emerging and will occur most likely as a result of USAID assistance to the CEAIU activity? 2. What key outputs and emerging outcomes of the CEAIU activity contribute to the Cities Development Initiative (CDI) project outcomes? 3. What are some major lessons learned to improve the effectiveness of CEAIU development strategies and the sustainability of its outcomes? 4. What are some key CEAIU characteristics and enabling factors/conditions that enhance the scalability and replicability of the CEAIU development approach? ACTIVITY BACKGROUND The CEAIU activity was implemented by the International Care Ministries Foundation, Inc. (ICM) and co-funded by the United States Agency for International Development-Philippines (USAID/PH) with a grant period from September 2, 2015 to February 1, 2017. CEAIU generally aimed to “facilitate inclusive economic growth for approximately 10,000 ultra-poor individuals from 2,000 households by delivering ‘Transform’, an intensive, community-based, 16-week training program in different communities near the CDI city of Iloilo”. Specifically, the training aimed to help ultrapoor participants become entrepreneurs and improve household wellbeing. CEAIU defined “ultrapoor” as those individuals who survive with PHP 22 (USD 0.50) in a day. The training involved a half-day session once a week for 16 weeks where the three modules on values, health, and livelihood were taught. These modules were designed to provide skills and capacities to participants for them to have hands-on exposure in running micro-businesses. As 3 of September 2016, there were 2,552 graduates from around 3,600 training participants in all three batches (A, B, and C). This evaluation covered only the results for batches A and B. EVALUATION DESIGN AND METHODS The evaluation used both quantitative and qualitative methods in order to gather insights on performance, realization of outcomes, and support to CDI activities. The quantitative component used data from the CEAIU database and from interviews of sample participants and other respondents. Qualitative methods include key informant interviews (KIIs) and focus group discussions (FGDs). Review of relevant documents, reports, and other secondary literature was also done. Much of the results of this evaluation came from the fieldwork in sample CEAIU communities in the province of Iloilo. These communities were randomly selected from participating towns in the province. From the FGDs and KIIs common threads of responses were analyzed which formed the bases of the answers to the relevant evaluation questions. In summary, the evaluation performed the following general methods: • Analysis of logframes • Review of relevant documentation and activity database • Collection and analysis of interview data from activity administrators, implementers, and participants through key informant interviews and focus group discussions during the field visit in Iloilo Province • Identification of lessons learned and areas of sustainability. LIMITATIONS This is a quick evaluation of activity performance and emerging results. Data limitation hindered the establishment of counterfactual information which could have provided more definitive conclusions on the effects of the activity on the participants. Nonetheless, the evaluation methods provided indications of results and identified important issues. In examining changes in entrepreneurial skills of participants, the literature suggests the use of very detailed questionnaires on entrepreneurial competencies. Since such information was not collected from the CEAIU participants, this evaluation gathered indications of entrepreneurial skills build-up from qualitative information provided by the FGDs and KIIs. The number of sample participants and communities covered in the evaluation was also a limitation. The use of FGDs and KIIs was the recourse in order to gather results adequate to answer the relevant evaluation questions. 4 FINDINGS On EQ 1: What major changes in household conditions and well‐being are emerging and will occur most likely as a result of USAID assistance to the CEAIU activity? • There were significant positive changes in mean household income among the CEAIU participants. However, there were no strong indications that those changes were definitely a result of the CEAIU activity. The rise in mean household income were primarily due to participants’ switching to wage work types with higher earnings, additional working household members, and possible changes in market conditions which were all beyond the direct control of the activity. • The activity was able to positively influence the savings behavior among the sample participants. They reported positive amounts of savings where little or none before the activity. • There were indications that participants exhibited sustained awareness of health-related practices that they learned from the activity. However, due to the lack of expert evaluation, no conclusion was made on the changes in their health status. • The activity provided a good opportunity to build entrepreneurial skills among the more successful participants. However, such competencies were apparent only in the respondents in three of the ten sample communities. On EQ 2: What key outputs and emerging outcomes of the CEAIU activity contribute to the Cities Development Initiative (CDI) project outcomes? • There appeared a connection between the CEAIU and the CDI in terms of sub-purposes and outputs in the latter’s logical framework. There were, however, unclear links with the indicators intended to deliver the CDI outputs. On EQ 3: What are some major lessons learned to improve the effectiveness of CEAIU development strategies and the sustainability of its outcomes? • CEAIU was effective in transferring skills and knowledge on livelihood and health. However, only participants in the three of the ten sample communities were able to sustain the enterprises they learned from the training. The most cited problems were the: (1) lack of operating capital for the next rounds of production after the training; and (2) difficulty in sourcing of ingredients. • The target deliverables of the activity were realized. Implementation and M&E is 5 commendable. However, the mobile banking facility was not actually used for their intended purposes. On EQ 4: What are some key CEAIU characteristics and enabling factors/conditions that enhance the scalability and replicability of the CEAIU development approach? • Respondents in only three of the ten few sample communities showed some success in applying their learnings from the activity. There were indications that sustaining the immediate results of the activity was dependent on characteristics such as community location, involvement of the pastors, and interest among the participants. CONCLUSIONS 1. The CEAIU activity was successful in providing knowledge and skills to participants, specifically on livelihood and health. However, most of the respondents did not sustain their BiB-related enterprises after the activity. 2. While there were positive changes in income among sample respondents, this evaluation found no indication that these were attributed to CEAIU. The changes in income were found to be due primarily to participants’ switching to wage work types with higher earnings, additional working household members, and possible changes in market conditions. 3. The positive amounts of savings among sample respondents can be directly attributable to CEAIU. The activity enjoined participants to form savings groups and regularly contribute some amount. However, among the sample communities, majority of these savings group have become inactive. 4. The participants exhibited sustained awareness of health-related practices that they learned from the activity. These include mindfulness and use of good hygienic practices and seeking medical professional help when needed. However, how this led to improved health well-being was not ascertained by the evaluation. 5. The activity provided a good opportunity to build entrepreneurial skills among the more successful participants. These were manifested in their capacities for networking, risk￾taking, opportunity seeking, and systematic planning for their new enterprises. 6. The link between the CEAIU and CDI was apparent in the sub-purposes and outputs of the latter’s logical framework. However, the connection is uncertain with the indicators intended to deliver the CDI outputs. The CDI is a top-level initiative which provides technical assistance mostly to city planners and implementors on various long-horizon development plans and programs. CEAIU is a grassroots-level, short-term training program in direct contact with ultrapoor beneficiaries. 6 7. Implementation and M&E of the activity were commendable, with excellent data capture and analysis. The target deliverables of the activity were realized and exceeded. It is premature to conclude, however, if these immediate outputs of the CEAIU would ultimately lead to the realization of the higher level outcomes of improved household well-being. 8. Even though the CEAIU was uniformly implemented across communities, differences in outcomes highlight the need to design the training according to community characteristics. Only three of the ten sample communities showed some success in continuing on with the micro-enterprises they learned from the activity. There were indications that the success of sustaining such enterprises long after the activity has concluded may be dependent on characteristics such as community location, involvement of the pastors, and interest among the participants. RECOMMENDATIONS Based on the findings and conclusions, the following are the recommendations of this evaluation: 1. The CEAIU participants stand to benefit more if the training design is suited to the community’s needs and conditions. For example, the BiBs introduced to each community should be subject to feasibility analysis in order to address the issues on input sourcing and marketing for the product. 2. The choice of participating communities should consider the different characteristics of the relatively more successful ones. The commitment of the pastor, especially after the training, was crucial to sustain the results of the activity. Communities near the town centers also did not show much success. While considering these specific characteristics may not guarantee success in all cases, the recommendation is to provide a careful analysis of potential communities to be included. 3. A relay activity for financing successive rounds of production of the enterprises of CEAIU graduates is worth considering. One of the main findings of this evaluation points to the difficulty of participants in obtaining operating capital after the training. Implementing this relay activity can benefit from the strong M&E capacity shown by the CEAIU implementors. 7 I. EVALUATION PURPOSE & QUESTIONS EVALUATION PURPOSE This evaluation was designed to assess the accomplishments and emerging outcomes of the recently-concluded CEAIU activity implemented by the International Care Ministries Foundation, Inc. (ICM). The activity was co-funded by the United States Agency for International Development-Philippines (USAID/PH) with a grant period from September 2, 2015 to February 1, 2017. The CEAIU generally aimed to “facilitate inclusive economic growth for approximately 10,000 ultra-poor individuals from 2,000 households by delivering ‘Transform’, an intensive, community-based, 16-week training program in different communities near the CDI city of Iloilo” 1 . Specifically, the training aimed to help the ultrapoor participants become entrepreneurs and improve household wellbeing. CEAIU defined “ultrapoor” as those individuals who survive with PHP 22 (USD 0.50) in a day. CEAIU implemented three batches (A to C) of the Transform training with Batch C concluded in September 2016. This evaluation covered only Batches A and B which were completed in January 2016 and May 2016, respectively. There were 1,632 graduates from these two batches. This cursory evaluation of activity performance and emerging outcomes broadly examined planned versus realized outputs of the activity and their possible links with the Cities Development Initiative (CDI), a USAID project. It evaluated the activity design and implementation and verified results in the field. This evaluation provided an evidence-based assessment of indicative results useful for USAID to gather preliminary appreciation of emerging outcomes and to identify lessons learned. It also proposed recommendations in terms of the scalability potential and replicability of the CEAIU model. EVALUATION QUESTIONS The design of this evaluation is guided by the four evaluation questions identified in the statement of work (SOW), as enumerated below. 1. What major changes in household conditions and well‐being are emerging and will occur most likely as a result of USAID assistance to the CEAIU activity? 1 Attachment 2 of the CEIAU Grant Award No. AID-492-F-15-00003. 8 2. What key outputs and emerging outcomes of the CEAIU activity contribute to the Cities Development Initiative (CDI) project outcomes? 3. What are some major lessons learned to improve the effectiveness of CEAIU development strategies and the sustainability of its outcomes? 4. What are some key CEAIU characteristics and enabling factors/conditions that enhance the scalability and replicability of the CEAIU development approach? 9 ACTIVITY BACKGROUND CEAIU defined “ultrapoor” as those individuals who survive with PHP 22 (USD 0.50) in a day. The activity focused on communities in Iloilo Province located in Geographically Isolated and Disadvantaged Areas (GIDA). These areas were classified into five categories, namely: 1) coastal; 2) urban slum; 3) rural slum; 4) rural mountain; and 5) rural plain. CEAIU completed three batches (A to C) of the Transform training covering 55 towns or cities in Iloilo province (Figure 1). Each batch included 30 communities. Each community involved at least 30 participants, one pastor, and six volunteer-counselors. Batch C was concluded in September 2016. Figure 1.Iloilo Communities that hosted CEAIU Batches A, B and C Source: CEAIU Final Report The Transform training was designed to address key issues in entrepreneurship among the ultrapoor such as poor health and lack of positive experience in creating new businesses2 . It had three modules on values, health, and livelihood. The livelihood module taught participants basic business skills and provided hands-on exposure in running microbusinesses. The module was designed to stimulate the “entrepreneurial spirit” of the participants by allowing them to engage in business transactions through the business-in-a-box (BiB), savings group, mobile money, and organic gardening. The BiB is a kit containing all the ingredients and materials needed to start and run a specific microenterprise. At the start of the training, the participants were made to 2 From CEAIU proposal submitted to USAID. • Batch A • Batch B • Batch C 10 rank their top five choices among twenty product types of the BiBs. Among these, the BiBs on puto cheese (rice cake with cheese), banana chips, detergent powder, fabric conditioner, and dishwashing liquid ranked highest. These BiBs were eventually used as instructional materials in all the three batches of the CEAIU training. Participants had the option to take on a BiB of their choice through a microloan to be repaid by installment within the duration of the training, preferably from the sales of the BiB enterprise. Participants in a community were also encouraged to form a savings group (SG). Each SG member made a weekly compulsory contribution in an amount agreed upon by the group. Voluntary contributions were also welcome. The mobile money strategy, in partnership with SMART Foundation and SMART Corporation, involved the distribution of free regular SIM cards to the participants and a mobile phone with retailer SIM cards to SG leaders. The SIM cards were registered in mobile bank accounts that can enable mobile transactions such as deposits and withdrawals. Organic gardening involved the creation of a community garden through the Garden-in-a-Box (GiB). The GiB is a kit containing vermicast, vegetable seeds, and other materials for setting up a small garden. Participants were allowed to take one GiB kit for free but additional purchases required payment. The health training module taught participants to practice proper hygiene and sanitation, safe motherhood, healthy child-rearing, first aid, and other health-related lessons. In the house-to￾house visits, treatment packages were distributed for conditions such as diarrhea and dehydration. Pregnancy and malnutrition assessment and tuberculosis screening were implemented along with referrals to the rural health unit (RHU) whenever needed. Manna packs (or food packs) containing rice and dehydrated vegetables were distributed weekly to participants. These food packs were intended to provide vitamins and minerals to household members and to compensate for the opportunity cost of attending the program. All these activities were designed to help the ultrapoor participants become entrepreneurs and eventually improve their household well-being. Figure 2 shows the Theory of Change and Implementation Logic Model of CEAIU. It shows the logical connection between activities and intermediate results leading to the desired outcomes. 11 Figure 2. Theory of Change and Implementation Logic Model (Source: CEAIU M, E and L Plan, 2014) As stated in CEAIU’s award document, the following were the intermediate results, outputs, and indicators of the activity: Intermediate Result 1: Transform participants from 90 Iloilo communities trained in basic entrepreneurial skills. Output Objective 1: -Healthy environment and social capital for entrepreneurship created. Indicators: -Number of households given nutrition and other support through Transform -Number of households reporting better health and well-being -Proportion of participants reporting improved attitudes towards local authorities and entrepreneurship Output Objective 2: -2,100 Transform participants trained in basic entrepreneurial skills. Indicators: -Person-hours of training completed by employees of microenterprises supported by USG assistance -Number of Iloilo communities with Transform participants trained in basic entrepreneurial skills. 12 Intermediate Result 2: Availability of capital for microbusinesses expanded. Output Objectives: -At least 1,575 participants open micro-loans -At least 150 participants use a mobile bank account -At least 69 communities start community savings groups Indicators: -Number of participants who open micro-loans -Number of participants who use a mobile bank account -Number of project communities creating community savings groups. Intermediate Result 3: New micro-businesses started. Output Objectives: -More than 1,182 new microbusinesses launched Indicators: -Number of microenterprises supported by USG enterprise assistance -Number of micro-businesses launched in target communities. As of September 2016, there were around 3,600 participants and 2,552 graduates of the Transform training in Batches A, B, and C. CEAIU’s key accomplishments are presented in Table 1. As seen in the table, the targeted outputs for all three batches were achieved and surpassed. Table 1. Cumulative Accomplishments of CEAIU Indicators Cumulative Target Cumulative Results Level of Performance Number of communities 90 90 100% Number of graduates 2,100 2,552 122% Participants who opened microloans 1,575 1,913 122% Microenterprises/ individuals supported by USG enterprise assistance started 1,182 1,632 138% Savings groups started 69 90 130% Participants using a mobile money account 150 814 543% Source: CEAIU Final Report 13 EVALUATION METHODS The evaluation used both quantitative and qualitative methods in order to gather insights on CEAIU’s performance, realization of outcomes, and support to CDI activities. The quantitative component used data from the CEAIU activity database and from interviews of sample participants and other respondents. Qualitative methods include key informant interviews (KIIs) and focus group discussions (FGDs). Review of relevant documents, reports, and other secondary literature were also done. Whenever possible, results by gender were analyzed and disaggregated in all the evaluation questions. In summary, the evaluation performed the following general methods:  Analysis of logical frameworks  Review of relevant documentation and activity database  Collection and analysis of interview data from activity administrators, implementers, and participants through key informant interviews and focus group discussions during the field visit in Iloilo Province  Identification of lessons learned and areas of sustainability. The specific methods used to answer each evaluation question (EQ) are explained below: On EQ 1: What major changes in household conditions and well-being are emerging and will occur most likely as a result of USAID assistance to the CEAIU activity? Answering EQ 1 entailed documenting changes in participants’ conditions on the following items: (a) household income, consumption, and savings; (b) health-seeking behavior; and (c) entrepreneurial skills and competence. The observed changes were further analyzed whether they were a direct result of the CEAIU activity. This evaluation used primary data collected by CEAIU from the participants. Prior to the activity, participants underwent a detailed pre-training interview to elicit information on household socio￾demographic profile, income and occupation, health and nutritional status, and values and spiritual life. After the training, the graduates were made to answer the same interview questions for comparison of pre- and post-training responses. Primary data collected from these surveys were encoded in the CEAIU database and summarized in monitoring reports. CEAIU maintained other databases for tracking participant performance on attendance, number of BiBs dispersed, amount of savings, and microloan repayment. All these datasets were analyzed in this evaluation. Descriptive statistics (e.g., frequency counts, percentages, cross-tabulations) were used to summarize results. Means testing were done to determine significant changes in the relevant indicators before and after the training. During the evaluation fieldwork, sample participant￾graduates were interviewed in order to see if those changes persisted beyond the 16-week training. At the time of the fieldwork, this meant gathering information a full 12 months after the graduation of Batch A participants and eight months after Batch B. Information from the 14 fieldwork was compared with those from the CEAIU database to document patterns of changes and to verify data capture of monitoring reports. Qualitative techniques were used to examine changes in the entrepreneurial skills of the participants. Secondary literature was used to identify key indicators on entrepreneurial competencies such as opportunity seeking, persistence, and commitment to work contract. Based on these indicators, this evaluation combed the CEAIU activity database for data which may have some connection with such competencies. For example, the change in the number of the participants’ enterprises may provide insight on their opportunity-seeking skill. The persistence of the participants could be reflected by data on the participants’ self-rating of their ability to carry￾on with their activities despite physical and emotional impediments. EQ 2: What key outputs and emerging outcomes of the CEAIU activity contribute to the Cities Development Initiative (CDI) project outcomes? In order to answer EQ 2, a clear link between CEAIU and CDI should be established. An analysis of the logframe or theory of change of CDI and CEAIU was done to determine in specific terms how the latter connects with the former in terms of activities and inputs leading to immediate outputs, intended outcomes, and goals. For this question, the evaluation analyzed reports and documents and interviewed CEAIU and CDI personnel. The detailed logframe matrix of the CDI was perused line by line to see where the CEAIU can have possible links, either directly or indirectly, with the CDI goals, sub-purposes, activities, and inputs. Also, the results framework of the Strengthening Urban Resiliency for Growth with Equity (SURGE), considered the flagship activity under the Cities Development Initiative (CDI), was subjected to the same procedure. Accomplishments of the CDI were compared with those of the CEAIU. Results were summarized in a matrix showing possible linkages, if any, and the extent of such connection. EQ 3: What are some major lessons learned to improve the effectiveness of CEAIU development strategies and the sustainability of its outcomes? The sustainability of results and outcomes was assessed from the KII and FGD responses of sample participants. They were asked to identify how the activity could have improved their well￾being even long after the culmination of the training. Specifically, the FGDs were designed to find out the most significant changes that the sample participants felt which they can attribute to the CEAIU activity. The FGDs also documented success stories and lessons learned. Suggestions for improvement were also solicited. KIIs of pastors and counselor-volunteers were done to gather insights on how to sustain the results based on their actual experiences in the training and on their knowledge of the participants’ well-being before and after the activity. A rapid process evaluation of activity implementation was also done. The evaluation gauged whether CEAIU implemented an effective M&E system with structures (organizational, documentation, reporting and feedback systems) that captured and analyzed indicators for use at 15 the different activity management levels. The evaluation team went over relevant reports and materials and conducted interviews with CEAIU implementers. The FGDs with participants also provided information on how the activity was conducted. EQ 4: What are some key CEAIU characteristics and enabling factors/conditions that enhance the scalability and replicability of the CEAIU development approach? To answer this evaluation question, relevant results that could be replicated or scaled up in similar future activities were compiled. Results were validated by triangulating information from the FGDs, KIIs, and the activity databases and reports. Success stories and lessons learned were highlighted in order to come up with recommendations on how to replicate such successes and to address areas for improvement. Field data gathering design: Iloilo fieldwork Much of the results of this evaluation came from the fieldwork of sample communities in the province of Iloilo. Ideally, data gathering should have been done from a larger sample size of participants. However, given limited resources, this evaluation is not designed for such a full￾blown primary survey. Instead, this evaluation employed FGDs and KIIs in the sample communities. Prior to their participation in the FGDs sample beneficiaries were interviewed using questionnaires to elicit socio-demographic and other activity-related information that can be compared with those in the database. A sample was taken from the 60 CEAIU communities in Batches A and B (30 communities each) which were situated in five geographical types: (1) coastal, (2) rural mountain, (3) rural plain, (4) rural slum, and (5) urban slum (Table 2). For the two batches, there were a total of 6 communities covered by CEAIU in coastal areas; 6 in rural mountains; 33 in rural plain; 1 in rural slum; and 14 in urban slums. In order to have full representation of communities in all these areas, the sample covered all of the geographical types except for rural slum which only had one community reporting. Table 2. Number of communities covered by geographical type, Batches A and B, CEAIU Activity Geographical Type Number of CEAIU Communities in Batches A and B 1) coastal 6 2) rural mountain 6 3) rural plain 33 4) rural slum 1 5) urban slum 14 TOTAL 60 16 In each of these geographical types (excluding rural slum), the evaluation filtered the towns (or cities) that hosted communities for both Batches A and B (see Figure 3). This would help compare the results between community batches located in the same town and geographical type. Among the five towns in the coastal communities, only Iloilo City hosted two batches of communities and as such was chosen to be included in the sample. Whenever possible, the evaluation was predisposed to include Iloilo City, which is a CDI city, in the sample in order to better answer the evaluation question on the CEAIU-CDI linkage. Figure 3. Selection of sample towns and communities in this evaluation. (Note: In parentheses is the number of CEAIU communities in a town or geographical area.) 17 In the rural mountains, since no town hosted communities in both batches, the towns of Dueñas (Batch A) and Leon (Batch B) were randomly chosen to be in the sample. In the rural plain, there were 5 towns (out of 17) that hosted two batches of communities. Of these, Passi City and Lambunao were randomly selected. In the urban slum, Iloilo City was the site of 10 of the 14 CEAIU communities (and the only one to host both Batches) and thus was chosen to represent this geographical type. After choosing a town, a random sample of communities was chosen whenever there were multiple communities reporting. The final sample consisted of a total of ten communities (five each batch) in five towns or cities in the four geographical types. The list of all the communities in the sample is found in Annex IV. Random samples of respondents were taken from the ten communities. Figure 4 shows the total number of respondents by type of data gathering activity done in the fieldwork. In all of the ten communities, the evaluation team conducted questionnaire-guided interviews and FGDs on a random sample of CEAIU participant-graduates. KIIs were done on the following: the local pastor of the community; a local counselor involved in the activity; a local leader (barangay captain or councilor); and, whenever available, a barangay health worker. Random samples of dropouts and non-participants were also interviewed. Local ICM staff were interviewed about project design and actual implementation. A SURGE-CDI personnel was interviewed on the potential CDI-CEAIU link. Figure 4. Total number of respondents in the Iloilo field work by type of data gathering activity. 18 LIMITATIONS Given its limited scope and depth, this is a quick evaluation of activity performance and emerging results. Data limitation hindered the establishment of counterfactual information which could have provided more definitive conclusions on the effects of the activity on the participants. Furthermore, the duration of the training may be too short a time (only 16 weeks) to bring about significant changes in well-being. Nonetheless, the quantitative and qualitative methods helped provide patterns of changes and identify important issues relevant to the evaluation questions. In examining changes in the entrepreneurial skills of participants, the literature suggests that entrepreneurial competencies, skills and intentions can be assessed using information from very detailed questionnaires on such areas. Since such was not collected from the CEAIU participants, this evaluation gathered qualitative information on entrepreneurial competencies from the FGDs and KIIs. As stated earlier, a larger sample size of participants and communities would have been ideal for a fuller quantitative analysis. The use of FGDs and KIIs was the recourse in order to gather insights and indications adequate to answer the relevant evaluation questions. Whenever applicable, the sets of information gathered from the database, FGDs, and KIIs were compared for validation and cross-checking. In fact, the key findings of this evaluation were an outcome of the triangulation of the results from the said sources and methods. 19 FINDINGS, CONCLUSIONS & RECOMMENDATIONS EQ 1: What major changes in household conditions and well‐being are emerging and will occur most likely as a result of USAID assistance to the CEAIU activity? FINDINGS 1. Emerging changes in household conditions of participants 1.a. Income A key indicator of CEAIU outcomes is the change in household income due to participation in the activity. The training should have catalyzed the entrepreneurial skills of the participants for them to establish micro-enterprises that can provide an additional source of income for the household. Figure 5 shows the mean household income of Batch A and B participants as reported in the CEAIU final report. It shows a rise in the mean household incomes in these batches. Figure 5. Mean household income of participants in Batches A and B, pre- and post-training survey. (Source: CEAIU Final Report) 20 This evaluation went over the database and performed a test of means on the incomes of participants with matching pre- and post-survey data. Results of the t-tests showed that indeed the changes in mean household income in the two batches were statistically significant at the 1% level (Table 3). Table 3. Result of t-tests on mean household income in Batches A and B (mean-comparison test for paired data). Variable Batch A (n=387) Batch B (n=345) Mean household income in PHP, pre-training survey 3,681.41 3,408.75 Mean household income in PHP, post-training survey 5,348.10 4,965.59 Difference (Post – Pre) 1,666.69*** 1,566.84*** t-stat 7.8904 6.1738 Source of basic data: CEAIU database ***Significant at the 1% level. These results, however, provide an aggregate, batch-wide picture across all households which may not be enough to explain specific indications of positive changes as a direct result of CEAIU. For this, the evaluation first went through the database to disaggregate results per community. Figures 6 and 7 show positive changes in that mean household income in practically all communities when comparing the pre- and post-training levels. 21 Figure 6. Percentage change in mean household income by community, Batch A. (Source of basic data: CEAIU Database) Figure 7. Percentage change in mean household income by community, Batch B. (Source of basic data: CEAIU Database) 22 In order to validate if such changes persisted long after the training, Figure 8 compares the mean household income of the sample respondents in the pre- and post-surveys with information during the evaluation fieldwork. In the figure, only household income from work and business was accounted for, excluding whatever transfers the household received such as financial assistance from government, relatives, and other sources. This is done to remove the effect of transfers and focus on work and business income, the latter being the main focus of the CEAIU activity. Figure 8 shows the continued increase in mean household income of respondents in all the ten sample communities up of the time of the evaluation fieldwork. A test of means done per sample community showed that the differences between the pre-training and fieldwork mean household income levels were statistically significant in 5 of these 10 communities (Table 4). Figure 8. Mean household income of respondents by community, pre- and post-training surveys and evaluation fieldwork. 23 The next step was to explain the nature of these changes and gather indications if these were a result of the CEAIU intervention. Without controlling for other confounding factors that could have contributed to the changes in income--such as favorable socio-economic conditions, interventions from other entities, and participants’ skills development over time--an evaluation cannot distinguish between change that occurred as a result of the CEAIU and change that would have occurred anyway because of these factors. One quick indication that the positive income changes can be attributed to CEAIU is that the earnings from the continued operation of the BiB enterprises up to the time of fieldwork contributed a considerable portion to household income. To show this, the evaluation analyzed in detail the work and business income sources of every respondent as reported in the pre-and post-training surveys and compared them with their responses during the evaluation fieldwork. Note again that the fieldwork was done one year after Batch A respondents have graduated from the training and around eight months after Batch B. The succeeding clusters of graphs show the sources and amount of household income per respondent per community at the time of the pre- and post-training surveys and at the evaluation fieldwork. They show that changes in income in those time periods can be explained by any one of the following reasons:  changes in work type of a household member  additional working household members reporting  possible changes in market conditions affecting earnings for the same work type Variable Sample community 1 2 3 4 5 6 7 8 9 10 Mean household income in Php, pre-training survey 3,292 8,640 3,220 3,483 1,780 1,900 2,300 1,750 4,900 2,980 Mean household income in Php, field work 14,888 21,491 9,900 6,200 11,500 3,751 5,221 14,975 8,165 7,197 Difference in mean household income in PhP (Fieldwork – Pre-training) 11,597 ** 12,851 ns 6,680 * 2,717 ns 9,720 ** 1,851 ns 2,921 ns 13,225 * 3,265 ns 4,217 ** t Statistic 2.927 1.045 2.578 1.574 3.180 1.345 2.062 2.504 1.870 4.442 t Critical (two-tailed) 2.571 2.776 2.776 2.571 2.776 2.571 3.182 3.182 2.776 2.776 Number of observations 6 5 5 6 5 6 4 4 5 5 **, * = Significant at the 5% and 10% levels, respectively. ns = not significant Table 4. Result of t-tests on mean household income of sample respondents per community during the pre-training survey and the evaluation fieldwork. 24 For example, Figures 9.a – 9.g show the income sources of all respondents in sample community 1 (Batch A) during the pre-training, post-training, and evaluation fieldwork interviews. Out of the seven respondents, only one reported earning BiB-related income at the time of the evaluation fieldwork. This came from being a vendor of various products including puto cheese (Respondent 1, Figure 9.a). Also for the same respondent, the increase in income from pre- to post-training is due to changes in the type of employment and accompanying earnings. At the pre-training survey, income sources came from being a household help and doing laundry, but the higher income post-training chiefly came from other occupation types3 and some amount of production. For respondent 3 (Figure 9.c), the increase in income during the fieldwork was accounted for by higher earnings from such occupations as a pedicab driver, delivery boy, and sari-sari store (mini￾grocery/variety store) owner. Note that during the pre- and post-surveys, there were only two occupations reported: tricycle driver and others. Whether the establishment of the sari-sari store was prompted by the learnings from the CEAIU cannot be established by the evaluation, although the training sought to introduce the participants to the basics in setting up and managing an enterprise. The respondent, however, reported no BiB-related income which is a more definitive indication of direct results. For respondent 4 (Figure 9.d), the increase in income during the fieldwork was accounted for by higher earnings from laundry and pedicab driving, with no indication of BiB-related income. For respondents 6 and 7 (Figures 9.f and 9.g), the higher income during the fieldwork was due to higher earnings from new occupations. The same pattern can be seen in other communities. For example, for respondent 5 in Community 2, (Figure 10.e), the contribution of BiB-related income (from fabric conditioner and detergent) was evident during the fieldwork, albeit in a small amount. There were BiB-related income reported by respondents 3 and 4, but they were in smaller amounts compared to other sources. Nonetheless, they indicate that those learnings from CEAIU were still being applied long after the culmination of the training. For the rest of the respondents in community 2, the changes in income could mostly be explained by changes in occupation type and earnings. In the rest of the communities, only a few reported earning income from BiB-related enterprises (0/5 in Community 3; 1/7 in Community 4; 1/6 in Community 5; 0/7 in Community 6; 2/6 in Community 7; 2/6 in Community 8; 1/7 in Community 9; and 1/7 in Community 10). The earnings from these BiBs were in smaller amounts than from those from other sources. 3 In the CEAIU questionnaire, the type of work listed were: (1) Fisherman; (2) Tricycle/trisikad driver; (3) Farmer; (4) Laundry; (5) Buy & sell; (6) Construction; (7) Household helper; (8) Others; and (9) No work. A question was also asked on income from production of any product such as snacks, vegetables, detergents and others. 25 These results indicate that the increases in household income can be attributed mainly to changes in earnings from non-business related occupations and from the addition of new working household members. These changes, however, cannot be fully attributed to CEAIU since they can be influenced by other factors beyond the control of the activity. Furthermore, there were only a few respondents who reported BiB-related income and the amounts were not that high compared to other sources. Nonetheless, these respondents who sustained these BiB￾related enterprises showed better capacities in applying the learnings from the CEAIU. 26 Figures 9.a – 9.g: Mean household income of respondents by source, Community 1, pre- and post-training surveys and evaluation fieldwork. (9.a) (9.b) (9.c) (9.d) (9.g) (9.e) (9.f) 27 Figures 10.a – 10.f. Mean household income of respondents by source, Community 2, pre- and post-training surveys and evaluation fieldwork. (10.c) (10.d) (7.c) (10.a) (10.b) (10.e) (10.f) 28 Figures 11.a – 11.e. Mean household income of respondents by source, Community 3, pre￾and post-training surveys and evaluation fieldwork. (11.a) (11.b) (11.c) (11.d) (11.e) 29 Figures 12.a – 12.g. Mean household income of respondents by source, Community 4, pre￾and post-training surveys and evaluation fieldwork. (12.a) (12.b) (12.c) (12.d) (12.e) (12.f) (12.g) 30 Figures 13.a – 13.f. Mean household income of respondents by source, Community 5, pre- and post-training surveys and evaluation fieldwork. (13.a) (13.b) (13.c) (13.d) (13.e) (13.f) 31 Figures 14.a – 14.g. Mean household income of respondents by source, Community 6, pre￾and post-training surveys and evaluation fieldwork. (14.a) (14.b) (14.c) (14.d) (14.e) (14.f) (14.g) 32 Figures 15.a – 15.f. Mean household income of respondents by source, Community 7, pre- and post-training surveys and evaluation fieldwork. (15.a) (15.b) (15.c) (15.d) (15.e) (15.f) 33 Figures 16.a – 16.f. Mean household income of respondents by source, Community 8, pre- and post-training surveys and evaluation fieldwork. (16.a) (16.b) (16.c) (16.d) (16.e) (16.f) 34 Figures 17.a – 17.g. Mean household income of respondents by source, Community 9, pre￾and post-training surveys and evaluation fieldwork. (17.a) (17.b) (17.c) (17.d) (17.e) (17.f) (17.g) 35 Figures 18.a – 18.g. Mean household income of respondents by source, Community 10, pre￾and post-training surveys and evaluation fieldwork. (18.a) (18.b) (18.c) (18.d) (18.e) (18.f) (18.g) 36 1.b. Savings Another key strategy of the CEAIU training was to enjoin participants to save and become members of a savings group in their community. CEAIU data from their pre- and post-surveys showed an increase in mean household savings among participants who reported some amount of savings in Batches A and B (Figure 19). On a per community basis, Figures 20 and 21 showed that indeed there was an increase in mean household savings in most of the communities in Batches A and B (pre- and post-training), with only 11 of the 60 communities reporting a decline. In order to see if this result is sustained up to the time of the fieldwork, the evaluation collected recent savings information of the sample respondents. Figure 22 shows that respondents in all except one of the sample communities reported an increase in their mean household savings compared with either their pre- or post￾training levels. Figure 23 shows that among the sample respondents, the percentage of households with any positive amount of savings has increased from just 11% before the training to 59% during the evaluation fieldwork. In the context of this evaluation, these positive changes can be directly attributable to the activity. These respondents had little or no savings at all pre-CEAIU and it was part of the training for participants to join a savings group (SG) and regularly contribute some amount. However, Figure 19. Mean household savings of participants in Batches A and B, pre- and post￾training survey. (Source of basic data: CEAIU Final Report) 37 among the sample communities, majority of these savings group have been rendered inactive at the time of the evaluation fieldwork (i.e., they have stopped collecting contributions), but participants still keep their savings in these SGs. Figure 20. Mean household savings of participants per community, Batch A, pre- and post-training survey. (Source of basic data: CEAIU Project database) Figure 21. Mean household savings of participants per community, Batch B, pre- and post-training survey. (Source of basic data: CEAIU Project database) 38 Figure 23. Percentage of sample respondents with some positive amount of savings during pre- and post-training survey and the evaluation fieldwork. Figure 22. Mean household savings of sample respondents per community during pre- and post-training survey and the evaluation fieldwork. 39 1.c. Debt The amount of household debt was also monitored by the CEAIU activity. Figure 24 shows that based on CEAIU data, Batch A participants fared better in bringing down mean household debt after the training compared with Batch B participants who saw a rise in debt. It is, however, pointed out that incurring debts to finance worthy investments cannot be discouraged. Looking at performance by community, Figure 25 shows that except in only three communities, all Batch A communities saw a decline in mean household debt. For Batch B participants, only nine of the 30 communities showed a decline in average household debt (Figure 26). Figure 24. Mean household debt of participants in Batches A and B, pre- and post-training survey. (Source of basic data: CEAIU Final Report) 40 Figure 25. Mean household debt of participants per community, Batch A, pre- and post￾training survey. (Source of basic data: CEAIU Project Database) Figure 26. Mean household debt of participants per community, Batch B, pre- and post￾training survey. (Source of basic data: CEAIU Project Database) 41 These same mixed results were observed during the evaluation fieldwork. Figure 27 shows that sample respondents in seven of the ten communities reported a decline in mean household debt compared to either their pre- or post-training responses. Figure 28 shows that the proportion of sample respondents with any positive amount of debt remained practically the same at around 70% during pre-training and the evaluation field work, indicating their inability to clear off their debt. Figure 27. Mean household debt of sample respondents per community during the pre￾and post-training surveys and the evaluation field work. 42 As mentioned earlier, household debt, while considered a burden, may also be welcome if it is used to finance a worthy investment that promises high returns in the future. The sample respondents were asked about the main reasons for incurring household debt. The top reason was to finance basic household expenses such as on food and schooling (Figure 29). This type of debt is not desirable since ultrapoor households finding difficulty to sustain even the most basic needs are not expected to manage these obligations well. Some, however, reported incurring debt to finance small businesses, mostly to put up a sari-sari store. Results showed that the possible effect of CEAIU on household debt is mixed. While some respondents saw a decline, others have not minimized or fully cleared their debt. In fact, a big majority (70%) of samples during field evaluation still reported some positive amount of debt. Furthermore, most of this debt was for basic needs which income-constrained households may have difficulty paying off. Figure 28. Percentage of sample respondents with some positive amount of debt during the pre- and post-training surveys and the evaluation fieldwork. 43 1.d. Consumption In the pre- and post-training surveys the CEAIU activity kept track of the amount of the participants’ household consumption of essential commodities, specifically: (1) rice and corn; (2) fruits and vegetables; (3) meat, poultry and fish; (4) mobile phone load; and (5) transportation. These are considered income-inelastic commodities, that is, demand for these goods are not greatly affected by changes in income. However, ultrapoor households who have difficulty buying these products in the desired quantities may be expected to have greater consumption of these items when they would realize improvement in their earnings. Figures 30 and 31 show the changes in consumption spending on these items before and after the training. In the case of rice, the country’s staple and typically considered as a highly income inelastic good, no significant changes in consumption can be observed. For the other items there is no clear trend in the direction of changes, with slight increases or decreases observed by item and batch. Figure 29. Top reasons for debt of sample respondents, evaluation field work. 44 Figure 30. Mean household consumption of various items, Batch A, pre- and post￾training survey. (Source of basic data: CEAIU Final Report) Figure 31. Mean household consumption of various items, Batch B, pre- and post-training survey. (Source of basic data: CEAIU Final Report) 45 Figure 32 shows the consumption expenditures of all the sample respondents before and after the training and during the evaluation fieldwork. There was a decline in the consumption of rice and corn but a rise in all the rest of the items. Consumption of meat and fish and cellphone load went up considerably. While this upward trend showed positive signs, there can be no significant conclusions pertaining to CEAIU results. The increase in consumption could have been a result of the rise in household incomes, but the improvement in earnings cannot be attributed to CEAIU as earlier stated. 1.e. Health Health is an important consideration in the CEAIU activity. The health module taught participants first-aid, home remedies, and hygienic practices. The activity also monitored the participants and their co-household members for signs of sicknesses and diseases and provided intervention such as provision of medicines or referral to health professionals. It is difficult, however, to assess changes in health well-being given the many variables related to health and nutrition which were not expertly captured by the evaluation. Nonetheless, indications of changes in health-seeking behavior were gathered from whatever the database and the fieldwork can inform. Figure 32. Mean household consumption of various items, all sample respondents, pre- and post￾training surveys and evaluation fieldwork. 46 Figure 33 shows the self-perception on the participants’ state of health pre- and post-training. Based on their own assessment, more participants reported ‘Very Good’ and ‘Excellent” state of health, taken together, after the training. More participants in the ten sample communities reported washing their hands three times a day during the post-training survey compared to before (Figure 34). This practice is important in the ultrapoor communities where poor sanitation and lack of running water supply are a problem. Whether this led to better health well-being of households cannot be assured, but this is an important hygienic practice universally accepted to help prevent the acquisition or spread of diseases and illnesses. Figure 33. Self-perception on overall state of health among participants, pre- and post-training surveys. (Source of basic data: CEAIU Final Report) 47 Figure 35 shows that at the time of the fieldwork, 26% of respondents reported having at least one family member who they perceived was malnourished. Again, this was self-reported and no expert validation was done in the evaluation. The figure also shows that all respondents still practiced proper hygiene methods taught to them in the CEAIU training. Also, 73% reported having visited a rural health unit (RHU) or hospital at least once in the last six months for check￾ups or treatment of sicknesses. In the context of this evaluation, these are indications that participants exhibited sustained awareness of health-related practices learned from the activity. Figure 34. Frequency of handwashing in the ten sample Batch A and B. (Source of basic data: CEAIU Project Database) 48 1.f. Entrepreneurial competencies An important thrust of the CEAIU was to introduce the participants to the basics of entrepreneurship. The livelihood component of the CEAIU training not only taught basic businesses concepts but also made the participants apply their learnings through the use of BiB kits. This was intended for participants to have a feel of actually producing and selling a product. Entrepreneurial competencies, however, go beyond production and making a sale. This evaluation used several indicators of entrepreneurial competencies widely-accepted in the literature. Mansfield et al. (1987)4 identified Personal Entrepreneurial Competencies (PECs) that distinguished successful entrepreneurs especially in developing countries. The so-called Empretec model of the United Nations Conference in Trade and Development (UNCTAD) further 4 Mansfield, R.S., D.C. McClelland, L.M. Spencer, and J. Santiago (1987). The Identification and Assessment of Competencies and Other Personal Characteristics of Entrepreneurs in Developing Countries. Final Report Project No. DAN-5314-C-00-3074-00. Figure 35. Percentage of sample respondents who answered: having malnourished members; still practicing proper hygiene; and visited a health facility in the last six months, as reported during the evaluation fieldwork. 49 summarized the PECs into the following ten competencies which are categorized into three clusters: 1. Achievement Cluster  Opportunity seeking and initiative  Risk taking  Demand for efficiency and quality  Persistence  Commitment to work contract 2. Planning Cluster  Information seeking  Goal setting  Systematic planning and monitoring 3. Power Cluster  Persuasion and networking  Independence and self-confidence This evaluation gathered manifestation of these PECs among the participants. They were based on responses in some indirect questions in the pre- and post-surveys and in more direct ones during the evaluation fieldwork. Specifically, insights from the FGDs of participants and KII of pastors and counselors were useful in the quick assessment of these competencies. The evaluation focused on the following PECs which the CEAIU could have helped develop among the participants:  Networking  Information seeking  Opportunity seeking  Risk taking  Systematic planning and monitoring Results of the fieldwork showed that such competencies were apparent only in communities whose participants continued to apply their learnings from CEAIU. These were those who sustained the operations of the savings group and BiB related enterprises. While the training was designed to build individual capacities, those who were able to continue with their BiB enterprises after the training did so in groups. Apparently, the group enterprises were able to take advantage of economies of scale and division of tasks. 50 Table 4 categorizes the sample communities based on how they sustained their learnings from key CEAIU strategies. It shows that only three of the ten sample communities were able to continue the operations of the savings groups and the group-managed BiB businesses. The enterprises in these “Type 1” communities were usually on the production of detergent powder, fabric conditioner, and dishwashing liquid. Under the guidance and leadership of the pastor, the group would centrally manufacture these products which were then sold by the individual members. The seven communities under “Type 2.a” and “Type 2.b” were those that did not show success in sustaining their CEAIU-related activities. The savings groups in these communities were either inactive or completely dissolved. Furthermore, the participants were no longer engaged in group BiB-related enterprises unlike those in the Type 1 communities. The evaluation of entrepreneurial competencies in all these community types is discussed in the succeeding sections. Table 4. Types of sample communities based on how they sustained their learnings from key CEAIU strategies (as of the time of the evaluation fieldwork). Community type and description Communities Key characteristics Type 1: Communities that still have active savings groups and entrepreneurial BiB enterprises Communities 2, 4, and 10 The remaining active members of the savings groups are still engaged in detergent, fabric conditioner and dishwashing liquid production. All of them are centrally manufacturing these products under the supervision of the pastors. The SG members are in charge of marketing and selling the products. Type 2.a: There are no more BiB enterprises sustained and savings groups are inactive but not yet dissolved Communities 3, 6, 7, and 9 Though most of the members are not contributing anymore to the savings group, the funds collected from all the members are still intact. However, the members are no longer engaged in any BiB-related enterprises. Type 2.b: There are no more BiB enterprises and savings groups are dissolved Communities 1, 5, and 8 These savings groups decided to return the funds to the members and discontinue the group operations. The members are no longer engaged in any BiB￾related enterprises. 1.f.1. Networking Networking is defined as the ability to develop and maintain business and social contacts. Pre￾and post-training CEAIU data showed that there was an increase in the number of participants who believed that after the training, there were more people who could lend them PHP40 (Figures 36.a and 36.b) or PHP1,000 (Figures 37.a and 37.b), and provide other forms of assistance when needed (Figures 38.a and 38.b). 51 Figures 36.a and 36.b. Perception of respondents on whether there are people who are still willing to lend them PHP 40 during time of need, pre- and post-training surveys. (36.a) (36.b) Figures 37.a and 37.b. Perception of respondents on whether there are people who are still willing to lend them PHP 1,000 during time of need, pre- and post-training surveys. (37.a) (37.b) Figures 38.a and 38.b. Perception of respondents on whether there are people who are willing to help them when needed, pre- and post-training surveys. (38.a) (38.b) 52 These suggest that through the CEAIU, some participants found an opportunity to build a network of people who may help them in their time of need. This network is important in setting up and expanding enterprises. This result is supported by the “most significant changes” reported by the participants during the FGDs. According to the sample respondents, they got to know more people (e.g. co-participants, ICM staff, pastors, counselors, suppliers) who could assist them if they wish to start or continue an entrepreneurial undertaking. In fact, the respondents discussed having created a strong bond with co-participants during the 16-week training. Some SG leaders also widened their network through the monthly “Prevail” 5 meetings conducted by ICM where they were able to learn from the experiences and strategies of SGs from other communities. Networking can also be demonstrated by the ability of participants to establish new contacts specific to their business operations. In the three Type 1 communities in Table 4, the SGs who managed their group business established networks with suppliers of raw materials from other areas (e.g., in Cebu and Iloilo City). The pastors in these communities communicated with each other in the sourcing of said ingredients. In the case of Community 10, the pastor and his wife led the production of fabric conditioner and dishwashing liquid while the active members were tasked to sell the products. The pastor established a network with other pastors to consolidate their purchases of raw materials in order to take advantage of bulk discounts. This helped reduce costs of buying the inputs. 1.f.2. Information seeking Information seeking is the ability to source important information from clients, suppliers or competitors in order to improve the operation of the enterprise. In the relatively more successful communities, the participants were able to acquire more sources of information particularly on the supply of raw materials. They also consulted the pastor, wife of pastor, co-participants, counselors, and SG leader for technical and business advice. 1.f.3. Opportunity seeking Opportunity seeking is the ability to seize opportunities for financing and starting a new business. As shown earlier, there were respondents who continued to earn income from the production of detergent powder, fabric conditioner and dishwashing liquid. These respondents saw opportunities to improve the performance of the business. In the three communities that continued the said BiBs, credit was extended to customers within the neighborhood so that the 5 According to the CEAIU Final Report, “Prevail is the continuation of Transform, and was not funded by CEAIU. To promote continuation of saving as a practice and foster long-term sustainability of Savings Groups formed during Transform, ICM’s Prevail program delivers monthly capacity building and networking opportunities for the leaders of the savings groups. The Prevail curriculum is based on a peer-mentoring strategy that strengthens the savings groups into sustainable structures.“ 53 products would sell faster. As mentioned earlier, in order to take advantage of discounts from bulk purchases and reduce transportation cost, SGs from these communities coordinated with each other when ordering raw material supplies from another province (Cebu). In one Type 1 community (Community 2), instead of producing all the detergent they sold, the participants repacked commercially available detergent for resale so that time and effort will be saved in production. At the start, the group would produce all the detergent powder, fabric conditioner, and dishwashing liquid they sold. Each active member would market the products and profits were plowed back to the savings group. Over time, the group saw opportunities to improve their business model. Instead of manufacturing all the detergent powder, they learned that they could buy similar ready-to-use products. In order to boost supply, they decided to source and repack the detergent instead of manufacturing them. This also reduced the amount of time they needed to devote on the enterprise. Furthermore, the group extended credit to its buyers which led to more orders and sales. Interestingly, there was no reported case of delinquency in payment. In order to keep up with the demand of the product, the group is contemplating on expanding production by purchasing higher-capacity mixing equipment. 1.f.4. Risk taking An important ability of an entrepreneur is risk taking. This involves the ability to take action to control outcomes of calculated risks. When some respondents had to decide to continue on with their enterprises after CEAIU, they needed to take risks on which among the five BiB types they would continue on their own. These respondents exhibited sound risk-taking behavior when they focused on detergent powder, dishwashing liquid, and fabric conditioner because of the following reasons: (1) these products required less capital expenditure; (2) they were not perishable (unlike the food-related BiBs); and (3) they were regularly consumed by potential customers. All the three Type 1 communities continued to produce these BiBs, with profits plowed back to the savings groups. The savings groups showed prudence in handling the funds. While there were opportunities to scale up production or diversify their operations, these groups remained cautious and have not embarked in more risky undertakings. They, however, reported that there were plans to expand their enterprises although they have not made any concrete steps at the time of the evaluation. 1.f.5. Systematic planning and monitoring Systematic planning and monitoring means acting in a logical way for the business. Those that exhibit this competency would keep financial records and set clear goals and objectives. One manifestation of systematic planning was the effort to accumulate savings. Based on the results discussed earlier, most communities still had positive amounts of savings where there was little or none at all before the CEAIU. While being in a savings group is mandated by the training, the ability of ultrapoor participants to join a savings group (SG) and regularly set aside some amount of money exhibited their planning and monitoring competency. Furthermore, the fieldwork 54 revealed that the respondents who continued on with their businesses had clear plans. The remaining active participants in the three Type 1 communities revealed plans to expand their production volume by sourcing more raw materials and upgrading their equipment. For example, in Type-1 Community 4, a remote community primarily dependent on small-scale agriculture (e.g., rainfed rice, vegetables, and backyard livestock production), the savings group was still active and operating the detergent business. Every week, around 42 kilograms of detergent were being produced under the leadership of the pastor. They would source raw materials from Iloilo City. The members sell the detergent powder at a price of PHP 120 per kilogram with a credit term of 7 days. All sales and profits were returned to the group for capital build-up. At the time of interview, the SG had a total savings of around PHP 12,000. They plan to engage in fabric conditioner and dishwashing liquid production in the future. While the above showed that these competencies were exhibited only by the relatively more successful respondents, there were other traits that the relatively unsuccessful ones lacked. Based on the fieldwork, most respondents lacked persistence and commitment to work contract. Persistence refers to the characteristic of not quickly abandoning activities. It also means taking actions, even repeatedly, or shifting to alternative strategies to achieve an objective. Based on the CEAIU database, there was no significant improvement in the perception of participants on whether personal and emotional problems prevent them from doing their work (Figures 39.a and 39.b). In both batches, after the training, there were more participants who believed that problems somewhat prevented them from doing their task. In Batch B, there was also a decrease in the number of individuals who believed that problems had very little influence on whether or not they could perform their work. 55 A number of participants stopped because of some obstacles during implementation. Based on the survey, some of the hurdles that many participants could not overcome were:  insufficient operating capital  lack of equipment for processing (e.g. steamer for puto cheese)  bad debt among customers  conflict of schedule with other activities Figures 39.a and 39.b. Perception of participants on whether personal or emotional problems prevent them from doing their work and other activities, pre- and post-training surveys. (39.a) (39.b) 56 Aside from those hurdles, the seven Type-2a and 2.b communities indicated their inability to internalize entrepreneurial competencies. For example, in Type 2.a Community 6, the group decided to stop the production of detergent powder, fabric conditioner, and dishwashing liquid due to unavailability of ingredients and lack of interest among participants. In Type 2.b Community 5, because of other livelihood opportunities in the nearby public market, the initial set of participants showed little interest in the CEAIU training. According to the pastor, the participants were reluctant to contribute to the savings group. The pastor had to raise funds from other members of the church to fund the enterprises. Unfortunately, the participants did not exert effort to sell the products. Some became inactive and the SG eventually stopped its business operation. In Type 2.a Community 7, the savings group attempted to continue on with the production of detergent powder and fabric conditioner. However, the participants became disinterested and uncooperative. Also, some counselors claimed that the ingredients for banana chips (i.e. banana oil and glucose) and detergent production were not available in the locality. The group eventually abandoned their business. In Type 2.b Community 8, one the poorest communities in their barangay, the participants initially tried to produce detergent powder. The participants later lost interest and the operation of the savings group was stopped. The same case happened in Type 2.b Community 1 where detergent production was discontinued when many members became inactive. Another trait lacking was the commitment to work contract, simply defined as the ability to keep a promise even if it would entail sacrifices. Entrepreneurs exhibiting this trait keep customers satisfied and give more importance to long-term relationship than in making a quick sale. Based on the evaluation fieldwork, the relatively more successful respondents manifested their commitment to work contract on various ways, such as: (1) repaying all the microloans taken for the BiBs using the group savings; (2) changing the packaging size of products when there were comments from customers about the desired amount of detergent; (3) utilizing marketing techniques such as credit transactions and door-to-door selling; and (4) asking feedback from customers. The respondents in the less successful communities did not show commitment and did the following: (1) stopped attending meetings; (2) stopped giving compulsory contribution to the SG (some even withdrew their contributions prematurely); (3) left the program when they realized that the activity would not hand out operating capital; (4) did not address comments of customers that the product was expensive; and (5) refused to pay back the microloans for BiBs. In general, the CEAIU activity gave participants opportunities to develop their entrepreneurial competencies such as risk taking, opportunity seeking, networking, planning and information seeking skills. Some respondents exhibited these competencies despite the challenges of sustaining the BiB-related business after the training. However, many communities who failed to sustain their enterprises apparently lacked these competencies and other important traits such as persistence and commitment to work contract. 57 CONCLUSIONS UNDER EQ 1: There were significant positive changes in mean household income among the CEAIU participants. However, this evaluation could not establish evidence that those changes were definitely a result of the CEAIU activity. The rise in income was due primarily to participants’ switching to wage work types with higher earnings, additional working household members, and possible changes in market conditions which were all beyond the direct control of the activity. The activity was able to positively influence the savings behavior of sample participants. They reported positive amounts of savings where little or none pre-CEAIU. In the context of this evaluation, this can be directly attributable to the activity. The training enjoined participants to become part of a savings group where they regularly contributed some amount. However, among the sample communities, majority of these savings group have been rendered inactive at the time of the evaluation fieldwork. Either the members have stopped giving contributions or the savings group was dissolved. However, the participants have not withdrawn their savings in most of these SGs. There were indications that participants exhibited sustained awareness of health-related practices that they learned from the activity. These include mindfulness and use of good hygienic practices and seeking medical professional help when needed. Due to the lack of expert evaluation, no conclusion was made on the changes in health status of the participants The activity provided a good opportunity to build entrepreneurial skills among the more successful participants. These were manifested in their capacities for networking, risk-taking, opportunity seeking, and systematic planning. However, such competencies were apparent only in three of the ten sample communities whose respondents continued the operation of the savings group and the BiB enterprises. In the less successful communities, participants did not exhibit these competencies and other important traits such as persistence and commitment to work contract. 58 EQ 2: What key outputs and emerging outcomes of the CEAIU activity contribute to the Cities Development Initiative (CDI) project outcomes? FINDINGS 2. CEAIU’s contribution to the Cities Development Initiative (CDI) The Cities Development Initiative (CDI) is a USAID project started in 2012. It is under the Partnership for Growth (PFG) forged between the Philippines and the United States to “promote broad-based, inclusive and resilient growth” (USAID Fact Sheet on the CDI). The CDI seeks to “advance the development of second-tier cities as engines of growth that is inclusive, environmentally sustainable, and resilient” by providing technical assistance to the cities on areas in governance, education, health, environment, and climate change. The CDI is a geographically￾focused program and has so far been piloted in the cities of Batangas, Cagayan de Oro, and Iloilo. The CDI logical framework (logframe) matrix6 specifies the most important features of the program and the logical connection between inputs, planned activities, and expected results. The goal describes the high-level development objective of the CDI. This is anchored on broad-based and inclusive growth accelerated and sustained outside of Metro Manila, with the purpose of strengthening economic competitiveness in second-tier cities. The three sub-purposes are on (1) Local capacity to manage urbanization and environmental resilience improved; (2) Enabling environment for local enterprise development enhanced; and (3) Linkages between urban and surrounding rural areas improved. Outputs under each sub-purpose are horizontally matched by indicators and targets. Inputs come in the form of specific activities and interventions. These include technical assistance given to development planners, review of plans and programs, and management of information systems. The CEAIU activity, with its presence in the CDI City of Iloilo, aimed to support the CDI through the latter’s logframe entry on “Sub-purpose 2: Enabling environment for local development enhanced”. Specifically mentioned under this sub-purpose was support to “Output 2.3: Local workforce skills base is responsive to enterprise needs” and “Output 2.7: Access of small and medium enterprises access to financial markets increased.” The evaluation went over other vertical and horizontal entries in the CDI logframe to see how CEAIU could possible connect their results and accomplishments. Table 5 below picks out the CDI logframe entries with which CEAIU may have a link. The table shows that at the Sub-Purpose and Output levels, there appears a conceptual link between the CDI and CEAIU. Specifically, these 6 The CDI logframe matrix is not shown here in full. The reader may wish to correspond with USAID-CDI for access to the matrix. 59 pertain to developing capacities, improving health and sanitation services, support to local enterprises through access to financial markets. However, at the level of indicators, there appears no specific connection between the CEAIU and CDI. For example, under Sub-purpose 2, the CDI indicators are on the following: (1) “number of “industry-responsive” academic programs proposed for University Board approval”; and (2) “approved assistance for vocational education and internship programs (through the Phil-Am Fund)”. By design, CEAIU does not have access to such academic programs and educational funds. Table 5. Possible linkages between CEAIU and CDI based on the CDI Logframe. CDI Logframe Sub-Purpose / Output / Indicators CEAIU Activities’ potential link Sub-Purpose 1: Local capacity to manage urbanization and environmental resilience improved Output 1.2: Public services (including health, education and infrastructure) improved Indicator bullet 3: Improved provision of health and education services Health training about preventing illness and disease and promoting good health choices. House-to-house visits of participants who require one of the treatment packages (e.g. Diarrhea and Dehydration Treatment Package) or follow up (e.g. tuberculosis screening and referral or a pregnancy assessment) were done for resolution. Short-term measure only as the activity was ongoing. Output 1.3: Environmental resilience and disaster risk reduction strengthened Indicator bullet 3:  Number of people gaining access to an improved drinking water source Indicator bullet 4:  Number of people gaining access to an improved sanitation facility Health training about proper hygiene and sanitation specifically about safe drinking water and hand washing; but this is not directly related to direct interventions on providing better water source and sanitation facility. (SPECIFICALLY IDENTIFIED BY CEAIU) Sub-Purpose 2: Enabling environment for local enterprise development enhanced Output 2.3: Local workforce skills base is responsive to enterprise needs Indicators:  Number of “industry-responsive” academic programs proposed for University Board approval by (target month and year  Approved assistance for vocational education and internship programs through the Phil-Am Fund) While Transform’s main result is the provision of skills in small enterprises, how it links to the indicators mentioned here is not clear 60 CDI Logframe Sub-Purpose / Output / Indicators CEAIU Activities’ potential link Output 2.4: Enterprises use electronic payments systems for transactions with suppliers and customers Indicator bullet 2:  Number of electronic payment systems adopted for industries and commercial establishments Enrollment to mobile banking was initiated, but it has not been used in all the communities sampled (SPECIFICALLY IDENTIFIED BY CEAIU) Output 2.7: Access of small and medium enterprises access to financial markets increased Indicators:  Number of SMEs with credible business plans and cash flow estimates developed by (target month and year)  Number of SMEs accessing DCA (Development Credit Authority) guarantees Mobile banking has not been fully operational; also, links to the indicators mentioned here is not clear, since participants are not SMEs Sub-Purpose 3: Linkages between urban and surrounding rural areas improved Output 3.4: Supply chain linkages between urban and rural areas improved Indicators:  Supply chain linkage for (commodity/industry) strengthened  Supply chain linkage for (commodity/industry) expanded Some sample communities that continued their group-managed BiBs (usually fabric conditioner, detergent, dishwashing) maintain linkage with suppliers of raw materials in urban areas (Manila, Cebu, Iloilo), usually through the efforts of the pastor and his wife. But improving the supply￾chain linkages is not part of the CEAIU activity. INPUTS Input 1.3.2 Training of various stakeholders in adapting to the impacts of climate variability and change Garden-in-a-box was introduced, for backyard growing of vegetables; but no direct training on climate change adaptation Input 1.3.3 Identification, rehabilitation or installation of improved drinking water sources in targeted second￾tier cities Input 1.3.4 Rehabilitation or installation of improved sanitation facilities in targeted second-tier cities Health training discusses importance of safe drinking water; but no direct intervention in providing better access Input 1.4.2 Facilitation of private sector adoption of ePayments Linkage of ICM with private sector providers of mobile banking was initiated; Enrollment of participants to mobile banking was done, but it has not been used in all the communities sampled (just used for e-loading) 61 CEAIU’s mobile banking intervention would appear to directly support CDI “Output 2.7: Access of small and medium enterprises access to financial markets increased”. However, while CEAIU participants’ enrollment to mobile banking was initiated, participants did not use the facility for their intended purposes. The retailing SIM cards were just used for selling phone credits. Also, the CDI indicators under this output are on the (a) “number of SMEs with credible business plans and cash flow estimates developed”; and (b) “number of SMEs accessing DCA (Development Credit Authority) guarantees”. Evidently, CEAIU does not have a clear link with these CDI-specific indicators. CEAIU could have a connection with the CDI’s sub-purposes and outputs related to health and sanitation. Examples are on Output 1.2, Indicator bullet 3: “Improved provision of health and education services”; Output 1.3, Indicator bullet 3: “Number of people gaining access to an improved drinking water source”; and Output 1.3, Indicator bullet 4: “Number of people gaining access to an improved sanitation facility”. CEAIU provided short-term health training about preventing illnesses and diseases, but how it could have continued in Iloilo City’s direct provision of health services is not clear. CEAIU’s health module also taught the importance of proper hygiene and sanitation (specifically on safe drinking water and hand washing), but these were not direct interventions in ensuring better access to improved water source and sanitation facility. The CDI inputs intended to deliver the outcomes come in the form of training and technical assistance to city planners which the CEAIU does not cater to. CEAIU’s connection with the CDI’s flagship project, Strengthening Urban Resiliency for Growth with Equity (SURGE), also appeared uncertain at the lower vertical levels. Figure 40 shows the Results Framework of SURGE. Encircled are the possible intermediate results and outcomes that CEAIU may have a link with. As in the CDI logframe, SURGE’s activities are mostly technical assistance to LGUs with no direct grassroots component. 62 CONCLUSIONS UNDER EQ 2: There appears a connection between CEAIU and CDI at the higher vertical levels of the latter’s logical framework. CDI’s goal of promoting inclusive growth in second-tier cities would necessarily cover the ultrapoor which the CEAIU specifically targets in their interventions. The conceptual links with CDI sub-purposes and outputs pertain to developing capacities, improving health and sanitation services, and support to local enterprises through access to financial markets. However, the link was not apparent at the lower levels, especially with the indicators of inputs and activities needed to deliver the CDI outputs. The CDI is a top-level initiative which provides technical assistance mostly to city planners and implementors on various long-horizon development plans and programs. CEAIU, however, is a grassroots-level, short-term training program in direct contact with ultrapoor beneficiaries. Figure 40. Results Framework of the Strengthening Urban Resiliency for Growth with Equity (SURGE) Project. 63 EQ 3: What are some major lessons learned to improve the effectiveness of CEAIU development strategies and the sustainability of its outcomes? FINDINGS 3. Lessons learned 3.a. From FGDs of participants The FGDs with participants revealed that the most significant change that happened in their lives because of the CEAIU activity was their acquisition of new knowledge and skills on livelihood, health, and values (Figure 41). Participants in all the FGDs unanimously declared that “Before, we did not know how to make puto cheese, banana chips, fabric conditioner, detergent powder, dishwashing liquid--but now we do.” They found the training useful and informative. The livelihood module provided them with hands-on experience in actually making and selling a product. They also appreciated the lectures on health where they learned home remedies for illnesses. Practical health applications often cited were on managing fever in children, using herbal medicines, quick remedies for toothache and diarrhea, proper and regular hand washing, and cleaning home surroundings. They also pointed out health-related misconceptions and malpractices that were corrected. Figure 41. Summary of focus group responses on the most significant changes felt by respondents because of the CEAIU activity. 64 The values module helped participants renew their faith and provided them guidance in dealing with their neighbors. The training became a venue for participants to mingle and become close with other community members especially during group activities. Overall, they found the training “easy and fun” with “very good trainers.” All expressed happiness and a feeling of accomplishment at graduation. For most of the participants, the food packs distributed during the sessions were not a major consideration in attending the training. While it can be concluded that the CEAIU was effective in imparting knowledge, most of the respondents did not sustain their BiB-related enterprises after the training. The most cited post￾training problems were the: (1) lack of operating capital for the next rounds of production; and (2) difficulty in sourcing of ingredients. As explained earlier, a participant was given a BiB kit as a loan to be repaid through weekly payments. After running the BiB enterprise, making a sale, and paying off the loan, the participant would not have enough working capital for the next round of production. While the training was ongoing, participants could avail of another BiB loan but once the activity ended, they had to find other ways to finance production. Ideally, the savings group would be there for this purpose but the amount of savings were usually not enough to fund the succeeding production cycles of all the members. Aside from the lack of operating the capital, the most cited problem was the difficulty in sourcing ingredients when the BiB kits became unavailable after the training. The most preferred BiBs were the fabric conditioner, dishwashing liquid, and detergent powder but the chemicals needed in their manufacture were not locally available. Participants in the more successful communities enterprising enough to source the ingredients did so by establishing contacts, usually through the pastors, with suppliers from faraway Cebu Province and Metro Manila. Most sample respondents, however, were not able to successfully link with suppliers either because of the remoteness of their communities or the pastors cannot easily make the connection. Also, supplies of these chemicals came in bulk requiring higher total costs. In some communities, continuing on with banana chips production was not feasible because raw bananas were in short supply. In at least one sample community, the glucose syrup that gives the banana chips the shiny look was not available even in the town center. Given these issues, the participants suggested that BiBs to be used in the training should have ingredients that can be easily sourced, preferably from within or near their locality. The training should also consider providing operating capital for production rounds after the training. They also suggested other training modules on services such as massages, manicure, pedicure, and dressmaking. 3.b. KII of Pastors and Counselors Pastors and counselors played a critical role before, during, and after the CEAIU training. They were actively involved in: nominating and selecting the ultrapoor trainees; making sure that participants regularly attended the training; conducting house-to-house visits to check on individual progress; and providing overall guidance and assistance to the participants even 65 during off-training days. In addition, the pastor was usually in charge of teaching the values module. The six counselors per community were handpicked by the pastor based on their capacity and dedication in providing assistance in the training. They were usually members of the church. Six out of ten pastors and 10 of 14 counselors interviewed found it difficult to implement the training. For them, the most challenging part was in dealing with uncooperative and unwilling participants. They often mentioned that many participants were initially interested only because of their mistaken notion of receiving dole-outs. The pastors and counselors had to expend a lot of time and effort to enjoin participants to regularly attend the training sessions and to encourage them to do their part during off-training hours. Furthermore, the training was on top of the pastors’ other regular tasks as leader of the local church. A couple of pastors reported that the church sometimes spent on snacks during the training and provided fares to participants who lived far from the training venue. The rest of the pastors, however, found the training easy and manageable. Whether they found their involvement difficult or easy, all of the pastors and counselors recognized that the training was part of their tasks in the church. When asked what they thought worked well, their responses validated the replies given by the participants in the FGDs and the interviews. They found the training very good overall and effective in giving knowledge to the participants. The livelihood module was informative and practical. They found the health component important especially in communities far from health centers, hospitals, and pharmacies. They also said that CEAIU provided an opportunity to create community awareness on the activities of the church. The training was usually held in the church premises. On areas which they thought did not work well, the pastors and counselors pointed to the frequent absenteeism and eventual dropping out of participants. It was difficult for them to motivate participants to regularly attend the training. They echoed the responses of the participants that some BiBs did not fit the community. They also reported the same post-training concerns on sourcing of ingredients, market for the products, and lack of operating capital for the next rounds of production. 3.c. KII of Dropouts A dropped participant is defined as “someone whose name was on the original participant list but has never attended at all [or] stops attending during the 16-week training and fails to complete the training” (CEAIU Final Report). A dropout is replaced by a “visitor”, defined as someone who was not on the original list but attended at least one training session. A visitor who attended the training regularly could be reclassified as a replacement participant. Figures 42a and 42b show the number of dropouts per community per batch. These include the visitors and replacements who also eventually dropped out of the training. The average number of dropouts was seven in Batch A and 18 in Batch B, which represented an average of 24% and 60% dropout rates in these batches, respectively. These dropout rates were considerable given that there were 66 only 30 participants originally screened and accepted per community. The KII of dropouts revealed that the top reason for dropping out was their difficulty in finding time to attend the training. Among the specific cases they cited were scheduling conflicts with their occupation; having to take care of their children or other family members; and being busy with household chores. Another main reason was sickness of the participants themselves. The KII of pastors and counselors revealed, however, that based on their own observation, the main reasons for dropping out were the lack of interest and the mistaken notion of receiving dole outs. They said that many participants would show interest only during the first few weeks of the training. Figure 43 shows that about half of the respondents dropped out within the first five weeks. These were those who responded that their busy schedules with work or family made them drop out. These could have been, from the point of view of the pastors and counselors, the participants who would lose interest after finding out that they would receive no dole-outs from the training. The figure also shows that about a third of the respondents dropped out after attending more than 10 weeks of the 16-week training. Half of them said that sickness was the main reason. When asked if they were willing to be participants again, all of the dropouts interviewed answered in the affirmative except for two respondents who mentioned their busy domestic schedules. (42.a) 67 (42.b) Figures 42.a and 42.b. Number of dropouts in the Batch A and B communities. Figure 43. Number of respondents who dropped out of the training within a certain number of weeks. 0 1 2 3 4 5 6 7 8 9 10 1-5 weeks 6-10 weeks More than 10 weeks Does not remember Number of respondents who dropped out within a certain number of weeks (n=19) (Note: mean number of weeks attended by all respondents = 6.8) 68 3.d. Performance 3.d.1. Accomplishments based on targets For each of the three batches, the performance indicators of the CEAIU based on the award document were on the following: a) List of communities to be included b) Number of participants who graduated from the training c) Number of participants who opened microloans (through the BiBs) d) Number of microenterprises/individuals supported by USG enterprise assistance started e) Number of savings groups started f) Number of participants using a mobile money account CEAIU was also required to submit its Performance Monitoring and Evaluation Plan (incorporating draft Gender Action Plan); Branding Strategy and Marking Plan; and the Environmental Mitigation and Monitoring Plan. For Batches A and B, Figure 44 shows that all of the specified targets for the indicators enumerated above were realized and even exceeded. For example, the number of Batch A and B graduates were, respectively, 21% and 13% more than the target. Participants who received at least one BiB kit (the main instructional device of the livelihood component) were 22% and 17% percent more than the Batch A and B targets, respectively. Those who were able to fully repay the BiB loans were 37% and 42% more than the respective batch targets. It should be noted, however, that for each Batch, the targeted number of participants who would receive at least one BiB kit is only 75% of the targeted number of graduates. Likewise, the targeted number of participants expected to repay the BiB loan is just 75% of those targeted to receive the said BiB. The biggest overachievement reported was on the number of mobile money account users which were 282% and 536% higher than the targets for Batch A and B communities, respectively. However, these do not pertain to actual users but to the number of participants who registered into mobile banking accounts. The registration with reputable mobile banking service providers was facilitated by CEAIU to enable participants to use their mobile phones in their banking transaction needs. This evaluation’s fieldwork found out, however, that this service was not being utilized at all by the respondents in the sample communities. Instead, some savings group leaders only used the retailing SIM cards for the business of selling cellphone “e-load” (or phone use credits). Some communities in the sample were not introduced at all to this mobile banking facility because of poor cellular phone signal in their areas. 69 Figure 44. Summary accomplishments of the CEAIU Activity, Batches A and B. (Sources: CEAIU milestone reports and final report) While the activity was able to deliver its targets as specified in the award, these accomplishments were for the low-level activities and indicators of CEAIU’s theory of change (Figure 2 earlier). Higher level indicators of intermediate results, such as on household income and health status, were not specifically promised by the activity but nonetheless monitored and analyzed by the activity. 3.d.2. Monitoring and evaluation (M&E) The internal M&E of the CEAIU activity was guided by its monitoring, evaluation and learning (M, E, & L) plan. Under this plan, there were different levels of responsibility in collecting, validating, and analyzing data in order to track the accomplishments of the activity deliverables. Roughly, the flow of data collection and analysis and corresponding responsibilities is given by Figure 45. 70 Figure 45. Flow of CEAIU activity data collection. CEAIU was able to implement this plan and comprehensively monitor weekly performance. The M&E process captured all the indicators under every strategy implemented by the activity. Data collection on the training site was done regularly for submission and validation to the next responsibility center until they reached the national headquarters for encoding and further analysis. All training-related indicators (such as attendance, BiB take-up, micro-loan repayment, and amount of savings) were collected and summarized in the relevant activity reports. All data was entered in the activity’s databases which were useful sources in the easy tracking and analysis of results. CEAIU was able to regularly produce and submit the reports required by USAID. The CEAIU databases made available to this evaluation include the following: 1. Pre- and post-survey database (using detailed household questionnaire) 2. Master list areas_GM2 Iloilo 3. Iloilo BiB Community Data (Batch B only) 4. Iloilo Community Data (Batch A only) 71 5. Health database 6. Livelihood Tracking Booklets (Batches A and B) 7. BiB-participant data (Batch B only) 8. BiB-participant payments (Batch B only) 9. BiB Matching Tool 10. Microsavings database 11. Policy Making 1 Monitoring: Savings and Meetings 12. SG Attendance Monitoring (Weeks 5-16) 13. Trainers’ kit inventory (Batches A and B) 14. Challenges (Batch B only) 15. Savings Summary (Batch B only) CONCLUSIONS UNDER EQ 3: The CEAIU activity was effective in transferring skills and knowledge, specifically on livelihood and health. The training was able to provide an opportunity for participants not only to learn basic business concepts but to have practical experiences in making and selling a product. They also reported sustained awareness of the teachings under the health and values modules. However, most of the respondents did not sustain their BiB-related enterprises after the activity. The target deliverables of the activity were realized and even exceeded. The mobile banking facility was not actually used for their intended purposes. Project implementation and M&E is commendable, with excellent data capture and analysis, which effectively tracked the delivery of the immediate results. It is premature to conclude, however, if these results would ultimately lead to the realization of higher level outcomes of improved household well-being. 72 EQ 4: What are some key CEAIU characteristics and enabling factors/conditions that enhance the scalability and replicability of the CEAIU development approach? FINDINGS 4. Scalability and replicability of the CEAIU development approach 4.a. Activity expenses The USAID grant of USD 230,000 to ICM is a fixed-amount award to implement the CEAIU Activity. The USAID assistance was used as a counterpart fund to finance Batches A, B, and C of this activity, with the rest of the expenses shouldered by ICM. Table 6 shows that the total expenses in implementing the three batches amounted to USD 567,805.56, with the USAID assistance accounting for 41%. Expenses per batch amounted to around USD 190,000 or USD 6,308.95 per community. Accounting for just the amount of USAID assistance, the table further shows the amount per training day (USD 159.72) and per graduate (USD 90.13). These can be useful when cost-benchmarking with similar activities supported by USAID such as assistance for training and capacity-building. Determining the investment worthiness of the assistance would be informative but this requires that all other costs should be accounted for and compared with all the benefits associated with the activity. This evaluation, however, is not designed to provide a full cost-benefit analysis of the CEAIU. Table 6. Expenses of the CEAIU activity. 73 4.b. KII of Barangay (Village) Chairs and Health Workers A CEAIU community is under the jurisdiction of a barangay (village), the smallest political unit in the country. It is headed by a chair chosen by the people in local elections. The barangay chairs in the sample communities were interviewed in order to get their impressions of the CEAIU activity. According to the chairs, CEAIU formally notified them in writing about the conduct of the training but no specific assistance was requested from them. Their participation, whenever sought, was limited to helping identify potential participants and attending the launching and graduation ceremonies of the training. Despite their limited knowledge on the actual conduct of the activity, the barangay chairs said that feedback from the CEAIU participants were generally good, specifically on their learning new skills and practical applications on livelihood and health. They suggested that such training should involve more participants (including children, if possible), help connect people to local industries and services, and provide financing for business startups. They also said that other activities targeting the poor in their respective barangays would usually be done by the local and national government, non-governmental organizations, local schools and universities, international institutions, and private businesses. Some of these activities were on scholarship for poor students, medical and feeding missions, and infrastructure support. According to the chairs, the types of assistance their poor constituents mostly needed were on food, medicines, livelihood, microfinancing, and health insurance. When asked about the challenges if the barangay or the local government will take on an activity similar to the CEAIU, they identified the potential lack of resources (materials, expertise, funds and logistics) and the attitude of participants (dole-out mentality). Interviews of barangay health workers revealed that they were not fully aware of the CEAIU activity because all formal communication was given only to the chair. They also said that they could not observe significant changes in the health status of the CEAIU trainees, although one reported that some participants had cleaner houses and surroundings after the training. They also reported that similar health-related projects were implemented in their barangays by the local government, specifically on disease prevention, waste management, feeding and nutrition programs, and distribution of vitamins and supplements. According to the health workers, the important needs of their barangays were on medical equipment, care for the malnourished, and addressing common diseases in their areas. Even though their participation was limited, the barangay chairs and health workers thought that the CEAIU activity was able to provide knowledge to participants on livelihood and health. They said that challenges in terms of resources and attitude of participants need to be hurdled if the barangay or the local government will replicate an activity similar to CEAIU. 74 4.c. Characteristics of sample communities that showed some amounts of success It was mentioned in an earlier section that among the ten communities sampled in this evaluation, only three showed some amount of success in sustaining the enterprises learned from the training. These three communities have participants continuing on with their BiB-related business even at the time of the evaluation fieldwork, which was a full year after the graduation of Batch A participants and around eight months after Batch B. The rest of the communities have stopped their BiB businesses and the operations of the savings groups. While the training was done in the same manner across all communities, different characteristics were observed that could have led to the variations in the outcomes. The following were the main observations on the characteristics of communities with different results:  Communities near the town centers or public markets did not show much success. In such communities, there were many opportunities for participants to find jobs in public markets or commercial areas. The opportunity cost of attending the training was presumably high since participants could have spent the time becoming vendors in the market or accepting odd jobs. There was also relatively higher number of dropouts in these communities. Furthermore, the participants’ BiBs (such as puto cheese and banana chips) would tend to have difficulty in competing with the other numerous products sold in the nearby town center or public market. Facing stiff competition with more established business enterprises in these areas would indeed be challenging for first-time entrepreneurs. The relatively more successful communities in the sample were far from commercial centers but participants can be easily gathered during and outside training hours. They usually have lower dropout rates and have stronger bonds among them. Given this observation, the challenge is how to strengthen training design so that participants in these communities can sustain their interest and immediate results will make it worth their while to complete the program.  The role of pastor and/or his wife is crucial in sustaining the savings group and the BiB-related enterprises after the training. In the more successful communities, the active role of the pastor and his wife was prominent. After the training, they acted as “champions” on whom the participants placed their trust in continuing on with the savings group and their BiB-related enterprises. They provided leadership in the group￾managed businesses and made important business decisions for the group. It was evident in the fieldwork that in the less successful communities, the deep involvement of the pastor or wife after the activity was not noticeable. Ideally, however, participants should be able to fend for themselves in managing their own entrepreneurial pursuits. For this reason, the pastors may consider phasing themselves out of direct roles in the group￾managed activities. However, this is contingent on the readiness of the participants to take on their own leadership roles in managing the enterprises. 75  Spin-offs in the form of group-managed businesses were sustained through the savings group, usually centered around the pastor/wife. While the CEAIU training was basically designed to develop individual capacities, the BiB-related businesses that were sustained after the activity were implemented in groups. This could have also been a result of the CEAIU activity since the participants’ interaction among each other was encouraged. After the training, the savings group evolved into a critical mass of participants that continued on with the BiB-related business enterprise, with the pastor or wife at the core. Specifically, they sustained the businesses by centrally producing the commodities to take advantage of scale economies in production. Individual members sold the products and all profits were plowed back to the savings group.  The fabric conditioner, detergent powder, and dishwashing liquid BiBs were the most promising enterprises. These enterprises were the ones sustained by the more successful communities because of three important considerations. First, they were not perishable and thus can be stored for a prolonged period of time while not yet put up on sale. Second, they were very competitive in the market since they were much cheaper than commercial brands with the same level of quality. In fact, although they cannot equal the more upscale packaging of commercial brands, participants said that they could make their own product better. For example, the detergent and fabric conditioner can be formulated to become more fragrant and bubbly. And lastly, these products were widely￾used and essential commodities in the household. Participants can use them for everyday home consumption instead of buying the more expensive brands. The biggest challenges for these enterprises, however, were the supply of chemical ingredients needed in their production.  Remoteness and socio-economic conditions of communities were limiting factors. Communities situated in isolated areas with rough roads naturally would have difficulty sourcing ingredients for their enterprises. Furthermore, the market for the products in these communities was quite limited given the sparse number of potential consumers in these areas. There were also communities where the garden-in-a-box cannot grow because of herbicide spraying in nearby sugarcane farms. In some communities, bananas used for the BiB enterprise on banana chips were in short supply.  Volunteer work by pastors and counselors were crucial pre-, during, and post￾training. The unpaid work of the pastors and counselors were very important in the implementation of CEAIU. Their voluntary services were for difficult tasks on: scouting for a limited number of participants from among a huge population of ultrapoor in their communities; assembling attendees and making sure they sustain their interest in the training; making house-to-house follow-up on participants’ progress; leading the group meetings outside training hours; leading the management of group businesses long after the training; and conducting the values lecture (by the pastor). The replicability of the CEAIU intervention should consider the enormous work put in by these pastors and counselors. 76 CONCLUSIONS UNDER EQ 4: Even though the CEAIU was uniformly implemented across communities, variations in outcomes highlight the need to design the training according to community characteristics. Only three of the ten few sampled communities showed some amounts of success in sustaining the micro￾enterprises. There are indications that the success (or not) of sustaining immediate results long after the activity has concluded is dependent on characteristics such as community location, involvement of the pastors, and interest among the participants. Local officials thought that the CEAIU activity was able to provide knowledge to participants on livelihood and health although they cited similar activities done in their barangays. If the barangay or the local government would replicate a similar activity, they opined that challenges in terms of resources and attitude of participants will need to be hurdled. OVERALL RECOMMENDATIONS Based on the findings and conclusions under all the evaluation questions, the following are the overall recommendations of this evaluation: 1. The participants stand to benefit more if the training design is suited to the community’s needs and conditions. For example, the BiBs introduced to each community should be subjected to a feasibility analysis in order to address the issues of input sourcing and marketing the product. Sustaining the enterprises was difficult when ingredients were scarce or costly and the products cannot easily compete in the market. While the BiBs were primarily designed for instructional purposes, participants would tend to easily continue on with these enterprises if their feasibility is established beforehand. 2. The choice of communities should consider the different characteristics of the relatively more successful areas identified in the evaluation. It appeared that the commitment of the pastor, especially after the training, was crucial. Communities near the town centers did not show much success. While considering these specific characteristics may not guarantee success in all cases, the recommendation is to provide a careful analysis of potential communities to be included. 3. A relay activity for financing successive rounds of production of the enterprises is worth considering. The CEAIU activity was a good facility to impart skills but the findings of this evaluation point to the difficulty faced by participants in financing the succeeding rounds of production after the training. Implementing and monitoring an activity for financing enterprises of CEAIU graduates can benefit from the strong M&E capacity shown by the CEAIU implementers. 77 ANNEXES 78 ANNEX I: EVALUATION STATEMENT OF WORK Evaluation of “Catalyzing Entrepreneurship among Iloilo’s Ultra-Poor” Activity I. PURPOSE OF THE EVALUATION This external or third party evaluation will focus solely on the USAID-assisted “Catalyzing Entrepreneurship among Iloilo’s Ultra-Poor” (CEAIU) Activity being implemented by International Care Ministries Foundation, Inc. (ICMFI) in selected communities in and around Iloilo City. CEAIU is part of ICMFI’s larger Transform Program. The main purpose of the evaluation is to provide emerging indications and empirical evidences of the contribution(s) of CEAIU’s ultra-poor focused approach, outputs and outcomes to achieving the main goal of the USAID Cities Development Initiatives (CDI) on accelerating and sustaining broad-based, inclusive economic growth in second-tier cities. In addition to measuring changes in beneficiary-level welfare as a result of CEAIU assistance, this evaluation will focus on the higher-level relationship between CEAIU and CDI. It will document in what way and to what extent the CEAIU activity contributes to the main goal, purpose and higher-level results of CDI. In the process, it can highlight challenges and barriers that may lessen the activity’s contribution to the goal and higher-level results of CDI. The evaluation will complement ICM’s efforts to monitor the results of CEAIU. The evaluation will also produce recommendations in terms of the scalability potential and replicability the CEAIU model. II. ACTIVITY INFORMATION Activity Name Catalyzing Entrepreneurship among Iloilo’s Ultra Poor Activity Contractor International Care Ministries Foundation, Inc. ICM Cooperative Agreement # AID-492-F-15-00003 Total Estimated Cost (TEC) $230,000 Life of Activity September 2, 2015 to February 1, 2017 Active Geographic Area Iloilo City and Iloilo Province, Western Visayas (Region VI), Philippines Mission Objective DO1 Broad-based and Inclusive Growth Accelerated and Sustained III. ACTIVITY BACKGROUND International Care Ministries Foundation, Inc. (ICMFI) received a 17-month, $230,000 grant from USAID in September 2015 to pilot an approach to catalyze entrepreneurship among ultra-poor7 households in Iloilo City8 . The organization has been working with the Filipino ultra-poor in urban slums, coastal fishing villages and interior mountain hamlets since 1992. USAID/Philippines’ current Country Development Cooperation Strategy (CDCS) seeks to accelerate and make economic growth more inclusive and sustainable through the Partnership for Growth (PFG), which promotes trade and investment, greater competition, increased transparency and improved fiscal policy and management. Through PFG, USAID supports the Philippines’ aim to increase employment opportunities by working with the Government of 7 ICM defines the ultra-poor as those who live on less than $0.50 (PHP 22) a day, less than half the $1.25 (PHP 55) that the World Bank considers as the daily income threshold of the extremely poor. The Philippine National Economic Development Authority (NEDA) reported that during the first quarter of 2014, there were 25.8 million poor Filipinos. ICM estimates that between seven and eight million Filipinos are ultra￾poor. 8 ICM’s geographical focus is Iloilo City, one of the three initial cities targeted by USAID/Philippines’ Cities Development Initiative. 79 the Philippines (GPH) to improve governance, expand its fiscal space, increase needed infrastructure, and strengthen its human capacity. The grant to ICM contributes to inclusive economic growth by promoting the participation of those at the very bottom of the lowest economic quintile, and will help achieve USAID’s goal of a broad-based, inclusive and sustainable growth in the Philippines. This CEAIU activity will facilitate inclusive economic growth for approximately 10,000 ultra-poor individuals from 2,000 households in Iloilo by delivering “Transform,” an intensive, community-based, 16-week training program. The activity will expand ICM’s current holistic development program to new areas in Iloilo, including slum, coastal fishing and remote interior communities. In the last fiscal year, 95 percent of Transform participants were extremely poor, living on less than $1.25 a day. This group included a subset of 63 percent who were ultra-poor. The remaining five percent of Transform participants earned between $1.25 and $2.00 per day. From pre-participation surveys of candidates for Transform, ICM has documented that the extremely poor, apart from being cash poor, are also food insecure and suffer from poor health. The children in ultra-poor households are often undernourished, and many of the mothers have lost a child. A significant proportion of the ultra-poor is functionally illiterate or did not finish primary school. In 2014, it was estimated that 580,937 (or 26 percent) of the population of Iloilo province lived under the annual poverty threshold of PHP18, 827, which translates to $1.20 per day based on an exchange rate of $1.00=PHP44.00. Of the more than 580,000 people living below the poverty line, 147,000 were considered to be ultra-poor. ICM designed the Transform program as an interactive learning experience for low-literacy oral learners, focusing on two key points that inhibit the creation of new businesses: poor health and lack of positive experience with entrepreneurship. ICM’s surveys document that poor health inhibits the ultra-poor from engaging in income￾generating activities. The CEAIU activity will leverage the Transform strategy developed over two decades to expand its coverage in partnership with local community leaders who provide ongoing support even after the activity period ends. The new knowledge, skills and tools, and the motivation to increase income, coupled with access to financing or an asset that could be converted into a business, will allow the ultra-poor participants to take steps to contribute to economic growth. The ICM’s technical approach focuses on four pillars: (1) Removal of obstacles faced by the ultra-poor that block their becoming entrepreneurs; (2) Equipping the ultra-poor with the skills and tools to plan and manage a small business; (3) Provision of micro-loans to start a business; and (4) Following up with technical support to sustain and grow the business. Its specific strategies to overcome problems facing the ultra-poor include: (1) Creation of a healthy environment for entrepreneurship; (2) Creation of healthy foundations for entrepreneurship; and (3) Creation of opportunities to develop entrepreneurial skills and experience. This activity is being implemented over the course of 17 months, divided into three 6-week “batches,” with each batch serving 30 households in 30 ultra-poor communities. It will reach 90 communities as follows:  Batch A: In July 2015, ICM identified 30 ultra-poor communities that have appropriately trained local leaders and community volunteers for program partnership. In August 2015, ICM staff and local partnering leaders utilized a poverty scorecard to identify and invite 30 families in each community to participate in the training. In September 2015, ICM trained community leaders to prepare for the program and the metrics teams surveyed each participant. From October 2015 to January 2016, ICM implemented the 16-week Transform training in the 30 communities. Each week, ICM livelihood trainers conducted a 30-minute session on entrepreneurship principles (financial literacy, savings groups and an array of micro-businesses such as food selling and cleaning products) and ICM health trainers conducted a 30-minute session on health practices (hygiene and infections, safe motherhood, healthy childrearing and other health lessons). In Batch A alone, ICM livelihood trainers distributed 539 micro loans in the form of “Business in a Box” (Bib) starter kit to participants who wished to try out new livelihoods. At the end of each 30- minute session, ICM staff distributed nutritional food supplements to program participants. In February 2016, ICM metrics teams surveyed every graduating participant. Results of the pre- and post-surveys for Batch A were made available in March 2016. 80  Batch B: Identification of 30 communities happened in November 2015 and the results of Batch B surveys should be available in July 2016.  Batch C: Identification of thirty communities will happen in March 2016 and the results of Batch C surveys should be available in November 2016. CEAIU will cover a total of 30 communities per 16-week batch, with at least 23 families in each community for about 700 families in each batch. Three batches during the overall project period will total approximately 2,100 families. Since each ICM family averages five people, the project could directly impact around 10,500 family members. Final reports for the entire project will be completed by January 2017. IV. EVALUATION QUESTIONS In order to explore the contribution(s) of ICM’s CEAIU activity to achieving the goals of the USAID Cities Development Initiatives (CDI) project in Iloilo, this evaluation will answer the following evaluation questions: 1. What major changes in household conditions and well-being – positive or negative - are emerging as a result of USAID assistance to the CEAIU activity? 2. What key outputs and emerging outcomes of the CEAIU activity contribute to the CDI project outcomes? What are the major challenges and barriers to achieving these CDI outcomes? 3. What are some evidences, if any, of the sustainability of CEAIU’s development approach and its outcomes? 4. What are some key CEAIU characteristics and enabling factors/conditions that enhance the scalability and replicability of the CEAIU development approach? V. EVALUATION DESIGN AND METHODOLOGY This section discusses indicative evaluation design and set of research methods to address the data and analytical requirements in order to answer the evaluation questions. Evaluation Design The evaluation shall be cursory in scope and depth since its focus will be on two aspects only: (1) the measurement of CEAIU’s emerging outcomes at the household level, and (2) the relationship between CEAIU and USAID’s Cities Development Initiative. The evaluation shall explore narrowly both performance and impact. It shall be guided by a Theory of Change (ToC)9 designed to lay the foundation for linking the CEAIU-level results framework to the overall CDI-level results framework. Specifically and based on the ToC, it shall trace the output/outcome relationships between CEAIU and CDI. Data Analysis Methods For Evaluation Question No.1, the evaluation shall use a before-after evaluation approach employing both quantitative and qualitative analyses of household-level data periodically and previously collected by ICM. Changes in household conditions and well-being shall be measured using key variables selected from the approximately 160 9 “A theory of change is the articulation of the underlying beliefs and assumptions that guide a service delivery strategy and are believed to be critical for producing change and improvement. Theories of change represent beliefs about what is needed by the target population and what strategies will enable them to meet those needs. They establish a context for considering the connection between a system’s mission, strategies and actual outcomes, while creating links between who is being served, the strategies or activities that are being implemented, and the desired outcomes.” (Source: INSP. 2005. Theory of Change Tool/Manual. P.6) 81 data points of the ICM survey questionnaire .The selection of (explanatory) variables will be done by the evaluation contractor in consultation with or vetted by the Evaluation Point of Contact and the AOR. It shall quantitatively test significant changes in before-after household conditions and well-being. It shall establish correlations and causalities of key explanatory variables with household condition/well-being variables. Multi-variate regression analysis, if appropriate, shall also be employed to establish and measure causalities. For Evaluation Question No. 2, the evaluation shall use descriptive analysis to determine output/outcome relationships between CEAIU and CDI. This will mostly be qualitative in nature and to a certain extent, use descriptive statistics to analyze quantitative data. Qualitative analysis shall provide a clear narrative to enable a better understanding of the “breadth and depth” of the relationship between CEAIU and CDI10 . For Evaluation Question Nos. 3-4, the evaluation shall collate and synthesize primary and secondary data gathered through triangulation to provide comprehensive discussions of major lessons on the sustainability of CEAIU approach and its outcomes. For Evaluation Question No. 3, the contractor may refer to the 2014 USAID publication on “Local Systems: A Framework for Supporting Sustained Development” as guide to analyzing the sustainability of CEAIU approach and its outcomes. The analysis of factors and conditions affecting scalability and replicability potentials shall mainly be descriptive and qualitative. Data Collection Methods Triangulation shall be used to gather data through a combination of techniques, namely: (1) desk review of secondary sources and survey data found in the ICM internal database; (2) primary data collection through key informant interview (KII) and focus group discussion (FGD); and (3) limited call-back or follow-through survey to verify collated data found in the internal database and/or primary data obtained from KIIs and FGDs. For Evaluation Question No. 1, a random sample of respondents will be taken from Batches A and B beneficiaries (expected number = 1800). Slovin’s formula or other formulae will be used to determine the appropriate sample size given a chosen confidence level. The descriptive analysis for Evaluation Question No. 2 shall rely on triangulation for gathering data. The number and type of KII and FGD shall depend on discussions with the CEAIU activity team and the review of secondary sources like activity implementation and M&E reports, etc. VI. DELIVERABLES AND REPORTING REQUIREMENTS Listed below are the specific deliverables, reporting requirements and timeframes of the evaluation: 1. Evaluation Design and Work Plan – Within five (5) business days after awarding of the award of the contract, a draft evaluation design and work plan for the evaluation shall be completed by the contractor and presented to the evaluation Contracting Officer’s Representative (COR) for review and approval. The evaluation design shall contain: a. Detailed Evaluation Design Matrix which outlines specific methods for data collection and analysis per evaluation question; b. Draft questionnaires for the survey, list of questions for the KII and FGD, and other data collection instruments; and c. Survey design (including sampling), list of potential interviewees and/or participants to the FGD. 2. In-briefing – Within two weeks of the award of the contract, the contractor shall have an in-briefing with the evaluation COR, Evaluation Point of Contact (EPOC), activity AOR and the CEAIU Project Team for introductions and to discuss the contractor’s understanding of the proposed evaluation, deliverables and 10 The CEAIU’s development hypothesis (and/or Theory of Change) and logical framework and the CDI project’s results framework will be important references in constructing the narrative. 82 reportorial requirements specified in the evaluation SOW, and identify initial assumptions and limitations of the evaluation. 3. Presentation of Evaluation Findings – The contractor shall present the evaluation findings in the following venues: i. Presentation of Initial, Major Evaluation Findings – A summary of initial, major findings shall be presented to the evaluation COR and EPOC for their review. This shall happen after data have been collated and subjected to initial analysis. This shall be scheduled, at the latest, 40 business days after awarding of the contract. ii. Presentation of Full Evaluation Findings – The draft full evaluation report shall be presented to the evaluation COR, EPOC, activity AOR, USAID senior officials and officers two weeks prior to the completion date of the evaluation. 4. Draft Final Evaluation Report - The draft final report shall be submitted no later than fifteen (15) calendar days before completion of contract. All project data (qualitative and quantitative) and records must be submitted in electronic format, in easily readable format (doc and pdf versions), organized and documented for use by those not fully familiar with the project or evaluation. 5. Final Evaluation Report – Copies (2 hard and 1 electronic) of the final report shall be submitted no later than five (5) calendar days before completion of contract. VII. EVALUATION TEAM COMPOSITION USAID shall procure the services of one qualified individual through a purchase order mechanism to deliver the requirements cited under Section VI. S/he may choose to engage other individuals to assist her/him in any phase of the evaluation under the purchase order at no additional cost to the US Government. The contractor should possess the following minimum qualifications: 1. Proven experience in conducting high-quality and authoritative evaluations of development assistance/programs/projects. At most 3 evaluation publications, journal articles, reports, technical papers and similar documents under his/her senior authorship shall be submitted for review by USAID. 2. Adequate knowledge and skill on appropriate research methods for conducting performance and impact evaluation. S/he must be adept in both quantitative and qualitative analyses, and in the use of appropriate software for data analysis. 3. Strong technical writing skills and familiarity with USAID technical and style guidance in report writing; and 4. Graduate degree in economics, business and management, public administration, policy studies or other related social sciences. VIII. EVALUATION SCHEDULE The table below shows the indicative tasks, level of effort and timelines for the evaluation. This shall be refined by the contractor when s/he submits the evaluation design and work plan cited in Section VI. The total level of effort of 42 work-days shall be spread over a period of three months (from the date of contract awarding to submission of the final technical report). The indicative three-month duration of this evaluation contract covers June 1 – August 31, 2016. 83 Task(s) Level of Effort (work￾days First Month Second Month Third Month First Week Second Week Third Week Fourth Week First Week Second Week Third Week Fourth Week First Week Second Week Third Week Fourth Week Mobilization 0.5 Present Evaluation Design/Work Plan 0.5 Prepare Data Collection Instruments 5 Desk Review of Secondary Data and ICM Database 5 Gather Primary Data using ICM Database 5 Data Gathering (KII and FGD) 5 Call￾back/Follow￾through Survey 5 Quantitative Data Analysis (of ICM Database 5 Other Data Analysis 5 Presentation of Initial Findings 0.5 Report Writing 5 Final Report Presentation 0.5 Total LOE 42 VIII. FINAL REPORT FORMAT The evaluation final report should include an executive summary; introduction; background of the local context and the projects being evaluated; the main evaluation questions; the methodology or methodologies; the limitations to the evaluation; findings, conclusions, and recommendations; and lessons learned (if applicable) The executive summary should be three to five pages in length and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations and lessons learned (if applicable). The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.) 84 The annexes to the report shall include:  The evaluation SOW;  Any statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team;  All tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides;  Sources of information, properly identified and listed; and  Disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of interest. All quantitative data collected by the evaluation team must be provided in an electronic file in easily readable format agreed upon with the COR. The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed. USAID/Philippines will make the final evaluation reports publicly available through the Development Experience Clearinghouse within 90 calendar days of official completion date of the evaluation contract. IX. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per USAID Evaluation Policy, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report:  The evaluation report should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate what worked in the project, what did not, and why.  Evaluation reports shall address all evaluation questions included in the SOW.  The evaluation report should include the SOW as an annex. All modifications to the SOW— whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline— need to be agreed upon in writing by USAID.  The evaluation methodology shall be explained in detail. All tools used in conducting the evaluation—such as questionnaires, checklists, and discussion guides—will be included in an annex in the final report.  Evaluation findings will assess outcomes and impact on males and females.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions. Findings should be specific, concise, and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings.  Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. 85 86 87 88 ANNEX II: EVALUATION METHODS AND LIMITATIONS As explained earlier, this evaluation is limited to gathering indications of the emerging outcomes of the CEAIU activity. The major sources of data were the CEAIU activity databases and reports and interview data from the fieldwork. It would have been ideal to gather information from a larger sample size from among the 1,632 participant-graduates of the activity, but this evaluation was not designed for such extensive primary data collection. This evaluation resorted to the use of FGDs and KIIs in ten sample communities in order to gather quick primary information from the target respondents. Quantitative methods used in this evaluation were limited to descriptive analysis of activity results such as percentage changes, frequency counts, and comparison of means. Qualitative information was taken from the FGDs and KIIs during the fieldwork covering a sample of CEAIU communities. The sample CEAIU communities were randomly selected from towns in Iloilo province. The KIIs of activity front liners (pastors, counselors, ICM personnel) provided useful information on the implementation issues of the CEAIU. Their responses were compiled to see a common thread in their actual experiences and learnings in implementing the activity. The FGDs provided useful information on the emerging outcomes of the activity. An FGD for each community involved at least six participant-graduates who were asked open-ended questions such as the significant changes they experienced because of the activity, what they liked about the activity, and what they thought needed to be improved. From their discussions, common threads of responses were noted which formed the bases of the answers in the relevant evaluation questions. Before the FGDs were conducted, the participants were subjected to a face-to-face interview using a short questionnaire in order to gather information that will be compared with their pre- and post-training responses. This provided further details on the outcomes of the activity long after its culmination. Whenever applicable, the sets of information gathered from the database, FGDs, and KIIs were compared among each other for validation and cross-checking. In fact, the key findings of this evaluation were an outcome of the triangulation of the results from the said sources and methods. 89 For participants: Livelihood: Q01: Names of adults in the household that earn income Q02: Type of work or business (use one row for each type of work or business) Q03: Daily/ Monthly income in the past month from work or business Use additional sheets if necessary 1 2 3 Q04 Does your household have debts in the past month? If yes, how much, and for what purposes? _________________ Yes No Don’t know Date: Time: Respondent: ( ) Participant ( ) Dropout ( ) Non-participant ( ) Counselor/Volunteer ( ) Pastor ( ) ICM Local Office staff ( ) Barangay Chairman/Councilor ( ) Barangay Health Worker ( ) SURGE Local Office Staff Name: Mobile no.: Batch: ( ) A ( ) B Community ID: Participant ID: Town/City: Geographical type: A1 ( ) Actual respondent ( ) Replacement A2 Survey successful? ( ) Yes ( ) No, reason: 1 2 3 4 5 6 Q05 In the past week, how many times did you eat in a day? Once Twice Thrice More than thrice Do not eat Don’t know Q06 How much did your household spend on rice, corn in the past week? _____________________PHP Use back pages for more notes, use question number for reference Page 1 ANNEX III: DATA COLLECTION INSTRUMENTS Enumerator’s name: 90 1 2 3 4 5 6 Q11 How much did your household spend on electricity last month? _____________________PHP (If zero, indicate if house has no electricity at all.) Q12 How much did your household spend on water in the past month? _____________________PHP (If zero, indicate where they get free water.) Q13 From which type of water sources do you currently get your drinking water today? Public Tap Tubewell/ Handpum p River/La ke/Pond In￾home Tap Other: _______ Don’t Know Q07 How much did your household spend on fruits and vegetables in the past week? _____________________PHP Q08 How much did your household spend on meat, poultry and fish in the past week? _____________________PHP Q09 How much did your household spend on mobile phone load in the past week? _____________________PHP Q10 How much did your household spend on transportation in the past week? _____________________PHP 1 2 3 4 5 Q14 Who decides more in the household before Transform? Father Mother Eldest child Grandmothe r/Grandfathe r Others: __________ Q15 Who decides more in the household after Transform? Father Mother Eldest child Grandmothe r/Grandfathe r Others: __________ Q16 Which among these areas did the project help you the most? Kindly rank the choices from 1 to 6 (1- highest, 6-lowest) ______ Income ______ Health ______ Relationship with others ______ Acquiring new skills (e.g. puto making, detergent making) ______ Faith/Values ______ Others (specify) Q17 What is the most significant change that you noticed in your life after CEAIU and why? (If no significant change, indicate reason.) Show Page 7 of this questionnaire for easier visualization Use back pages for more notes, use question number for reference Page 2 91 Training: BiBs/Microloans: For participants who did not take a BiB: Q18 For graduates: What motivated you to attend the weekly training and finish it until the end? Kindly rank the choices from 1 to 5 (1-highest, 5- lowest) ______ Skills to be learned from the program ______ Opportunities to mingle with others ______ Make idle time productive ______ Food packs good for 1 week ______ Others (specify) Q19 Did you purchase a BiB? How many and what are these BiBs? If not, proceed to Q23 Q20 Did you find it easy to use the BiB kits? Why or why not? Q21 Were you able to repay the BiB(s)? If not, why? Q22 Currently, do you still have a BiB or microenterprise that you started with the Transform? ___Yes ____No If no, when did you stop and why? Proceed to Q24 Q23 Why didn’t you take the opportunity to try the BiBs offered? Proceed to Q36 Show Page 7 of this questionnaire for easier visualization Use back pages for more notes, use question number for reference Page 3 92 Entrepreneurial skills and competencies: For those with microbusinesses: Commitment to work contract: Q24 If you still have microbusiness from your BiB, where do you buy or source your ingredients? Are they cheaper or costlier? Q25 Do you have additional business(es) established aside from the BiBs? If yes, what are these business(es) and when did you establish them? Q26 In the past month, how much did you earn from these microenterprise(s)? Specify each. Q27 What are the problems you encountered in running a microenterprise in the past month? Q28 How did you overcome these problems? Q29 Who among the family members in your household help you in this business(es)? Q30 What are your plans for your microenterprise(s)? SUGGEST ANY OR ALL OF THE FOLLOWING IF WITH NO DIRECT ANSWER: Continue the BiB(s)? Expand the product line? Search for other target markets? Branch out? Cease operations indefinitely? Others: 1 2 Q31 Did you strive to keep the customers satisfied? If yes, how? If no, Why? Yes No Use back pages for more notes, use question number for reference Page 4 93 Information seeking: Systematic planning and monitoring: Persuasion and networking: Savings group: Q32 Did you obtain credit for your business? If yes, do you repay them on time? _________________ If no, why? Yes No 1 2 Q33 Who do you consult to improve your business? Yes No 1 2 Q34 Do you keep records of your sale? Why or why not? Yes No Q35 Have you established new contacts (e.g. customers, suppliers, competitors) in your microbusiness after the Transform? Q36 Are you still a member of a savings group up to this day? If yes, what made you stay? If no, why did you stop participating in the savings group? If no, proceed to Q38 Q37 How much is your total savings as of this time? Q38 Did you experience withdrawing partially/fully your savings? ___Yes __No If yes, for what reasons? Use back pages for more notes, use question number for reference Page 5 94 Mobile banking: Health: Q39 What are the problems you encountered as a member or officer of a savings group? Q40 Is there anything you want to change in the management of the savings group? Recommendations? Q41 Are you a participant of Prevail? ___Yes When did you become a participant? _______________________ ___No Q42 Did you participate in the mobile banking service?  If Yes, are you still a member as of this time? ___Yes ___No. Why not? __________________________  If not, why did you not participate? Q43 What benefits are you getting from mobile banking service? Q44 Are you planning to continue using the mobile banking service in the future? Why or why not? 1 2 Q45 Were there sicknesses in the family in the last six months? If yes, what were these sicknesses? Yes No Q46 Are there still malnourished children in the household in the past 6 months? If yes, why? Yes No Use back pages for more notes, use question number for reference Page 6 95 At this point, please inform the participant to proceed to the designated area for the start of the focus group discussion and serving of snacks. GUIDE SHEET FOR QUESTIONS WITH RANKING Q16: Which among these areas did CEAIU help you the most? Kindly rank the choices from 1 to 6 (1-highest, 6-lowest)  Income  Health  Relationship with others  Technical skills (e.g. puto making, detergent making, etc.)  Faith/Values  Others (specify) Q18: For graduates: What motivated you to attend the weekly training and finish it until the end? Kindly rank the choices from 1 to 5 (1-highest, 5-lowest)  Food packs good for 1 week  Skills to be learned from the program  Opportunities to mingle with others  Make idle time productive  Others (specify) Q47 Do you still practice proper hygiene and sanitation measures taught to you by Transform trainers up to this day? Yes No Q48 Did you share these practices to other members of your family? Yes No Q49 Did you visit the RHU or hospital for medical assistance in the past six months? If yes, how often? _____________ Yes No Use back pages for more notes, use question number for reference Page 7 96 For dropouts: A1 ( ) Actual respondent ( ) Replacement A2 Survey successful? ( ) Yes ( ) No, reason: Q01: Names of adults in the household that earn income Q02: Type of work or business (use one row for each type of work or business) Q03: Daily/ Monthly income in the past month from work or business Use additional sheets if necessary Date: Time: Respondent: ( ) Participant ( ) Dropout ( ) Non-participant ( ) Counselor/Volunteer ( ) Pastor ( ) ICM Local Office staff ( ) Barangay Chairman/Councilor ( ) Barangay Health Worker ( ) SURGE Local Office Staff Name: Mobile no.: Batch: ( ) A ( ) B Community ID: Participant ID: Town/City: Geographical type: Q04 How many weeks did you attend the Transform training before you dropped out? Q05 Why did you drop out? (Can cite multiple reasons) Enumerator’s name: Use back pages for more notes, use question number for reference Page 1 97 Q06 Why did you not like the opportunity to try out the BiBs? Mobile banking? Savings group? Q07 What could have motivated you to continue attending the Transform training? Q08 Would you have wished to continue with the program? Why or why not? Q09 Were you able to start your own microbusiness even when you dropped out of Transform training? What were these businesses? Q10 Would you be willing to participate in a similar program in the future? Why or why not? Use back pages for more notes, use question number for reference Page 2 98 For non-participants: A1 ( ) Actual respondent ( ) Replacement A2 Survey successful? ( ) Yes ( ) No, reason: Q01: Names of adults in the household that earn income Q02: Type of work or business (use one row for each type of work or business) Q03: Daily/ Monthly income in the past month from work or business Use additional sheets if necessary Date: Time: Respondent: ( ) Participant ( ) Dropout ( ) Non-participant ( ) Counselor/Volunteer ( ) Pastor ( ) ICM Local Office staff ( ) Barangay Chairman/Councilor ( ) Barangay Health Worker ( ) SURGE Local Office Staff Name: Mobile no.: Batch: ( ) A ( ) B Community ID: Town/City: Geographical type: 1 2 Q04 Did the pastor of this community communicate with you regarding the possibility of being a participant of the project? Yes No Q05 Even if you were not a participant, what do you know about the project? Enumerator’s name: Use back pages for more notes, use question number for reference Page 1 99 Q06 Are you willing to join the project given the chance? Why or why not? Q07 Were you able to notice changes in the lives of participants of the project? What did you notice? Q08 Do you have a business? ____Yes ____No If yes, what is this enterprise and when did you start it? How did you start your microenterprise? Q09 Do you think your present condition is better or worse than the participants in terms of: Household Income: ______Better _____Worse ______Just the same (Explain your answer.) Health status of family members: ______Better _____Worse ______Just the same (Explain your answer.) Use back pages for more notes, use question number for reference Page 2 100 For pastors: Date: Time: Respondent: ( ) Participant ( ) Dropout ( ) Non-participant ( ) Counselor/Volunteer ( ) Pastor ( ) ICM Local Office staff ( ) Barangay Chairman/Councilor ( ) Barangay Health Worker ( ) SURGE Local Office Staff Name: Mobile no.: Batch: ( ) A ( ) B Community ID: Participant ID: Town/City: Geographical type: A1 ( ) Actual respondent ( ) Replacement A2 Survey successful? ( ) Yes ( ) No, reason: Q01 What were your main tasks and responsibilities in the Project? Q02 What convinced you to apply for the Transform program? How were you able to determine that your community was ready to apply? How was it like to lead/implement the Transform program and why? 1. Very difficult 2. Difficult 3.Neutral 4. Easy 5. Very Easy -------------------------------------------------------------------------- ----------------------- Q03 How did you choose the nominated ultrapoor participants in your community? Performed house visits before nominating them Asked the neighborhood who wants to be participants Own judgment Others: ________ Q04 How did you choose your 6 counselors? Did they apply for the position? Use back pages for more notes, use question number for reference Page 1 101 Q05 What are your observations of the behavior of the participants? Are they cooperative and attentive? Q06 Were there participants who were beneficiaries of other development programs? 4Ps? SPECIFY______________ Q07 What are the important challenges and problems you faced during program implementation? Q08 What do you think were the strategies of the Transform program that worked well and why?  Selection of participants  Identification of enterprises to be introduced to participants  Length and manner of delivery of the training  Amount of assistance (e.g. BiB, food packs) provided to participants  Monitoring of participants’ progress  Sustainability of results  Others Q09 What do you think are the strategies of the Transform program that did not work well? Why?  Selection of participants  Identification of enterprises to be introduced to participants  Length and manner of delivery of the training  Amount of assistance (e.g. BiB, food packs) provided to participants  Monitoring of participants’ progress  Sustainability of results  Others Q10 What is Prevail and how can a participant proceed from Transform to Prevail? How many participants did proceed to Prevail per batch? Q11 What did you do for the community in the last 9 months (Batch A) or 6 months (Batch B) after Transform? Follow through? Continuously kept in touch with the participants? Q12 Looking back over the last month, what do you think was the most significant change in the quality of the people’s lives in this community? Can this change be attributed to CEAIU? Use back pages for more notes, use question number for reference Page 2 102 Q13 Do you think that the outputs of Transform will be sustainable in your community and replicable in other areas? How? Q14 How are M&E data regularly collected, validated, and reported to headquarters? And what were the major problems encountered, if any, in collecting and reporting data and information? Q15 Were you able to meet your targets for the community? Q16 What were the major reasons for some participants to  drop out of the program?  not purchase a BiB?  not repay the loans?  not join the savings group?  leave the savings group?  not enroll in the mobile banking facility? Q17 Are you aware of the USAID – Cities Development Initiative (CDI)? IF yes, how do you think the CEAIU support the CDI? Q18 What are your suggestions in further improving the program? Use back pages for more notes, use question number for reference Page 3 103 For Counselors: Date: Time: Respondent: ( ) Participant ( ) Dropout ( ) Non-participant ( ) Counselor/Volunteer ( ) Pastor ( ) ICM Local Office staff ( ) Barangay Chairman/Councilor ( ) Barangay Health Worker ( ) SURGE Local Office Staff Name: Mobile no.: Batch: ( ) A ( ) B Community ID: Participant ID: Town/City: Geographical type: A1 ( ) Actual respondent ( ) Replacement A2 Survey successful? ( ) Yes ( ) No, reason: General Questions: Q01 What were your main tasks and responsibilities in the Project? Q02 How was it like to be part of the Project and why? 1. Very difficult 2. Difficult 3.Neutral 4. Easy 5. Very Easy -------------------------------------------------------------------------- Q03 What are your observations of the behavior of the participants? Are they cooperative and attentive? Q04 Were there participants who were beneficiaries of other development programs? 4Ps? SPECIFY______________ Q05 What are the important challenges and problems you faced during program implementation? Q06 What do you think were the strategies of the Transform program that worked well and why?  Selection of participants  Identification of enterprises to be introduced to participants  Length and manner of delivery of the training  Amount of assistance (e.g. BiB, food packs) provided to participants  Monitoring of participants’ progress  Sustainability of results  Others Use back pages for more notes, use question number for reference Page 1 104 Q07 What do you think are the strategies of the Transform program that did not work well? Why?  Selection of participants  Identification of enterprises to be introduced to participants  Length and manner of delivery of the training  Amount of assistance (e.g. BiB, food packs) provided to participants  Monitoring of participants’ progress  Sustainability of results  Others Q08 What is Prevail and how can a participant proceed from Transform to Prevail? How many participants did proceed to Prevail per batch? Q09 What did you do for the community in the last 9 months (Batch A) or 6 months (Batch B) after Transform? Follow through? Continuously kept in touch with the participants? Q10 Looking back over the last month, what do you think was the most significant change in the quality of the people’s lives in this community? Can this change be attributed to CEAIU? Q11 Do you think that the outputs of Transform will be sustainable in your community and replicable in other areas? How? Q12 How are M&E data regularly collected, validated, and reported to headquarters? And what were the major problems encountered, if any, in collecting and reporting data and information? Q13 Were you able to meet your targets for the community? Q14 What were the major reasons for some participants to  drop out of the program?  not purchase a BiB?  not repay the loans?  not join the savings group?  leave the savings group?  not enroll in the mobile banking facility? Use back pages for more notes, use question number for reference Page 2 105 More specific questions: Q15 What are your suggestions in further improving the program? Q16 Would you be willing to be part of a similar project in the future? Q17 In your opinion, do you think that the food pack good for 1 week was a major motivating factor on why they attended the training? Q18 These food packs were composed of what? Why good for 1 week not 1 day? Q19 What is the system in terms of repayment of BiBs? How much did you charge them per day/per week? For how many days/weeks? Q20 Do you still monitor the performance of savings groups after CEAIU? Q21 Did monitoring of health stop after CEAIU? Q22 How is mobile banking managed after CEAIU? Use back pages for more notes, use question number for reference Page 3 106 For Barangay Chairman: Date: Time: Respondent: ( ) Participant ( ) Dropout ( ) Non-participant ( ) Counselor/Volunteer ( ) Pastor ( ) ICM Local Office staff ( ) Barangay Chairman/Councilor ( ) Barangay Health Worker ( ) SURGE Local Office Staff Name: Mobile no.: Batch: ( ) A ( ) B Community ID: Town/City: Geographical type: A1 ( ) Actual respondent ( ) Replacement A2 Survey successful? ( ) Yes ( ) No, reason: Q02 Did the CEAIU coordinate with your office in implementing the activities? If yes, how often, and what sort of coordination or support did they request? Q03 What feedback did you receive from your constituents regarding the project? Have you witnessed significant changes in the well￾being of participants? Q04 What are the strong points of the project in terms of helping your community? Q01 Describe how you got to know about the CEAIU project. 1. Formal written notice from CEAIU 2. Informal verbal advice from CEAIU 3. From participants 4. From other non-participant residents 5. From other sources (Specify)__________________________ ---------------------------------------------------------------------------------------- Put other details/description here: Use back pages for more notes, use question number for reference Page 1 107 Q05 What areas do you think need improvement? Q06 What can you recommend to ICM in terms of its implementation of CEAIU? Q07 What other similar programs/projects from other GOs and NGOs have been conducted in your community? Q08 What do you think are the challenges if the government would hold a regular activity like the CEAIU project in your community? Q09 What specific assistance do you think is most needed by the ultrapoor in your community? Q10 Do you think the project has been a success? Why or why not? Use back pages for more notes, use question number for reference Page 2 108 For Barangay Health Worker: Date: Time: Respondent: ( ) Participant ( ) Dropout ( ) Non-participant ( ) Counselor/Volunteer ( ) Pastor ( ) ICM Local Office staff ( ) Barangay Chairman/Councilor ( ) Barangay Health Worker ( ) SURGE Local Office Staff Name: Mobile no.: Batch: ( ) A ( ) B Community ID: Town/City: Geographical type: A1 ( ) Actual respondent ( ) Replacement A2 Survey successful? ( ) Yes ( ) No, reason: Q02 Did the CEAIU coordinate with your office in implementing the activities? If yes, how often, and what sort of coordination or support did they request? Q03 What feedback did you receive from your constituents regarding the project? Have you witnessed significant changes in the well￾being of participants? Q04 What are the strong points of the project in terms of helping your community? Q01 Describe how you got to know about the CEAIU project. 1. Formal written notice from CEAIU 2. Informal verbal advice from CEAIU 3. From participants 4. From other non-participant residents 5. From other sources (Specify)__________________________ ------------------------------------------------------------------------------------ Put other details/description here: Use back pages for more notes, use question number for reference Page 1 109 Q05 What areas do you think need improvement? Q06 What can you recommend to ICM in terms of its implementation of CEAIU? Q07 What other similar programs/projects from other GOs and NGOs have been conducted in your community? Q08 As a staff of the barangay health center, how do you monitor the health of the members of your community? Q09 What do you think are the health programs you need in this community? Use back pages for more notes, use question number for reference Page 2 110 Q10 Have you experienced an increase in the number of visits / consultations in barangay health center because of CEAIU? Q11 What do you think are the challenges if the government would hold a regular activity like the CEAIU activity in your community? Use back pages for more notes, use question number for reference Page 3 111 For ICM Local Staff: Date: Time: Respondent: ( ) Participant ( ) Dropout ( ) Non-participant ( ) Counselor/Volunteer ( ) Pastor ( ) ICM Local Office staff ( ) Barangay Chairman/Councilor ( ) Barangay Health Worker ( ) SURGE Local Office Staff Name: Mobile no.: A1 ( ) Actual respondent ( ) Replacement A2 Survey successful? ( ) Yes ( ) No, reason: Q01 What were your main tasks and responsibilities in the Project? Q02 Was the project able to adequately select the target ultrapoor households in the communities? Explain. Q03 To what extent was the project able to respond to their identified needs? Q04 Did the project operate in the way it was designed? Or were there modifications along the way? Please elaborate Q05 Briefly describe the project management structure. Has it been effective? Why or why not? Q06 Has the project always delivered its milestone targets on time? On budget? Q07 How was the project able to deliver its targets? Explain ___As expected ___Beyond expectation ___Below expectation Q08 Do you think that the project’s expected outcomes in terms of livelihood, health, and values are occurring or have occurred? Q09 Did gender play in the implementation and results of the program? Why so? Use back pages for more notes, use question number for reference Page 1 112 Q10 What do you think were the strategies of the Transform program that worked well and why?  Selection of participants  Identification of enterprises to be introduced to participants  Length and manner of delivery of the training  Amount of assistance (e.g. BiB, food packs) provided to participants  Monitoring of participants’ progress  Sustainability of results  Others Q11 What do you think are the strategies of the Transform program that did not work well? Why?  Selection of participants  Identification of enterprises to be introduced to participants  Length and manner of delivery of the training  Amount of assistance (e.g. BiB, food packs) provided to participants  Monitoring of participants’ progress  Sustainability of results  Others Q12 Do you think that the results of the project will be sustainable in your community and replicable in other areas? How? Q13 How are M&E data regularly collected, validated, and reported to headquarters? And what were the major problems encountered, if any, in collecting and reporting data and information? Q14 How could implementation be improved? Q15 What lessons can be identified that apply to similar future projects? Q16 Are you aware of the USAID – Cities Development Initiative (CDI)? If yes, how do you think the CEAIU support the CDI? Use back pages for more notes, use question number for reference Page 2 113 For SURGE Local Office Staff: Date: Time: Respondent: ( ) Participant ( ) Dropout ( ) Non-participant ( ) Counselor/Volunteer ( ) Pastor ( ) ICM Local Office staff ( ) Barangay Chairman/Councilor ( ) Barangay Health Worker ( ) SURGE Local Office Staff Name: Mobile no.: A1 ( ) Actual respondent ( ) Replacement A2 Survey successful? ( ) Yes ( ) No, reason: Q0 Explain the SURGE and how it supports the CDI objective  c/o Briefing Q01 How aware are you of the CEAIU and its strategies? Q02 How do you think the CEAIU specifically links with CDI? (Be specific and cite examples: use any or all of the sub-questions below for follow-up) How do you think do the CDI and CEAIU complement each other? How do you think do the CDI and CEAIU substitute/duplicate each other’s activities and strategies? How do you think do the CDI and CEAIU contradict each other? What specific accomplishments of SURGE in the past year does the CEAIU support? Q03 Are you aware of projects similar to CEAIU that are specifically implemented under the CDI? How about future similar projects? Kindly describe them. Q04 Are there specific SURGE activities related to livelihood and health for the ultrapoor? Use back pages for more notes, use question number for reference Page 1 114 FOCUS GROUP DISCUSSION GUIDE (All questions and instructions must be translated in the local dialect) Ground Rules: 1. Only participants and survey team will be in the room. There should be no ICM / Transform / Barangay officials present. 2. Documentation through written notes, audio recording, and photographs will be done. 3. Brief and clear answers are requested. Maximum of two hours for the focus group will be allotted. 4. All participants will be given opportunity to speak. Moderator: Good morning / afternoon. Thank you for taking the time to join our discussion about the CEAIU Activity. My name is ___________ and I will serve as the moderator for today’s discussion. With me are _________. We are from UPLB, contracted by USAID to make the evaluation of the CEAIU Activity. Assisting us are enumerators and interpreters from UP Visayas (Iloilo Campus). The purpose of today’s discussion is to get information from you about the results of the Activity and finding ways to possibly improve it. You were invited because you became participants during ____ (Batch A: Oct 2015–Jan 2016 / Batch B: Feb-May 2016). We have asked your names during the mini-survey, but your individual answers will be treated confidentially and your names, if ever not required by the contractee (USAID), will not appear in the reports. (We will only have summaries of group answers in the reports.) There are no right or wrong answers to the questions we are about to ask. What is important is you share your thoughts about the activity. In fact, we expect that you will have different answers and points of view in some questions, and similarities in others. Please feel free to share your answers even if it differs from what others have said. You can always follow up on something that someone has said, and agree, disagree, or give an example. MAIN QUESTIONS: (Short follow-up questions should be readied when participants cannot give direct answers) 1. Tell us again briefly your names and how you joined the activity. 2. During the interview earlier, you were asked about the most significant change that happened in your life, including your family’s life, because of the Activity. Can you discuss them again in the group?  During the past months long after the activity has finished, how did the activity help you in your life? (Specifically, in terms of income, health, welfare, capacity built, faith, etc.)  What did you learn from the Transform program? 3. What about the activity that you found impressive and most helpful?  What motivated you to attend the weekly training and finish it until the end? 4. One of the goals of the activity is to help you become entrepreneurs and get the feel of starting a small business. (At this point, ask the participants to raise their hands if they were able to purchase and repay a BiB, and if they continue to have a microbusiness to this day). 115  Because of the activity, can you explain how ready are you to continue on or expand your microbusiness?  What concerns you most about running your own microbusiness after the activity ended? 5. What specific aspect of the activity did you find most difficult? 6. What are your suggestions to improve the activity? 7. Why do you think majority of the participants are women?  Are your husbands aware that you are attending the Transform program? 8. How did you feel when the activity was ending? 9. Would you be willing to participate in a similar program in the future? Why or why not? 116 ANNEX IV: SOURCES OF INFORMATION Sites visited Date Time Town/ City/ Province Barangay/Sitio/ District Address Geographical Area Batch Community ID 9-Jan-17 AM Iloilo City Block 9, Barangay Sinikway, Lapuz urban slum A 150812-14 PM Iloilo City New Site, Barangay Mansaya, Lapuz urban slum B 150813-16 10-Jan-17 AM Lambunao Barangay Jayubo rural plain A 150812-25 PM Lambunao Barangay Supoc rural plain B 150813-06 11-Jan-17 AM Passi City Salvacion Street, Barangay Poblacion Ilaya rural plain A 150812-08 PM Passi City Sitio Arimayan, Barangay Salngan rural plain B 150813-27 12-Jan-17 AM Dueñas Barangay Capuling rural mountain A 150812-29 PM Iloilo City Rizal, Estanzuela coastal A 150812-13 13-Jan-17 AM Leon Sitio Dugo, Poblacion rural mountain B 150813-04 PM Iloilo City Coastal View, Barangay Balabago, Jaro coastal B 150813-20 PM Iolilo City Interview with ICM Local Implementers - La Paz 14-Jan-17 (Departure from Iloilo) PM Laguna Interview with SURGE Personnel – Sta. Rosa City (originally scheduled in Iloilo City) 117 Respondents Interviewed By Community Date Time Site Visited (Community ID) Number and Class of Respondents Number of Other FGD participants 9-Jan-17 AM Block 9, Barangay Sinikway, Lapuz, Iloilo City (150812-14) 7 participants 0 4 dropouts 3 non-participants 1 pastor 2 counselors 1 barangay chair 1 barangay health worker 1 barangay councilor PM New Site, Barangay Mansaya, Lapuz, Iloilo City (150813-16) 6 participants 0 2 dropouts 3 non-participants 1 pastor 1 counselor 1 barangay councilor 10-Jan-17 AM Barangay Jayubo, Lambunao, Iloilo (150812-25) 6 participants 5 3 dropouts 1 non-participant 1 pastor 4 counselors 1 barangay chair 1 barangay health worker PM Barangay Supoc, Lambunao, Iloilo (150813-06) 7 participants 0 2 dropouts 3 non-participants 1 pastor 1 counselor 1 barangay councilor 11-Jan-17 AM Salvacion St., Barangay Poblacion Ilaya, Passi City, Iloilo (150812-08) 8 participants 4 2 dropouts 1 pastor 5 counselors PM Sitio Arimayan, Barangay Salngan, Passi City, Iloilo (150813-27) 7 participants 3 non-participants 3 1 pastor 1 counselor 118 Date Time Site Visited (Community ID) Number and Class of Respondents Number of Other FGD participants 12-Jan-17 AM Barangay Capuling, Dueñas, Iloilo (150812-29) 6 participants 3 3 dropouts 2 non-participants 1 daughter of pastor 4 counselors 1 barangay chair 1 barangay health worker PM Barangay Rizal Estanzuela, Iloilo City (150812-13) 6 participants 3 1 dropout 1 non-participant 1 pastor 1 counselor 1 barangay health worker 13-Jan-17 AM Sitio Dugo, Barangay Poblacion, Leon, Iloilo (150813-04) 7 participants 3 1 pastor 1 counselor 1 barangay chair 1 barangay health worker PM Coastal View, Barangay Balabago, Jaro, Iloilo City (150813-20) 7 participants 2 2 dropouts 3 non-participants 1 pastor 1 counselor 1 barangay councilor 1 barangay health worker 119 Documents Reviewed International Care Ministries.n.d. Attachment 2 of the CEIAU Grant Award No. AID-492-F-15-00003. International Care Ministries.2017.Catalyzing Entrepreneurship Among Iloilo’s Ultrapoor Final Report. International Care Ministries.2016.Catalyzing Entrepreneurship Among Iloilo’s Ultrapoor Milestone #3 Report. International Care Ministries.2016.Catalyzing Entrepreneurship Among Iloilo’s Ultrapoor Milestone #4 Report. International Care Ministries.2014.Catalyzing Entrepreneurship Among Iloilo’s Ultrapoor Monitoring, Evaluation and Learning (M,E,&L) Plan. International Care Ministries.2015.Catalyzing Entrepreneurship Among Iloilo’s Ultrapoor Submitted Proposal. International Care Ministries.n.d.Livelihood Curriculum. International Care Ministries.n.d.Metrics Survey Form v3.8 Updated. International Care Ministries.2015.Transform and Jumpstart Health Trainer’s Manual. International Care Ministries Foundation, Inc.2017.Iloilo Expenses with Nutripack Consumption. International Care Ministries Foundation, Inc.2017.Iloilo Expenses without Nutripack Consumption. United States Agency for International Development-Philippines.2015. Catalyzing Entrepreneurship among Iloilo’s Ultrapoor AID-492-F-15-00003 Grant Agreement. United States Agency for International Development-Philippines.2014. Cities Development Initiative Logical Framework. United States Agency for International Development-Philippines.2017. Cities Development Initiative – Promoting Cities as Engines of Growth.Retrieved from: https://www.usaid.gov/philippines/partnership-growth-pfg/cdi 120 Database Reviewed Note: All database were provided by International Care Ministries Foundation, Inc 1. Pre- and post-survey database 2. Master list areas_GM2 Iloilo 3. BiB Matching Tool 4. Microsavings database 5. Policy Making 1 Monitoring: Savings and Meetings 6. SG Attendance Monitoring (Weeks 5-16) 7. Trainers’ kit inventory (Batches A and B) 8. Livelihood Tracking Booklets (Batches A and B) 9. 2015b3_bib_participant_data (Batch B only) 10. 2015b3_bib_participants_payments (Batch B only) 11. BiB Batch B summary from Project Tomorrow and BER report from LL (Batch B only) 12. Challenges (Batch B only) 13. Savings Summary (Batch B only) 14. Iloilo BiB Community Data (Batch B only) 15. Iloilo Community Data (Batch A only) 16. Health database: a) Iloilo 2015-2016 Diarrhea and Dehydration b) Iloilo 2015-2016 Home Based Feeding (HBF) Report Card c) Iloilo 2015-2016 Weighing Day 2 Logbook d) Iloilo 2015-2016 Pegnancy and Newborn Report Card e) Iloilo 2015-2016 Skin Disease TP f) Iloilo 2015-2016 g) Iloilo Health Trainers’ Logbook h) Iloilo TB report card i) Iloilo Transform Health Screening Logbook 17. End of Batch Report (Batches A and B) 18. Monthly Microsavings Performance Report 19. Weekly Livelihood Updates and Comments (Batches A and B) 20. Weekly Report together with monthly report and overall summary 21. Pre-batch training feedbacks (Batch B only) 121 ANNEX V: DISCLOSURE OF ANY CONFLICTS OF INTEREST Name Richard B. Daite Title Assistant Professor Organization University of the Philippines Los Baños Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) AID-492-O-16-00111 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) “Catalyzing Entrepreneurship Among Iloilo’s Ultrapoor” implemented by the International Care Ministries Foundation, Inc. I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date X X 122 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-4521 www.usaid.gov