February 11, 2012 This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by Management Systems International (MSI) under the Independent Monitoring and Evaluation Contract (IMEC). EVALUATION Women’s Economic Empowerment: Balochistan Evaluation Report USAID WOMEN’S ECONOMIC EMPOWERMENT: BALOCHISTAN EVALUATION REPORT FEBRUARY 11, 2012 This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by Management Systems International (MSI) under the Independent Monitoring and Evaluation Contract (IMEC). WOMEN’S ECONOMIC EMPOWERMENT: BALOCHISTAN EVALUATION REPORT Contracted Under No. GS-23F-8012H and Order No. AID-391-M-11-00001 Independent Monitoring and Evaluation Contract (IMEC) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT i TABLE OF CONTENTS Introduction .................................................................................................................................... 1 Sector Context ..................................................................................................................... 2 Overview of the Embellished Garments Sector 3 The Development Problem and USAID’s Response ................................................................... 4 USAID’s Intervention in Response .................................................................................... 4 Project Objectives, Components and Outputs 5 Approach to Marketing 6 Monitoring and Evaluation 6 Purpose of the Evaluation ............................................................................................................. 6 Evaluation Questions .......................................................................................................... 7 Evaluation Methodology ............................................................................................................... 8 Findings and Conclusions ............................................................................................................ 10 Findings and Conclusions on Relevance .......................................................................... 10 Findings and Conclusions on Effectiveness ..................................................................... 12 Findings and Conclusions on Efficiency .......................................................................... 19 Findings and Conclusions on Impact ................................................................................ 22 Findings and Conclusions on Sustainability ..................................................................... 24 Findings and Conclusions on Replication ......................................................................... 25 Findings and Conclusions on Gender ............................................................................... 26 Findings and Conclusions on Reporting ........................................................................... 27 Findings and Conclusions on Public Relations/Media Coverage ..................................... 28 Findings and Conclusions on Coordination ...................................................................... 29 Recommendations ........................................................................................................................ 29 bibliography ................................................................................................................................. 31 Annexes ......................................................................................................................................... 33 Annex 1: Complete List of Products Produced by WEGs ................................................ 33 Annex 2: List of Project Documents Reviewed for the Evaluation .................................. 34 Annex 3: Survey Questionnaire ........................................................................................ 35 Annex 4: Questions for FSA Group Discussion ............................................................... 48 Annex 5: Questions for Shop Keepers .............................................................................. 49 Annex 6: Description of Selected WEE:B Products ......................................................... 50 Annex 7: WEE:B Budget and Expenditure as of 30th June, 2011 ..................................... 52 Annex 8: Targets Versus Achievements as of 30th December 2011. ............................... 53 Annex 9: Change in Production, Before and After the Project ......................................... 59 Annex 10: Email from FAO Regarding WEE:B’s Reporting and Branding Performance ...................................................................................................................... 61 Annex 11: Email from FAO Regarding Coordination Between WEE:B and US￾ABBA ............................................................................................................................... 63 WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT ii LIST OF TABLES AND FIGURES Table 1: Women’s Economic Empowerment: Balochistan Project Summary ............................... iii Table 2: Location and Number of interviews .................................................................................. 9 Table 3: Reasons for Increased Income ......................................................................................... 14 Table 4: Sources of Embroidery Inputs ......................................................................................... 18 Table 5: Overview of Project Budget and Expenditure ................................................................. 19 Table 6: Monetary Benefits by Year .............................................................................................. 20 Table 7: Project Financial Analysis Calculations .......................................................................... 20 Table 8: Summary of Efficiency Measures .................................................................................... 21 Table 9: Change in Income Attributable to the Project.................................................................. 22 Table 10: Gender Breakup of Project Staff .................................................................................... 26 Table 11: Project Budget and Expenses ......................................................................................... 52 Table 12 : Change in Production by Item ...................................................................................... 59 Figure 1: Map of Project Districts .................................................................................................. iv Figure 2: Trends in Female Labor Force Participation, 2001-2011 ................................................. 1 Figure 3 : GENDER Gap in Labor Force Participation by Province, 2010-11 ................................ 2 Figure 4 : Literacy Rates (10 years and older) by Province, 2006-07 ............................................. 2 Figure 5: Effectiveness of Project-Supported Training ................................................................. 13 Figure 6: Average Number of Items Produced Annually .............................................................. 16 Figure 7: Percentage of Women Producing Each Item .................................................................. 16 Figure 8: Use of Embroidery Income ............................................................................................ 23 Figure 9: Examples of Branding .................................................................................................... 27 Figure 10 : Participants’ Beliefs About Project Funding ............................................................... 28 WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT iii PROJECT SUMMARY The Women’s Economic Empowerment: Balochistan (WEE:B) project is designed “to increase incomes and economically empower women producers in northern Balochistan, by facilitating their access to viable markets with appropriate products.” Table 1 summarizes basic project details, while Figure 1 identifies the target districts and locations of project-supported Women’s Embellisher Groups (WEGs) on the map of Pakistan. TABLE 1: WOMEN’S ECONOMIC EMPOWERMENT: BALOCHISTAN PROJECT SUMMARY USAID objectives addressed The project contributes to USAID’s Assistance Objective (AO) 2: Improved conditions for broad-based economic growth. It also contributes to Intermediate Result (IR) 1: Increased income generation opportunities, and cross-cutting objective IR 2: Increased gender equity. Implementing partners Primary recipient: Food and Agriculture Organization of the United Nations (FAO) Sub-awardee: Mennonite Economic Development Associates (MEDA) Key Facilitating Partner: Water, Environment and Sanitation Society (WESS) Cooperative agreement (or other term as applicable) Cooperative agreement # 391-A-00-09-01122-0 Project dates October 2009 to December 2011. No-cost extension until September 2012. Project budget USD 1.1 million Project location Districts of Quetta, Mastung, Killa Saifullah, Loralai and Zhob in Balochistan Province, Pakistan WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT iv FIGURE 1: MAP OF PROJECT DISTRICTS WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT v ACRONYMS AHAN Aik Hunar Aik Nagar AO Assistance Objective BDS Business Development Services BRSP Balochistan Rural Support Programme CBSGs Community Based Saving Groups CEO Chief Executive Officer CO Community Organizations CRS Catholic Relief Services DACAAR Danish Committee for Aid to Afghan Refugees ECDI Enterprise and Career Development Institute ECI Empowerment through Creative Integration FAO Food and Agriculture Organization of the United Nations FBS Federal Board of Statistics FC Frontier Corps FGD Focus Group Discussion FSA Female Sales Agent GoP Government of Pakistan G2A Getting to Answer HBW Home Based Workers HDI Human Development Index IDO Innovative Development Organization ILO International Labor Organization IMEC Independent Monitoring and Evaluation Contract IR Intermediate Result IRR Internal Rate of Return KADO Karakoram Area Development Organization KFP Key Facilitating Partner KIs Key Informants KP Khyber Pakhtunkhwa LFS Labor Force Survey LOP Life of Project MEDA Mennonite Economic Development Associates MSI Management System International MTE Mid-Term Evaluation NGO Non-Governmental Organization NPV Net Present Value PAK Pakistan PKR Pakistani Rupees PMF Performance Measurement Framework SABAH SAARC Business Association of Home Based Workers SAARC South Asian Association for Regional Cooperation SDPI Sustainable Development Policy Institute SMEDA Small and Medium Enterprise Development Agency SOW Statement of Work SPDC Social Policy Development Center SPSS Statistical Package for The Social Sciences TGP Towards Gender Parity in Pakistan TVO Trust for Voluntary Organization WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT vi UNICEF United Nation International Children’s Emergency Funds US United States US ABBA United States Assistance to Balochistan Border Areas USAID United States Agency for International Development USD United States Dollar USG United State Government VC Value Chain WCC Women Chamber of Commerce WEs Women Embellishers WEE:B Women’s Economic Empowerment: Balochistan WEGs Women Embellishment Groups WESS Water, Environment and Sanitation Society WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT vii EXECUTIVE SUMMARY Balochistan is the least developed province of Pakistan. Besides a higher incidence of poverty than other provinces, Balochistan also exhibits a higher level of gender disparity due to lack of education and conservative social norms. According to the Labor Force Survey 2010-11, Balochistan’s female labor force participation was at six percent. This is much lower than the national average of 15 percent or that of Punjab at 20 percent. Most women lack mobility and are largely homebound. Even women who are involved in agricultural activities serve as unpaid family workers and their ownership of important assets or land appears to be extremely limited. While homebound, the women of Balochistan inherit a rich tradition of embroidery. The quality of the embroidery is borne out by its demand from within the province as well as from other parts of the country. Mennonite Economic Development Associates (MEDA) manages the WEE:B project under a 27-month (October 2009 to December 2011), USD 1.1 million sub-award from the Food and Agriculture Organization’s (FAO) US Assistance to Balochistan Border Areas (USABBA) project. MEDA implements activities in Balochistan through its local partner, the Water, Environment, and Sanitation Society (WESS) with capacity building support from Empowerment through Creative Integration (ECI). The project contributes to USAID’s Assistance Objective (AO) 2: Improved conditions for broad-based economic growth and Intermediate Result (IR) 1: Increased income generation opportunities. It also contributes to the cross-cutting objective IR 2: Increased gender equity. Evaluation Methodology The evaluation used a variety of methods to collect data. These included reviewing relevant documents and secondary data; conducting a small-scale survey of project-assisted embroiderers and semi-structured interviews with Female Sales Agents (FSAs), shopkeepers, project staff, and other stakeholders; and visiting sites. The evaluation addressed questions of project relevance, effectiveness, efficiency, impact, sustainability, replication, gender, reporting, public relations/media coverage and coordination. Conclusions Key conclusions include: Relevance  Many elements of the project design are relevant to the needs of beneficiaries and appropriate in the context of empowering women in northern Balochistan. In particular, the project’s focus on the embellished product’s value chain is an appropriate approach to engaging women in economic activity in the project area. It capitalizes on indigenous skills and practices, serves a demonstrated market, and is well suited to work within the home. The approach of using mobile women as sales agents to facilitate quality improvement, to link women embellishers (WEs) to markets, and to communicate market demands back to WEs addresses real constraints that WEs in the region face when accessing markets. Consequently, many of the project-supported WEs have changed the way they were doing business. Given the generally low levels of human capacity in Balochistan, the project’s focus on building the capacity of local partners to carry on the project’s activities is essential.  The project has maintained its relevance to some extent by supporting activities (i.e., CBSGs) to address emerging needs of WEGs. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT viii Effectiveness  Project activities have been effective in increasing production of embellished items among project-assisted WEs and ensuring delivery of a high-quality product to markets. It is not possible with existing data to determine whether quality has increased as a result of the project.  The market linkages (both input and output) seem accepted and beneficial to WEs.  The project has struggled to some extent building the capacity of national level partner organizations/institutions due to the absence of a proper KFP for capacity building.  Given the overachievements of the targeted activities it seems that the original targets could have been set higher. Efficiency  By any measure (in the context of the caveats cited above) the project has used its resources efficiently. Impact  The WEE:B project has substantially affected the income women earn from embroidery. Project￾supported WEs have increased incomes by increasing production and quality, diversifying the range of products they produce in response to market demands and earning higher prices for what they do produce (by gaining access to higher value markets).  The project has built human capacity in the WEs and FSAs it has trained. Based on the large number of women who appear to have started embroidery work as a business as a result of the project, the human capacity benefits may be substantial.  Increased incomes from embroidery improve the welfare of women’s households since the women use earnings from embroidery largely for food and clothing. Sustainability  Project results have good prospects for sustainability. The tested approach leverages market incentives to encourage sustainability, builds the necessary skills and relationships among FSAs to promote sustainable results, and minimizes donor dependence.  The income potential – especially for FSAs – appears to provide a strong incentive for sustainability.  The factors that make the project approach relevant and necessary (i.e., strict social norms, demand for embroidered products, and the embroidery skills of the women of Balochistan) are not likely to change in the near future thus suggesting that the incentives that drive project results are likely to remain relevant. Replication  The approach is widely replicable – both in the specific example of the embroidered product value chain and more generally in other value chains. It is most easily replicated for embroidered WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT ix products in other areas of Pakistan where conservative norms and indigenous embroidery skills are prevalent.  While the WEE:B project was designed specifically to address limited market access due to immobility of WEs, the extension of the approach to other value chains by other implementers suggests that it is relevant when other factors limit producers’ access to markets. Gender  The project has directly benefited as many as 5,000 WEs and 200 FSAs, all of whom are women, by increasing income over which most have some level of control and by enhancing the respect they receive within the household and community. Reporting  The project has been reporting to FAO as required in an appropriate and timely manner. The project does not consistently brand public events in a manner that complies with USAID guidelines. It appears to brand appropriately when FAO must approve banners, but not as well when it does not receive FAO input on branding.  The unusually high frequency of reporting suggests that project staff spends a considerable amount of time on this activity. Public Relations/Media Coverage  The project has not done a particularly good job of communicating the role of USAID and the American people in funding project activities.  The project has not been particularly effective in communicating its message to local audiences. MEDA’s quarterly newsletter has a limited audience and does not serve to inform the target audience in Balochistan who need to hear about the beneficial influence of USAID on their communities. Coordination  The project has coordinated its training, capacity building, and marketing activities with a wide variety of organizations working in relevant sectors. Overall Assessment The main reason for the project’s achievement is the effectiveness of the Female Sales Agent model that works to leverage the skills of the beneficiaries without violating any social norms. Some of the other reasons for its success are as follows:  The project does not offer any direct financial support or in kind transfers to the beneficiaries thus allowing it to keep its costs in check.  MEDA has seven years of experience in implementing the FSA model in different parts of Pakistan and Afghanistan. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT x Key Recommendations 1. The survey showed that many of the respondents were unaware of the input costs as well as FSA commissions. This leaves these women vulnerable to exploitation from FSAs. The project needs to intervene to check whether the linkage between WE and the FSA is holding on as originally envisioned. Since monitoring all groups would not be feasible, random checks on embellisher groups would serve as an effective oversight tool. Project staff should ask a standard set of questions to ensure that the WEs are aware of the costs as well as payoffs associated to their work. 2. The project needs to enhance their media efforts. Given the precarious situation in Balochistan, a more thorough propagation of the WEE:B success story will contribute to disseminating the project’s message. Presently, the project staff is not experienced enough to deal with the local news media. WEE;B should employ an experienced person who could propagate the WEE:B story through local newspapers and radio. Furthermore, future media efforts need to collaborate more with USAID’s overall media efforts. 3. The project has been on the ground for only about two years and has only recently begun to enable producers to benefit from marketing linkages. Additional time is needed to enhance the gains from the project by deepening its impact and broadening its outreach, which suggests the need for an additional phase of the project. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 1 INTRODUCTION Balochistan is the largest of Pakistan’s four provinces, covering 44 percent of the country’s total land area, but contains only 7.1 million people – the equivalent of 5.1 percent of the national population. The province is largely arid and remote, and is the least developed of Pakistan’s four provinces with poverty rates above the national average. The poor quality of rural roads and tracks, and inadequate access to public services, safe drinking water, health services, and education are common in rural areas throughout the province. Balochistan has the lowest Human Development Index (HDI) value of all of Pakistan’s provinces and experienced the least growth in the HDI between 1998 and 2005. Furthermore, Balochistan contains seven of the eleven Pakistan districts with the lowest HDI scores (Jamal & Kahn, 2007) Agriculture is the largest economic sector in Balochistan accounting for over 65 percent of the gross provincial domestic product and employing 60-65 percent of the labor force1 . In Pakistan – and particularly in Balochistan – conservative social and family norms limit women’s mobility outside the home or local community. Immobility limits women’s access to economic opportunities, education, formal employment and participation in development. Even women who are involved in agricultural activities often serve as unpaid family workers and their ownership of important assets or land appears to be extremely limited (Sustainable Development Policy Institute). The World Bank identifies mobility as a key determinant of women’s empowerment and finds the norms of seclusion the ‘most important constraint’ in women’s access to services and participation in society and the economy (Sustainable Development Policy Institute). Consequently, Balochistan has the lowest rate of female participation in the formal labor force of any province in the country. Furthermore, women’s labor force participation in Balochistan has actually fallen in contrast to the national trend over the past decade (Figure 2) and in 2010-11 lagged well behind rates for men leaving Balochistan with the largest gender gap in labor force participation in Pakistan (Figure 3). Balochistan also has the lowest female literacy rate among Pakistan’s four provinces and lags well behind that of men (Figure 4). FIGURE 2: TRENDS IN FEMALE LABOR FORCE PARTICIPATION, 2001-2011 Source: Labor Force Survey, various years. 1 Finance Department, Government of Balochistan. 2010. White Paper on Budget. 0% 5% 10% 15% 20% 25% 30% 35% 2001‐02 2003‐04 2005‐06 2006‐07 2007‐08 2008‐09 2009‐10 2010‐11 Percentage (≥ 10 years) Pakistan Punjab Sindh NWFP Balochistan WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 2 FIGURE 3 : GENDER GAP IN LABOR FORCE PARTICIPATION BY PROVINCE, 2010-11 Source: Labor Force Survey, various years. FIGURE 4 : LITERACY RATES (10 YEARS AND OLDER) BY PROVINCE, 2006-07 Source: Sustainable Development Policy Institute (SDPI) Women who have independent income and earnings generally have greater decision-making power than women without independent earnings. However, those who earn by working from home for pay seem to be worse off than those who work outside the home for pay (SDPI, 2008). Amartya Sen (Sen, 1990) notes that “the division of a family’s joint benefits is less likely to be unfavorable to women if 1) they can earn an income, 2) their work is recognized as productive, 3) they own some economic resources and have some entitlements to fall back on and 4) there is an understanding of the ways in which women are deprived and a recognition of the possibilities of changing this situation. This last category can be influenced by education for women and by their participatory political action.” Sector Context Because cultural norms restrict women’s mobility and participation in the formal labor force, an estimated 45 percent of employed women in Balochistan work as home based workers (HBW) in the 0% 10% 20% 30% 40% 50% 60% 70% 80% Percentage (≥ 10 years) Male Female Gap 67% 67% 67% 58% 48% 42% 28% 22% 0% 10% 20% 30% 40% 50% 60% 70% 80% Punjab Sindh NWFP Balochistan Percentage of population Men Women WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 3 informal economy. The Pakistan Labor Force Survey definition of the informal sector includes household enterprises owned and operated by own-account workers regardless of the size of the enterprise and excludes all enterprises engaged in agricultural activities or involved in non-market production (GoP, 2008). The International Labor Organization (ILO) in 2009 concluded that “it is likely that the informal economy has become a possible fallback position for women who are excluded from paid employment. For many of them it is often the only source of income, especially in those areas where cultural norms restrict them from work outside home or where, because of conflict with household responsibilities, they cannot undertake regular employee working hours” (Government of Pakistan, 2009). In 2009, approximately 29 percent of the total HBWs in Pakistan were male and 71 percent were female. Similarly, the percentage of HBWs in Balochistan was estimated to be 55 percent men and 45 percent women (ILO, 2011). Some of the reasons that women turn to home-based work are low wages that carry high opportunity costs, lack of necessary qualification and training to get formal work, absence of child support, and social and cultural constraints (Sudarshan & Sinha, 2011). The findings of a Karachi study of home based workers limited response to higher wages available for equivalent work outside the home revealed the importance of strong social constraints to women’s work in the Pakistani society (Kazi & Raza, 1989). The overwhelming majority of women (80 percent) were not permitted by their families to take up outside employment. Income earning activities undertaken at home were far more acceptable as they did not violate social sanctions. Restrictions on mobility of such women leave them vulnerable to exploitation by middlemen as a source of cheap labor. They are a group who would be necessarily out of the purview of any labor legislation and are also likely to be least informed about the marketability of their products and competitiveness of the piece rates they get (Kazi & Raza, 1989). Overview of the Embellished Garments Sector In Pakistan, many women manufacture embroidered products since it builds on traditional skills and gives women the opportunity to work from the home without violating social norms. Balochistan has a rich tradition of embroidery and many women work in the embellished garment sector. In fact, a recent survey of 131 rural Balochistan households found that 42 percent reported income from embroidery, making it the single most important source of income earned by women by a wide margin (Food and Agriculture Organization, 2011). Balochistan’s embroidery is also unique. One estimate identified around 200 different types of stitches in the area surrounding Quetta (MEDA, 2009). Mai Pusht, Kodoro, Pokhtak, Zartar, Khamak are some of the different styles of embroidered embellishing that are unique in their aesthetic characteristics and native to Balochistan. The skills for creating these designs are usually the domain of those to whom these skills have been transferred through generations. Furthermore, the demand for embroidered embellishment has expanded beyond garments to include embroidered wall panels, caps for men, floor and sofa cushions, mobile covers, purses/small bags, pouches, embroidered strips for use on dresses, and photo frames (MEDA, 2011a). While the skill of embroidery has been passed from one generation to the next, the purchase of inputs constitutes a constraint given the poverty levels in Balochistan. For this reason, women embroiderers may rely on shopkeepers to provide inputs. This transaction, however, requires a certain level of trust between the embellisher and the shopkeeper. Several national and international development agencies have worked to support the embellished garments sector. These include Behbud, Save the Children (US), Danish Committee for Aid to Afghan Refugees (DACAAR), Karakoram Area Development Organization (KADO), Aik Hunar Aik Nagar (AHAN), SAARC Business Association Of Home Based Workers (SABAH) and Karwan. Most of these organizations sought to empower women economically by improving their skills and capacities and expanding their customer base by linking them to national and international markets. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 4 THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE Within the context of their limited mobility and education, economically empowering women in Balochistan presents a particular challenge. The traditional skills of making embellished garments – a skill from which many women already earn income – presents an opportunity that builds on existing skills and practices. However, homebound women face a number of barriers to full participation opportunities in the embellished garment sector. These include:  Limited access to markets – Because many women are immobile, they find it difficult to access markets outside their village or local area. This limitation creates dependence on others, often male family members, and may reduce the control women have over the money they earn from embroidery. Limited mobility also affects their ability to directly interact with buyers and receive money, thereby transferring control over income to others - typically male family members. These male family members usually lack the expertise and motivation to sell their products in the market. Consequently, women are likely to fetch lower prices and are vulnerable to losing control over their income.  Limited knowledge of market demands – While women in Balochistan are skilled embroiderers, the products, colors and designs in demand in Balochistan villages may not appeal to markets outside of the local area. These women have little knowledge of the current fashion trends in embroidered products in the rest of the country. Immobility limits women’s ability to gather market intelligence and meet market demands. As a result, they are not able to adapt their products to needs of this section of the market and miss out on potentially lucrative revenue streams.  Minimal business skills – High rates of illiteracy and low educational attainment limit women’s business skills. Women in such setting are unaware of basic business concepts such as costing, packaging, pricing, and negotiation skills.  Limited access to business development services – The remoteness and distances coupled with limited access to public and private sector services limit women’s access to business development services. Women are either unaware of available business development services or have no access to them. These services include access to financial services, usage of courier/postal services as a mode of transportation, and packing/ packaging. USAID’s Intervention in Response In response to these problems, USAID provided a 27-month, $1.1 million dollar grant in 2009 to Mennonite Economic Development Associates (MEDA) through a sub-award from the FAO￾implemented United States Assistance to Balochistan Border Areas (USABBA) project. The MEDA project – the Women’s Economic Empowerment: Balochistan (WWE:B) project – specifically seeks to economically empower secluded women by building their capacities to access targeted value chains (i.e., embellished products). MEDA implements activities in Balochistan through its local partner, the Water, Environment, and Sanitation Society (WESS) with capacity building support from Empowerment through Creative Integration (ECI). The project seeks to empower largely homebound women in five districts2 in the culturally conservative and economically disadvantaged northern Balochistan province in which the USABBA project operates (Figure 1). To do so, it builds on indigenous skills in embroidery – teaching Women Embellishers (WEs) modern market demands for design and quality and linking them to markets through mobile Female Sales Agents (FSAs). Through these FSAs, the project aims to develop market 2 Killa Saifullah, Loralai, Mastung, Quetta and Zhob. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 5 linkages for the WEs and thereby allowing them to overcome social barriers and limitations in value chains, reach consumers with suitable products, and realize a higher return on labor. The project contributes to USAID’s Assistance Objective (AO) 2: Improved conditions for broad￾based economic growth and Intermediate Result (IR) 1: Increased income generation opportunities. It also contributes to the cross-cutting objective IR 2: Increased gender equity. Project Objectives, Components and Outputs The WEE:B project aims to integrate women with limited mobility into high value markets by “providing them with sustainable market linkages and support services that will enable them to access effective markets for the products demanded by modern consumers” (MEDA, 2009).The specific goal and objectives stated in the work plan include: Goal: “To increase incomes and economically empower women producers in up to five districts of Balochistan, by facilitating their access to viable markets with appropriate products.” Objectives: 1) To provide homebound and isolated rural women with sustainable market linkages, product information, quality control, skills development, improved input supplies and other support that will enable them to access growing and dynamic markets with the products demanded by contemporary consumers. 2) Achieve scale by collaborating with other organizations and as appropriate, developing the capacity of local organizations to implement market driven value chain development projects, and to partner with the public and private sectors to accomplish this goal. The project model works largely through FSAs – women who enjoy some mobility – that the project trains to provide WEs with business support services, market information (e.g., the designs demanded by urban and export markets), embroidery skills training, and access to high quality inputs. In addition to business support, the FSAs also deliver WEs’ products to markets outside their home communities. Thus, the model improves the quality of WEs’ output while also putting in place mechanisms to access markets and generate feedback from the market. Project activities focus on three areas: (a) market linkages: linking WEs and Women’s Embellishment Groups (WEGs) to wholesale and retail markets through FSAs and Key Facilitating Partners (KFPs); (b) support services: facilitating provision of support services (e.g., input supply, commercial contract arrangements, industry training (technical, organizational and business management), microfinance) to FSAs, KFPs, and WEs; and (c) product development: improving the design, branding, quality control, packaging etc. of embellished products. The sub-award target outcomes are: (a) improved WE/FSA/WEG skills in the production of traditional hand-crafted products, new product development and quality control; (b) improved support services to WE/FSA/WEG in the areas of input supply, market linkage, market information and microfinance; and (c) strengthened capacity of KFPs to support market-driven and women-focused value chain development. In addition, MEDA Pakistan aims to achieve scale by collaborating with local Quetta-based organizations and as appropriate, developing their organizational capacity to implement market driven value chain development projects to accomplish this goal. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 6 Approach to Marketing In order to integrate women with limited mobility into high value markets and provide them with sustainable market linkages and access to support services, the project introduced the “sales agent” mode. The mobile FSAs provide the link between the producers and the buyers. Through this model, the project aims to connect homebound women with the markets. The model was implemented through the following steps: (a) identifying skilled home bound women embellishers; (b) organizing women embellishers in groups; (c) identifying and selecting a group leader, i.e. an FSA with entrepreneurial skills and mobility; (d) building the capacity of FSAs (through training in production management, market linkages, exposure visits and entrepreneurial competencies); and (e) facilitating FSAs to increase engagement of associated women embellishers in work. In addition, the project facilitates the establishment of informal FSA associations. Through these associations, the project intends to link WEs with each other so they can help each other choose colors, share market information, and improve work quality. The project also helps WEs register with the Women Chamber of Commerce and Department of Social Welfare. By virtue of registration, FSAs may have representation in the Chamber, which provides them a stronger position to negotiate with input suppliers and, at the same time, gain the trust of buyers. The project’s marketing strategy aims to enhance opportunities for WEs and FSAs by increasing their awareness of product designs demanded by the market and building their capacity to improve product quality. The project also aims to develop sustainable linkages between FSAs and market functionaries (e.g., input suppliers, wholesalers, final consumers, etc.). Monitoring and Evaluation MEDA developed a monitoring and evaluation plan for the WEE:B project. In particular, MEDA has done the following: a) Established a Performance Measurement Framework: The project maintained a detailed performance measurement framework that defined results and targets, activities, performance indicators, data sources, data collection methods, data collection frequency and the person responsible for collecting the information. MEDA and WESS review results statements and indicators on an ongoing basis. b) Assigned a Monitoring and Evaluation Officer for WESS: WESS appointed a monitoring and evaluation officer who has continued with the project since the start of the project. c) Designed and conducted a baseline survey: MEDA conducted a baseline survey within the first six months of project implementation. d) Reports regularly on the project: The project team reports to FAO and the MEDA management team on a regular basis regarding progress towards achieving expected results and accounts for any variances between planned and achieved results. PURPOSE OF THE EVALUATION This evaluation is intended to provide USAID/Pakistan with an independent assessment of the WEE:B project. As stated in the SOW (see “Women’s Economic Empowerment Balochistan: Statement of Work” produced as a separate annex to this report), the evaluation focuses on the following: (a) High-level questions related to relevance, effectiveness, efficiency, impact, sustainability, and demonstration. The first five of these terms are standard evaluation criteria. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 7 (b) Cross-cutting questions pertaining to gender, reporting, public relations/media coverage, and coordination, some of which may be dealt with in terms of the standard evaluation criteria. (c) Recommendations, including ideas for strengthening specific interventions, ties with the government, and sustainability. The evaluation focuses on the period 2009 – 2011. Evaluation Questions The evaluation questions are clustered under five standard evaluation criteria (relevance, effectiveness, efficiency, impact and sustainability), and the over-arching criteria of replication, gender, reporting, public relations/media coverage, and coordination. The specific questions are: (a) Relevance – Was the project designed to address needs relevant to beneficiaries and has it maintained its relevance throughout implementation? The question of relevance should address whether the project intervention, as designed, was relevant to the needs of intended beneficiaries and appropriate in the project context. It should also determine whether the project has adjusted its approach, if necessary, to improve or maintain relevance. (b) Effectiveness – To what extent has the project achieved its stated objectives? The question of effectiveness should address the extent to which the project has achieved stated objectives and targets and, if it failed to do so, explore the reasons. It should draw out any lessons for setting targets for similar activities in the future. (c) Efficiency – How efficiently has the project used its resources to achieve results? To the extent possible with available data, the evaluation may use measures of efficiency such as Internal Rate of Return (IRR), cost-benefit analysis, cost-effectiveness, or the ratio of overhead and administrative costs to estimated monetary benefits. (d) Impact – To what extent has the project affected household incomes of beneficiaries and what, if any, impact have changes in income had on household welfare or women’s empowerment? It is somewhat premature to determine the ultimate impact of the project. Nevertheless, to the extent possible with available data, the evaluation should determine changes in beneficiaries’ household income attributable to the project. The evaluation should also examine secondary benefits associated with changes in household income and unintended benefits and costs. (e) Sustainability – To what extent are project results likely to be sustained after the project ends and how could prospects for sustainability be enhanced? At one level, the question of sustainability should address whether the project was designed and implemented in a manner which focuses on sustainability after the project ends without building donor dependence. It should also ask whether the project timeframe was sufficient to achieve sustainable results. On another level, the evaluation should also assess whether the project has built the capacity of local institutions to undertake similar activities without direct USAID capacity-building support. (f) Replication – What is the potential for replicating the activities and results of the project in other areas and what characteristics of those areas would likely affect project success? (g) Gender – To what extent, and how, has the project benefited women? The evaluation should assess the project’s effect on women’s empowerment. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 8 (h) Reporting – Has the grantee reported on time and in a useful manner? (i) Public Relations/Media Coverage – How effective has the project been in getting its story out (e.g., outlets, frequency, content, compliance with USAID branding)? (j) Coordination – To what extent has the project coordinated with stakeholders and others (e.g., the Government of Pakistan, the USABBA project, other USG projects, other donors, the private sector) and what concrete steps can be taken to improve coordination? EVALUATION METHODOLOGY Management Systems International (MSI) conducted the fieldwork for the evaluation between November 23 and December 4, 2011 under the Independent Monitoring and Evaluation Contract (IMEC). A team consisting of three MSI staff members, an Independent National Consultant, two female survey coordinators and two local female enumerators conducted the field work with the assistance of the project’s staff. The MSI team members designed the evaluation approach, managed the evaluation, reviewed documents, interviewed partners and others as possible, designed data collection instruments, analyzed data and wrote the evaluation report. The two survey coordinators and two enumerators focused on interviewing WEs and FSAs – a task that the male evaluation team members could not perform. A five person team of data entry specialists supervised by an MSI survey coordinator coded questionnaires and entered and cleaned quantitative data from a small survey. The evaluation team utilized a variety of methods to gather and analyze quantitative and qualitative data. Specific data collection methods included: Document review: The evaluation team reviewed all available project documents and reports to develop a thorough understanding of project goals and objectives, planned and actual activities, outputs, and results. The team reviewed the project’s baseline and annual assessment survey reports and data to evaluate the quality of the research design and data for interpreting reported survey findings. Annex 2 lists documents the team reviewed. Small-scale survey: The evaluation team conducted a survey of 104 WEs to validate key outcomes reported by WEE:B. Based on the project’s objectives, the main impact hypothesis suggests that WEs will increase their incomes from embellished goods by producing higher quality or more marketable and appropriate products and selling them in higher value markets. The survey contained a few tightly focused questions designed to provide evidence on the following general questions: (a) Are WE’s producing a greater quantity of embellished items and, if so, why? (b) Are WEs producing higher quality or more market appropriate embellished items and, if so, how and why? (c) Are WEs selling embellished items to different markets and, if so, how and why? (d) Are WEs receiving higher prices for embellished items and, if so, why? (e) Are WEs earning higher net incomes from the sale of embellished items? Annex 3 contains the questionnaire used in the survey. The evaluation team, with input from consultants, developed the questionnaire specifically to address the evaluation questions posed by USAID in the SOW. The team selected enumerators to suit the needs of the assignment (i.e., ability to speak Pashto, a graduate degree). MSI staff trained the enumerators prior to collecting data. The security situation in Balochistan at the time of the survey largely dictated the choice of project districts that the evaluation team visited with Killa Saifullah and Mastung deemed too dangerous. Within districts, locations were selected according to their accessibility as well as security level. The data was entered using SPSS, under the supervision of MSI Staff. Table 2 details the locations within each district where the interviews were conducted. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 9 TABLE 2: LOCATION AND NUMBER OF INTERVIEWS Location (village) Number Quetta Killi Hussain Abad 9 Killi Shaikh Hussain 9 Hazara Town 9 Killi Kotawal 9 Loralai Godi Mohalah 8 T.T.C. Colony 8 Babu Mohalah 9 Killi Bawar 9 Zhob Ghareeb Abad 8 Sher Khan Mohalah 7 Killi Naseer Abad 19 Interviews: The evaluation team conducted interviews with FSAs, shopkeepers, and project staff, as described below: (a) FSAs: The team interviewed 16 FSAs to ask about project benefits, sustainability of their role as FSAs, and dynamics within their family and within their groups of WEs. The questionnaire used for the purpose is attached as Annex 4. (b) Shopkeepers: The team interviewed 11 shopkeepers to ask about their demand for embroidered products, the quality of the final products, and inputs. The questionnaire used for the purpose is attached as Annex 5. (c) Project Staff: The team interviewed project staff, both at MEDA’s Islamabad office and at the WESS office in Quetta to get the project’s view of its own performance. Group Discussion: The team conducted a group discussion with 12 FSAs. These FSAs represented WEGs from Quetta and Mastung. The questions used for the discussion were the same as those used for individual interviews with the FSAs. The discussion was moderated by the Evaluator/Training Specialist. Orientation Workshop: The evaluation team facilitated an orientation workshop at the MEDA office in Islamabad, in which project staff presented their perspectives on the evaluation questions. This workshop acted as a source of input as it gave an opportunity for project staff to describe the project to the evaluation team and for the evaluation team to review the evaluation process with the partners. Furthermore, issues raised during the workshop were used in designing the questionnaires for the interviews and survey. Site Visits: The team visited 104 WEs and 16 FSAs in the districts of Quetta, Zhob and Loralai for the survey and interviews, respectively. The questions focused on the changes in the income of the beneficiaries, taking into consideration production data, their perceptions about the market, and their experience with trainings. The team also visited 11 shops that sold embroidered products in Meezan Chowk, Liaquat Bazar, and Shabnum Market in Quetta. The shopkeepers were asked about market trends and the quality of products produced by WESS beneficiaries. Secondary Data: The team also relied on secondary sources for data including the Labor Force Survey of 2010-11 conducted by the Federal Bureau of Statistics (FBS) along with project data on sales figures, which were used to calculate benchmark prices for each product category. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 10 The team also contacted FAO officials by email to get a verdict on the quality of the project’s reporting and coordination performance. FINDINGS AND CONCLUSIONS Findings and Conclusions on Relevance Evaluation Question: Was the project designed to address needs relevant to beneficiaries and has it maintained its relevance throughout implementation? Findings The project aims to increase incomes and to empower women producers of five districts in northern Balochistan. These women are the beneficiaries of a strong tradition of embroidery that has been passed through generations. Many women in the region (42 percent of 131 female respondents to the 2010 USABBA baseline) make and sell embroidered products to earn income. Prior to the initiation of the WEE:B project, research conducted by FAO Balochistan, MEDA, and Karavan concluded that there was a strong market for embroidery and “potential to involve significant number of female producers in this hand embellished garments value chain”(MEDA, 2009). The project has assisted embroiderers to produce several traditional embroidered products characteristic of the five districts the project has been involved in. 3 Shopkeepers interviewed by the evaluation team related that similar embroidery patterns are being replicated by machine, with the factories mostly based in Punjab. Hand embroidered shawls were found to be more expensive than machine-produced shawls. Prices for machine-produced shawls ranged from Rs. 900 to 3,500 while the hand stitched shawls ranged from Rs. 500 to 30,000. The conservative social norms prevalent in the target districts restrict women from interacting with men and also confine women largely to their homes. Consequently, few women (9 percent) participate in the formal labor force (Labor Force Surveys, various years) and many (42 percent according to the USABBA baseline survey) earn income making embroidered products in their homes. The USABBA baseline also found that selling embroidered products represented the single largest source of income for women in the project area. Prior to the WEE:B project, homebound WEs often relied on male family members to market their embroidered products. A large majority of the FSAs in the group discussions reported that this marketing channel was not reliable and recounted instances where male family members did not return the money due their female relatives after selling the products. All of the participants of the group discussions agreed that male family members were generally not interested enough or knowledgeable enough to get insights from the market about the products and designs that consumers demanded. The WEE:B project established Female Sales Agents (FSAs) to act as a link between the market and Women Embellishment Groups (WEGs). FSAs deliver the finished products from the WEGs to the market. They also bring back order specifications from the market, thus updating the home bound artisans on the latest trends and demands in the market. As the project got underway, project staff observed that some WEGs found it difficult to obtain financing for their enterprises. The project responded by introducing Community Based Saving Groups (CBSGs), a mechanism for generating working capital. These voluntary groups have between 12 and 15 members, where members pool their funds together for withdrawal at a later stage. Twenty￾one project-supported groups have established such savings plans to date. 3 Annex 1 contains a complete list of the products produced by project-supported embroideries. Annex 2 contains a brief introduction of the five main products. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 11 Balochistan currently has the lowest provincial literacy rate in Pakistan, which at 37 percent is much lower than the national average at 55 percent and of Sindh at 58 percent (Pakistan Social and Living Standards Survey: 2010-11, 2011). The low literacy rate makes it difficult for the project to find qualified staff. Management staff from MEDA, WESS, and FAO all voiced this view during the debriefing meeting. Given this shortage of skilled personnel along with the specialized nature of the project’s activities in value chain development, the project aims to develop the capacity of its implementing partner to “implement market driven Embellished Garments Value Chain Development Project.4 . Interviews with project staff also revealed that there is a need for such capacity development within the project. For this reason, MEDA engaged a capacity building partner to increase the capacity of the implementing partner. The marketing strategy of the project (MEDA, 2011a) includes: 1. a description of the stitches used in Balochistan’s embellished apparel; 2. general guidelines on pricing and promotion; 3. identification of marketing towns, in and around Balochistan; and 4. thirteen general recommendations. The project’s marketing strategy lacks guidelines on consumer preferences and the choice of appropriate marketing channels. It also does not match products with markets or specify its target income groups and price range. By comparison, the non-governmental organization (NGO) called the SAARC Business Association of Home-Based Workers (SABAH) prepared a more comprehensive marketing strategy on embellished apparel (SAACRC Business Association of Home Based Workers, 2010) that includes the following aspects: 1. introduction to the textile industry and its segmentation into apparel and other products, and between women’s and men’s apparel industry; 2. a focus on the fashion apparel industry, including the main trends; 3. identification of specific fashion designers; 4. quantitative estimates of market size and the market share of men’s and women’s apparel; 5. list of boutiques and designers in Pakistan, and their price ranges and distribution channels; 6. analysis of the Strengths, Weaknesses, Opportunities and Threats (SWOT) of the apparel industry; and 7. analysis of competitive forces. Conclusions  Many elements of the project design are relevant to the needs of beneficiaries and appropriate in the context of empowering women in northern Balochistan. In particular, the project’s focus on the embellished products value chain is an appropriate approach to engaging women in economic activity in the project area. It capitalizes on indigenous skills and practices, serves a demonstrated market, and is well suited to work within the home. The approach of using mobile women as sales agents to facilitate quality improvement, to link WEs to markets and to communicate market demands back to WEs addresses real constraints that WEs in the region face when accessing markets. Consequently, many of the project-supported WEs have changed the way they are doing business. Given the generally low levels of human capacity in Balochistan, the project’s focus on building the capacity of local partners to carry on the project’s activities is essential. 4 Outcome 3 – Project PMP WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 12  The project has maintained its relevance to some extent by supporting activities (i.e., CBSGs) to address emerging needs of WEGs.  The project’s marketing strategy is rudimentary in comparison with a more comprehensive strategy such as the one from SABAH. In particular, it does not provide adequate guidance on key aspects of marketing, including customer profiles, products that cater to specific markets and pricing. Findings and Conclusions on Effectiveness Evaluation Question: To what extent has the project achieved its stated objectives? Findings The WEE:B project organizes its activities under three outcomes.  Outcome 1: Increased production capacity and quality of products made by women embellishers in targeted districts of northern Balochistan.  Outcome 2: Women producers linked with sustainable markets and support services.  Outcome 3: Develop capacity of local and national level partner organizations/institutions to implement market driven embellished garments value chain development projects. According to the project’s “Targets vs. Achievements” report (WESS, 2011), the project has met or exceeded targets for most activities under these outcomes. In particular: Outcome 1: The project exceeded its stated targets for 19 of 20 activities/sub-activities under this outcome – particularly in conducting sessions to encourage FSAs to improve input quality (350 percent of the target). It fell short only on a sub-activity concerned with conducting product development and design trainings, where the project was supposed to organize 20 trainings, but managed to conduct 19. Outcome 2: The project exceeded its targets for 15 of 17 activities/sub-activities. It exceeded targets substantially for facilitating FSAs’ participation in (Balochistan) exhibits (770 percent of target) and organizing meetings between buyers and sellers (615 percent of target). It met targets for all other activities/sub-activities under this outcome. Outcome 3: The project exceeded targets for four out of seven activities/sub-activities under this outcome. It particularly exceeded targets for the number of meetings conducted with national level experienced organizations/institutions having greater understanding of entrepreneurship development (200 percent of target) and conducting value chain development trainings for the partners (179 percent of target). It fell short of the target for conducting value chain trainings for project staff. The project initially planned to conduct eight separate sessions to cover various topics of its planned curriculum. However, due to the security situation in Quetta, the project decided to consolidate individual trainings into a single 20-day session with a follow-up refresher of six days. Therefore, out of the original targeted plan of eight trainings the project conducted two trainings of longer duration. Annex 8 reports targets and achievements for each activity/sub-activity. Table 3 summarizes the number of activity/sub-activity targets for which the project failed to meet, met, or exceeded stated targets. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 13 TABLE 3: ACHIEVEMENT OF PROJECT TARGETS Outcome No. Total activities/sub activities (number) Achievement relative to targets (number) Less than Equal to Greater than 1 20 1 0 19 2 17 0 2 15 3 7 1 2 4 Total 44 2 4 38 Source: WESS, 2011. Findings relative to achieving the outcomes themselves include: Outcome 1: Several indicators suggest that participating WEs have increased production capacity and quality since the start of the WEE:B project. In particular, WEs reported that project-supported training was effective in building their skills, they had increased the quantity and quality of products they produced, and they were committing more household resources to embroidery. Most WEs who participated in project-supported training found the training effective in improving their skills. Of the 104 respondents to the small scale survey, 19 (18 percent) participated in embroidery skills training, 29 (28 percent) in market awareness training, and 31 (30 percent) in design training. None of the women reported that the trainings had been ineffective and a majority found the market awareness and design trainings very effective. Figure 5 summarizes responses. FIGURE 5: EFFECTIVENESS OF PROJECT-SUPPORTED TRAINING Source: MSI small-scale survey of WEs. Improved skills led to higher quality products, higher prices, and increased income. A vast majority (90 percent) of WEs that the evaluation team surveyed reported that their annual incomes from embroidery had increased since they began participating in the WEE:B project. Almost a third (30 percent) cited improved quality as one of the two most important reasons for increased income from embroidery. Table 3 summarizes responses. 0% 10% 20% 30% 40% 50% 60% 70% Embroidery skills Market awareness Design Percentage of respondents Very effective Effective No response WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 14 TABLE 3: REASONS FOR INCREASED INCOME Reasons WEs Number Percent Made more pieces 62 67% Made more expensive items 32 34% Selling in different places 30 32% Improved quality 28 30% Received higher prices 14 15% Source: MSI small-scale survey of WEs. WEs that engaged with the WEE:B project seem to have improved the quality of their embroidered work. The MSI small-scale survey of WEs found that 98 percent of the respondents reported improving the quality of their inputs as a result of the WEE:B intervention. Most of the shopkeepers the evaluators interviewed declared that the quality of inputs used by project-supported WEs was “the best.” Table 3 also suggests that project-supported WEs have increased production (i.e., are making more products) and productivity (i.e., increased income from making and selling more expensive items). Detailed production data collected by MSI in the small-scale survey of WEs found that, on average, WEs had increased the number of pieces produced annually by a factor of ten – from an average of just under eight per year before the WEE:B project to 82 after the project. While the women increased production of most of the 27 specific items included in the survey, the largest absolute increases were in smaller items like bangles, bags, dopatas, pouches, mobile phone covers, etc. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 15 Figure 6. Differences in production before and after the project began are statistically significant at least a ten percent level of significance (Annex 9). WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 16 FIGURE 6: AVERAGE NUMBER OF ITEMS PRODUCED ANNUALLY Source: MSI small-scale survey of WEs. Project-assisted WEs also appear to have changed their production patterns to some extent since the WEE:B project began. The proportion of women making each of the 27 individual items included in the MSI survey has not changed dramatically. However, there has been a noticeable, if small, shift away from some items and into others. Figure 7 shows the percentage of the 104 surveyed WEs who reported making a particular item in the year prior to the start of the WEE:B project and in the time since the WEE:B project started. Items at the top of the chart exhibited the largest percentage increase in the number of WEs producing the item while items at the bottom exhibited the largest decrease. FIGURE 7: PERCENTAGE OF WOMEN PRODUCING EACH ITEM 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 Koreshia sweaters Koreshia table mates Wall hanging Wool frocks Photo frame Door bells Mirrors Bed sheets Pathani Suit Zartar gala File covers Balochi Suit Zartarpaties Hand fan Shawls Decoration pieces Cushion set Simple suit Bags Key chains Mobile covers Ribbon paties Balochi paties Balochi gala Dopatas Bangles Pouch Average number of items per year Before After WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 17 Source: MSI small-scale survey of WEs. While a total of 27 products are being produced by the WEs, there is considerable variation in levels of production. Fifty-four percent of the respondents reported producing “dopatas”, followed by “simple suits” at thirty-eight percent, whereas less than one percent reported producing wool frocks and Koreshia table mats. Although the preference for “dopatas” was also evident in the pre WEE:B scenario, in the after project situation there has been a considerable increase in the interest of WEs in items such as “photo frames,” “zartar paties,” “Balochi gala,” “pouches,” “Balochi paties” and “ribbon paties”. The common factor among these products is that they are different from the traditional products and are more in line with changing market demands. Surveyed WEs also increased production by increasing the number of women in their households who make and sell embroidered products. A majority (59 percent) reported that the number of women in the household making and selling embroidered products had increased relative to the year prior to the start of the WEE:B project. Outcome 2: The MSI small-scale survey found that project-supported WEs had shifted largely from using family members to obtain inputs to buying inputs from FSAs – a more formal link to input supplies of reliable quality. Prior to the WEE:B project, 52 percent of respondents obtained inputs from family members and none obtained inputs from FSAs. Since the project, the percentage 0% 2% 4% 6% 8% 10% 12% 14% 16% 18% Simple suit Balochi Suit Cushion set Dopatas Koreshia sweaters Ribbon paties Koreshia table mates Wool frocks Shawls Pouch Wall hanging Balochi gala Decoration pieces Photo frame Hand fan Mirrors Door bells Bed sheets File covers Bags Zartar gala Balochi paties Pathani Suit Zartarpaties Key chains Mobile covers Bangles Percentage of surveyed women Before After WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 18 obtaining inputs from family members has fallen to 19 percent while the percentage obtaining inputs from FSAs has increased to 54 percent (Table 4). TABLE 4: SOURCES OF EMBROIDERY INPUTS Source of inputs After WEE:B Before WEE:B Get them yourself 19% 28% Family members 19% 52% Female Sales Agents 54% 0 Source: MSI small-scale survey of WEs. Additional results include:  Ninety-three percent of respondents reported selling their products themselves prior to the project. Now only 25 percent reported selling their products themselves while 58 percent reported selling through a sales agent.  Prior to the project, most women (58 percent) reported selling most of their products within the village. After the project, 59 percent reported selling most of their products outside the village.  Ninety-five percent of respondents were either “very satisfied” (73 percent) or “satisfied” (22 percent) with the WEE:B project. Outcome 3: The project aims to build the capacity of its staff, especially for value chain development. However, staff members reported that they could benefit from more training. MEDA engaged Empowerment through Creative Integration (ECI) to build the capacities of local partners, including WESS, to implement market driven value chain projects. However, the Chief Executive Officer (CEO) of ECI admitted that her organization was not equipped to impart trainings on value chains. Project staff also expressed the opinion that the trainings imparted by ECI were not beneficial given their original purpose. The training program was not developed around a comprehensive and well-articulated marketing strategy. The Key Facilitating Partner (KFP) for capacity building was not familiar with the value chain concepts required by the project. According to the CEO of ECI, her organization needed interaction with the project to better understand its needs. Even so, MEDA terminated the contract with ECI because of dissatisfaction with its performance. The project also provided training for WEs as well as trainings and exposure visits for FSAs. Respondents to the survey of WEs seemed satisfied with the trainings provided by the project, as the training was deemed “very effective” by 43 percent of the respondents on embroidery skills, 76 percent on market awareness and 71 percent on demand designs. Survey results show that beneficiaries seem to be satisfied with the project. On the question of overall satisfaction with the WEE:B Project, 76 percent of the respondents replied with “Very satisfied” while 23 percent replied with “Satisfied”. Furthermore, 91 percent of the respondents associated trainings with increases in income. Detailed sales records maintained by FSAs show increased sales since the beginning of the project. Total sales by FSAs increased 22 percent in the April to October period of 2011 relative to the same period of 2010. Conclusions  Project activities have been effective in increasing production of embellished items among project-assisted WEs, ensuring delivery of a high-quality product to markets and shifting production to items more consistent with the preferences of modern consumers. It is not WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 19 possible with existing data to conclusively determine whether product quality has increased as a result of the project but the quality of inputs has increased.  The market linkages (both input and output) seem accepted and beneficial to WEs.  The project has struggled to some extent building the capacity of national level partner organizations/institutions due to the absence of a proper KFP for capacity building and a training program that was not grounded in a thorough marketing strategy. A training program based on a more comprehensive and well-articulated marketing strategy may have better addressed trainees needs.  Given the overachievements of the targeted activities it seems that the original targets could have been set higher. Findings and Conclusions on Efficiency Evaluation Question: How efficiently has the project used its resources to achieve results? Findings Measures of efficiency typically compare spending with outcomes (financial and otherwise) over time. This section thus begins by summarizing the trajectories of project expenditures and monetary benefits from 2009 through 2011. Key findings relative to spending include:  According to the project’s original budget, it should have spent $891,751 of its total $1.117 million award by June 30, 2011. However, as a result of a delay in signing the sub-award agreement with FAO and the security situation in Balochistan, which constrained the capacity of the project to carry out planned activities, the project spent only $567,549 through June 30, 2011 (the most up-to-date spending figures available to the evaluation team). This represents 64 percent of planned spending and 51 percent of the entire project budget. Spending by MEDA accounts for 58 percent of expenditure to date while WESS accounts for 42 percent. MEDA has spent 54 percent of the total amount allocated to it in the project budget while WESS has spent 36 percent.  Records provided to the evaluation team did not report expenditure by calendar year. However, based on the rate of expenditure in the first half of 2011 (roughly half of expected 2011 expenditure) it is on track to spend the budgeted $451,310 in 2011. Lacking better detail, the project’s expenditure for 2009, 2010, and 2011 is probably close to $0, $332,758, and $451,310.  As of June 30, 2011, payments to local partners represented the single largest expense and accounted for 42 percent of total costs. Personnel costs (MEDA and WESS) and technical support services accounted for 26 percent and 18 percent of the budget, respectively. Table 6 summarizes project expenditures. Annex 7 contains more budget detail. TABLE 5: OVERVIEW OF PROJECT BUDGET AND EXPENDITURE Allocated budget ($) Percentage of allocated budget Actual expenditure ($) % of expenditure All budget and expense items WESS 400,000 36% 238,895 42% ECI 110,000 10% - 0% MEDA 607,399 54% 328,654 58% Total 1,117,399 100% 567,548 100% WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 20 Personnel and administrative budget and expense WESS 134,728 44% MEDA 174,967 56% Total 309,695 100% Source: Project records The project made a payment of $63,000 to ECI for its services, but the allocation was made through the MEDA allocation, and not through the original amount allocated for ECI. Project management has blamed the error on the inefficiencies of its finance staff. The project is mostly staffed by residents of Balochistan, and is supported by other Pakistani staff in Islamabad as well as short term expat consultants. Based largely on the impact section, findings about the trajectory of monetary benefits include:  The project’s reports to FAO state that it registered no WEs in 2009 (due to a delayed implementation), 3,817 in 2010, 1,279 in 2011 and expects to register an additional 204 in 2012, for a total of 5,300 WEs.  Table 9 reports average annual increases in income per WE of Rs. 90,115. If the project registers WEs throughout the year and they begin earning increased incomes immediately, then, on average over all WEs registered in a given year, WEs will earn half of the average annual income increase in the year they are registered and the full increase thereafter. If the project is sustainable, then they will earn these increased incomes indefinitely.  These findings and assumptions imply the trajectory of monetary returns to the project summarized in Table 6. For purposes of the cost benefit analysis, Table 6 projects benefits for five years beyond 2011 and assumes that they remain constant (i.e., that the project does not register additional WEs and that registered WEs continue to earn the same increment to their incomes. TABLE 6: MONETARY BENEFITS BY YEAR Year Number of WEs registered Benefits (Rs. X 1000) By year For at least one year In year Ongoing Total benefits by a b Rs. 45,057 * a Rs. 90,115 * b year 2009 0 0 0 0 0 2010 3,817 0 171,983 0 171,983 2011 1,183 3,817 53,302 343,969 397,271 2012 0 5,000 0 450,575 450,575 2013 0 5,000 0 450,575 450,575 2014 0 5,000 0 450,575 450,575 2015 0 5,000 0 450,575 450,575 2016 0 5,000 0 450,575 450,575 Total 5,000 2,822,129 Table 7 summarizes the calculation of project efficiency measures given the monetary benefits in Table 6 and the pattern of expenditure described above. TABLE 7: PROJECT FINANCIAL ANALYSIS CALCULATIONS Benefits Costs WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 21 Year Rupees (1,000) Dollars Dollars (present value) Dollars (nominal) Dollars (2011 prices) Dollars (present value) 2009 0 0 0 0 0 0 2010 171,983 2,001,664 2,201,831 332,758 327,389 360,128 2011 397,271 4,566,333 4,566,333 451,310 451,310 451,310 2012 450,575 5,179,023 4,708,203 0 0 0 2013 450,575 5,179,023 4,280,184 0 0 0 2014 450,575 5,179,023 3,891,077 0 0 0 2015 450,575 5,179,023 3,537,342 0 0 0 2016 450,575 5,179,023 3,215,766 0 0 0 Present value 2,822,129 26,400,736 811,438 Note: 2011 prices determine from historic US dollar inflation rates (i.e., 1.64 percent in 2010 and -0.34 percent in 2009). Rupees to dollar conversions based on average annual exchange rates (i.e., Rs. 81.77=$1 in 2009, Rs. 85.92=$1 in 2010, and Rs. 87.00=$1 in 2011 [http://www.oanda.com/currency/average]. Present values are based on an annual (real) discount rate of 10 percent. Typical measures of efficiency include:  Net Present Value (NPV) – The project’s net present value is the difference between the present value of benefits and the present value of costs. A positive value indicates that a project has returned benefits in excess of costs. Since the measure is independent of the size of a project’s budget, however, it provides no evidence of the return on the project investment. For example, the returns implied by a NPV of $1 million are very different for a $5 million project than for a $20 million project.  Benefit Cost Ratio – The benefit cost ratio is the present value of benefits divided by the present value of costs. A benefit cost ratio of one implies that a project returned one dollar for each dollar expended. A benefit cost ratio provides a rough measure of economic returns because it represents the proportional return on investment. For example, a benefit cost ratio of 1.5 implies that project benefits are 150% of costs.  Internal Rate of Return (IRR) – The IRR is the rate of return on a stream of costs that equates the present value of costs with the present value of benefits. It thus represents the potential financial return on the investment. The IRR, however, provides no information on the distribution of benefits. Table 8 summarizes efficiency measures for the WEE:B project. TABLE 8: SUMMARY OF EFFICIENCY MEASURES Indicator Indicator value Benefit cost ratio 32.54 Net present value (thousand $) 25,589 Current (i.e., 2009-2011) internal rate of return (%) 516% Projected (i.e., 2009-2016) internal rate of return (%) 517% WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 22 Conclusions  By any measure (in the context of the caveats cited above) the project has produced benefits well in excess of costs. Findings and Conclusions on Impact Evaluation Question: To what extent has the project affected incomes from embroidery work and household incomes of beneficiaries and what, if any, impact have changes in income had on household welfare or women’s empowerment? Findings Project Impact on Income Data from the MSI small-scale survey of project-supported WEs suggests that the project has substantially increased women’s incomes from embroidery. Across all 1035 respondents, average annual income in the time period since the start of the WEE:B project was Rs. 93,612 relative to an average annual income from embroidery of Rs. 3,497 before the WEE:B project. These figures are misleading however since 74 percent of respondents reported no income from embroidery work prior to the WEE:B project. The “before” average income values are thus depressed by a large number of zero values – and measures of percentage change in income are meaningless. However, even the 27 women who reported earning income from embroidery prior to the WEE:B project reported substantial gains in income (195 percent on average). Table 9 summarizes survey results on changes in income from embroidery attributable to the WEE:B project. TABLE 9: CHANGE IN INCOME ATTRIBUTABLE TO THE PROJECT District All respondents Respondents with prior sales Number Average annual income (Rs.) Number Average annual income (Rs.) Before After Before After Change (%) Quetta 35 5,889 215,484 16 12,883 45,852 256% Loralai 34 2,714 38,769 6 15,379 36,185 135% Zhob 34 1,818 22,999 5 12,362 22,384 81% All districts 103 3,497 93,612 27 13,341 39,357 195% Source: MSI small-scale survey of WEs. The estimates of annual income from embroidery before the WEE:B project are virtually identical to estimates obtained from a survey of 131 women in the project area conducted by USABBA in 2010 (Food and Agriculture Organization, 2010). That survey concluded that women not engaged in the WEE:B project – but engaged in embroidery – earned an average of Rs. 13,707 per year from embroidery work. Estimates from the USABBA baseline also put changes in income in the 5 One observation was excluded from the analysis because the magnitude of the income increase was 9.7 standard deviations from the mean strongly suggesting a data error. All other values fell within three standard deviations of the mean. IMPORTANT NOTE REGARDING EFFICIENCY MEASURES It is very important to bear in mind that the efficiency measures presented in Table 8 consider gross changes in income attributable to increased embroidery activity. Since surveyed WEs reported committing more time to embroidery – and recruiting other women in the household to embroider – it is very likely that they sacrificed other income-earning opportunities (cash or otherwise) to put additional resources into embroidery. The evaluation was not able to estimate the magnitude of this substitution effect. The estimates of Table 8 therefore almost certainly overestimate actual efficiency measures but the magnitude of the error is not known. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 23 perspective of average household income. Using data from a sample of 441 rural households in the project area, the survey estimated annual average household income from all sources in 2010 at Rs. 395,557. The percentage of household income from embroidery thus increased, on average, from 3.4 percent of total household income before the WWE:B project to 9.2 percent after the project (using the more conservative estimates from respondents with prior embroidery earnings) to as much as 23.1 percent (using data from all women).6 A large majority (94 percent) of respondents to the MSI small-scale survey of WEs reported an increase in income after the project. When asked for the reasons behind the increase, 66 percent attributed it to increased production and 31 percent identified a greater diversity in terms of markets as an important reason. Project Impact on Women and Household Welfare The project has built human capacity by training over 5,000 WEs and 207 FSAs in embroidery and business skills. Many households have responded to the enhanced earning potential of embroidery by increasing the number of women in the household engaged in embroidery. Sixty percent of the respondents to the MSI small-scale survey of project-supported WEs reported increasing the number of women in the household who were making and selling embroidered products. Extrapolating survey results from 104 WEs to the 5,000 households engaged by the project suggests that as many as 3,700 households are earning income from embroidery now that were not engaged in embroidery as a business prior to the WEE:B project. Furthermore, 60 percent of the surveyed households reported increasing the number of women in the household who were engaged in the business of embroidery. Therefore, the number of women newly engaged in the embroidery business is even greater than 3,700. Project-supported WEs reported using most of the income from embroidery for household expenses (food and clothing) with the remainder going to reinvestment in embroidery, health care expenses, education, and savings (Figure 8). FIGURE 8: USE OF EMBROIDERY INCOME Source: MSI small-scale survey of WEs. See the “Gender” section for additional findings on the project’s impact on women’s empowerment within the home and community. Impact at the Community Level 6 Assuming (perhaps unrealistically) that women did not give up other income-earning activities to increase income from embroidery. Food 45% Clothing 34% Health 6% Education 3% Reinvestment 7% Saving 1% Don’t Know 4% WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 24 The project has created WEGs at the community level involving more than 5,000 WEs. Taking this mobilization a step further, the project is overseeing the creation of district level associations of FSAs. The project expects the creation of these groups to provide a platform for the FSAs and their associated WEGs to take advantage of economies of scale in production and to gain access to government business support services. Conclusions  The WEE:B project has substantially affected the income women earn from embroidery. Project-supported WEs have increased incomes by increasing production and quality, diversifying the range of products they produce in response to market demands, and earning higher prices for what they do produce (by gaining access to higher value markets).  The project has built human capacity in the WEs and FSAs it has trained. Based on the large number of women who appear to have started embroidery work as a business as a result of the project, the human capacity benefits may be substantial.  Increased incomes from embroidery improve the welfare of women’s households since the women use earnings from embroidery largely for food and clothing. Findings and Conclusions on Sustainability Evaluation Question: To what extent are project results likely to be sustained after the project ends and how could prospects for sustainability be enhanced? Findings Since the project is still underway, it is too early to tell if project results are sustainable. The evaluation examined whether the project was designed and implemented in a way that built sustainable business relationships (i.e., relationships that could survive without project support) and the skills necessary to operate businesses sustainably. The FSAs were committed to their role in the value chain and believed they could continue playing their role without project support. In the group discussions as well as the interviews, an overwhelming number of the FSAs said they were very happy with their current role, and would be willing and able to continue even if the project ends. Some even suggested that they would resist anyone else in the group trying to take up their role. While many project-supported FSAs appear confident and knowledgeable of the markets, the MSI small-scale survey of WEs suggests that as many as 16 percent of WEs did not understand that they were paying commissions to FSAs. This demonstrates a lack of understanding of how the project works. Project-supported FSAs had also built relationships of trust with shopkeepers. Project-supported FSAs act as liaisons between the shopkeepers and the beneficiaries. Some shopkeepers said that they were not willing to trust just any WEs as there have been instances where embellishers have taken the raw materials and then not delivered the finished product. While the project does introduce its FSAs to the shopkeepers, it does not make any guarantees. If the FSA does not deliver, the WEE:B project is not responsible. Despite the lack of a guarantee, the shopkeepers trusted FSAs associated with the WEE:B project. The FSAs that the evaluation team interviewed in group discussions said that they would be able to leverage these market linkages without project support. The project has also facilitated access to working capital that is important to the sustainability of embroidery enterprises. During the orientation workshop, project staff mentioned that financing has been a major hurdle for WEGs, especially in meeting larger orders as the necessary inputs require a financial investment. The project responded by encouraging the beneficiaries to form Community Based Saving Groups (CBSG) and training them in group formation. As a result 32 such groups have been formed with an average of 12 to 15 members each. Membership is voluntary and the project WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 25 records report that these groups have saved a total of $2,377 to date. The project also provides business development services (BDS) such as advice on marketing, packaging, and branding. The project is also encouraging associations of WEs at the district level to magnify the economies of scale that larger WEGs are already experiencing as larger associations can accept larger orders. The project has already formed informal associations in Mastung and Zhob. Furthermore, the project encourages, and helps, associations register with Chambers of Commerce so they can leverage financing from the Government of Balochistan to attend exhibitions and conferences. The project is extending the associations into the other three project districts. The project specifically aims to minimize the potential for creating donor dependence by relying entirely on a market-driven approach that does not provide any direct monetary or in-kind support to its beneficiaries. The project uses a tested “sales agent model” approach that MEDA used in Pakistan in the “Behind the Veil” project from 2004 to 2007. An external evaluation of that project concluded that it produced monetary benefits for 9,800 women in the rural areas (Management Systems International, 2008). Conclusions  Project results have good prospects for sustainability. The tested approach leverages market incentives to encourage sustainability, builds the necessary skills and relationships among FSAs to promote sustainable results, and minimizes donor dependence.  The factors that make the project approach relevant and necessary (i.e., strict social norms, demand for embroidered products and the embroidery skills of the women of Balochistan) are not likely to change in the near future thus suggesting that the incentives that drive project results are likely to remain relevant.  The FSAs play a crucial role in linking WEs to markets, but they are also in a position to exploit less mobile WEs who cannot directly access markets and have limited knowledge of how the markets work. Such exploitation, if it occurs, becomes widespread and known could destroy the relationships of trust that underpin the project results and threaten the sustainability of results. Findings and Conclusions on Replication Evaluation Question: What is the potential for replicating the activities and results of the project in other areas and what characteristics of those areas would likely affect project success? Findings According to the World Economic Forum’s Gender Gap Report of 2011 (Hussmans, Tyson, & Zahidi, 2011), Pakistan ranks 132 out of 135 countries. On one of the sub pillars of the Gender Gap Index, i.e. Economic Participation/Opportunity, the ranking is even worse, i.e. 134 out of 135. This sub index includes a myriad of indicators such as unemployment rates, land ownership, access to finance, and enterprise leadership. This low ranking nationally (along with the low rates of female participation in the labor force and female literacy described in the introduction) suggests that women throughout much of Pakistan face issues similar to those in Balochistan. The factors that make the project approach relevant – i.e. social constraints, limited access to markets and indigenous embroidery skills – exist in other parts of Pakistan and Balochistan according to a local expert on handicrafts. Other areas where these skills exist include Haripur, Swat, Malakand and Chitral in Khyber Pakhtunkhwa province, Bahawalpur, and Multan in Punjab province, and Sukkur, Khairpur, Tharparker in Sindh. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 26 The survey indicates some demonstration effect of the project within the household, as 60 percent of the survey respondents reported an increase in the number of female household members who are earning income from embroidery. There are many examples of projects that follow similar, if not identical, approaches in Pakistan. The “Behind the Veil” Project, implemented by MEDA and the Enterprise and Career Development Institute (ECDI) from 2004 to 2007, also focused on embroidery and used the FSA model. The project was implemented in the target districts of Multan, Quetta, Karachi, Hyderabad and Thatta and, according to a 2008 evaluation, was successful in empowering women and increasing incomes (Management Systems International, 2008). Similarly the “Livelihoods Recovery Support to Flood￾Affected Artisans of Pakistan” project used a similar approach in all the four provinces “to connect women sales agents” to embroidery markets. The project worked in D.I.Khan, Haripur, Nowshera and Swat in the Khyber Pakhtunkhwa province, Jhang, Khushab, Mianwali, Muzzafargarh in the Punjab Province, Dadu, Jacobabad, SH Benazirabad, Shikarpur, and Thatha in Sindh Province, and Jaffarabad and Sibi districts in Balochistan. The MEDA-implemented Entrepreneurs project is also currently using a similar approach in the dairy, honey, medicinal plants, and embellished fabrics value chains in the rural areas of all the four provinces. Conclusions  The approach is widely replicable – both in the specific example of the embroidered product value chain and more generally in other value chains. It is most easily replicated for embroidered products in other areas of Pakistan where conservative norms and indigenous embroidery skills are prevalent.  While the WEE:B project was designed specifically to address limited market access due to immobility of WEs, the extension of the approach to other value chains by other implementers suggests that it is also relevant when other factors limit producers’ access to markets. Findings and Conclusions on Gender Evaluation Question: To what extent, and how, has the project benefited women? Findings The project systematically and exclusively targets women beneficiaries. It has directly engaged approximately 5,000 WEs and more than 200 FSA and has identified an additional 2,000 WEs for potential future engagement. Women beneficiaries (WEs and FSAs) largely control the income from embroidery activities. When asked about decisions regarding the income from embroidery, only 2 percent replied that the men in the household decided how to spend the money, 54 percent responded that women make the decisions and 37 percent said it was a joint decision by male and female members of the family. Ninety-one percent of the WEs surveyed reported an increase in respect within the family associated with increased incomes from embroidery and 54 percent reported an increase in respect within the community. In both cases, the WEs attributed increases in income to the WEE:B intervention. During the group discussions, FSAs also confirmed that they retain complete control over the money earned through their sales agent activities. The project also employs a significant number of women on the staff at both the headquarters and field levels. Of a total of 30 staff members, just over half (53 percent) are women. Table 10 summarizes the number of men and women on the project staff for both WESS and MEDA. TABLE 10: GENDER BREAKUP OF PROJECT STAFF Organization Office Number of staff WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 27 Male Female Total WESS Quetta head office 4 6 10 Field Staff 8 8 16 MEDA 2 2 4 Total 14 16 30 Percentage 47% 53% 100% Source: Project records. Conclusions  The project has directly benefited as many as 5,000 WEs and 200 FSAs, all of whom are women, by increasing income over which most have some level of control and by enhancing the respect they receive within the household and community. Findings and Conclusions on Reporting Evaluation Question: Has the grantee reported on time and in a useful manner? Findings The WEE:B project (i.e., WESS) submits reports to MEDA, which includes WEE:B results in its own reporting to USAID. MEDA then delivers them to FAO. The project reports on a weekly, monthly, quarterly and annual basis. According to estimates from project officials, the time requirements for delivering each report are as follows: weekly takes two hours, monthly report takes two days, seven days for quarterly report, and 20 days for the annual report. The concerned official at FAO expressed satisfaction with the timeliness and quality of reporting from the WEE:B project and also affirmed that the project seeks FAO approval (to confirm compliance with branding requirements) before printing banners for major activities. The left-hand image in Figure 9 shows a banner from one such activity, which is properly branded. For smaller activities, however, WESS does not seek approval from FAO and banners do not always adhere to USAID branding requirements. The right-hand image of Figure 9 shows a banner from a small exhibition at a girl’s high school in Loralai that is not properly branded. FIGURE 9: EXAMPLES OF BRANDING Source: WEE:B project office. Conclusions  The project has been reporting to FAO as required in an appropriate and timely manner. The project does not consistently brand public events in a manner that complies with USAID guidelines. It appears to brand appropriately when FAO must approve banners, but not as well when it does not receive FAO input on branding. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 28  The unusually high frequency of reporting suggests that project staff spends a considerable amount of time on this activity. Findings and Conclusions on Public Relations/Media Coverage Evaluation Question: How effective has the project been in getting its story out (e.g., outlets, frequency, content, compliance with USAID branding)? Findings The success stories of the project have appeared in MEDA’s quarterly newsletter that MEDA circulates to about 40 organizations. Besides that, there has been not much coverage in the regional press except for the opening ceremony and exhibitions. Project staff claim to have had local media coverage but nobody maintained a record or stored clippings. Thus the evaluation team could not verify the project’s claims. Since its inception, the project has held 12 exhibitions – 9 at the regional level and 3 at the national level – and 2 conferences. Each of these were public events, which generated media coverage – again undocumented by the project. Just over one-third (39 percent) of respondents to the MSI small-scale survey of WEs understood that USAID was funding the project – and only 7 percent believed that USAID was the sole funder. Remaining respondents either did not know who was funding the project (32 percent) or believed that WESS, FAO, or MEDA were funding it. Figure 10 summarizes project participants’ perceptions about the source of project funding. FIGURE 10 : PARTICIPANTS’ BELIEFS ABOUT PROJECT FUNDING Source: MSI small-scale survey of WEs. Conclusions  The project has not done a particularly good job of communicating the role of USAID and the American people in funding project activities.  The project has not been particularly effective in communicating its message to local audiences. MEDA’s quarterly newsletter has a limited audience and does not serve to inform USAID and others, 32% USAID only, 7% WESS, FAO, MEDA, 29% Don't know, 32% WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 29 the target audience in Balochistan who need to hear about the beneficial influence of USAID on their communities. Findings and Conclusions on Coordination Evaluation Question: To what extent has the project coordinated with stakeholders and others (e.g., the Government of Pakistan, the USABBA project, other USG projects, other donors, the private sector) and what concrete steps can be taken to improve coordination? Findings The WEE:B project has been coordinating with the district level governments. The Social Welfare Department has an embroidery training center in Loralai with which the project coordinates in identifying its graduates as potential WEs and FSAs. The project has a similar arrangement with the Frontier Corps as it also has a training center in Zhob. The project has also conducted exhibitions with the Frontier Corps in Zhob. The project has also engaged with district level education departments in all five target districts. This has resulted in exhibitions in girls’ colleges and schools, allowing WEE:B beneficiaries to interact directly with their customers. The project also coordinates with the USABBA project. Since there is little overlap between the communities in which the two projects work, they do not coordinate on operations. Coordination between the two projects includes monthly Progress Review meetings, coordination meetings at the country level and annual progress review/work planning consultations. Furthermore, when USABBA organizes staff training events that involve international consultants, it invites WEE:B project staff to participate (example invitation email in Annex 10 and Annex 11). FAO officials said they are “extremely satisfied” with the results of coordination. The project is also coordinating with other organizations in the industry, but this has been mostly limited to mutual invitations to exhibitions and/or other events. The exhibitions conducted by the WEE:B project have had participation from Catholic Relief Services (CRS), United Nations Children’s Fund (UNICEF), Innovative Development Organizations (IDO), Concern Intl., Trust for Voluntary Organization TVO and Balochistan Rural Support Programme (BRSP). The educational NGO Caritas operates a handicraft display center in Lahore in which the project displays WEs’ products. The project also coordinates with Aik Hunar Aik Nagar (AHAN), a handicrafts capacity building organization, and the Small and Medium Enterprise Development Agency (SMEDA) to arrange invitations to conferences and exhibitions and to obtain technical assistance. Conclusions  The project has coordinated its training, capacity building, and marketing activities with a wide variety of organizations working in relevant sectors. However, at the operational level, it has limited cooperation with organizations that have a similar mandate, such as AHAN. RECOMMENDATIONS The main recommendations are as follows: 1. The project needs a comprehensive marketing strategy based on research in Balochistan and other parts of Pakistan that would include: a. the depth of research that is found in the marketing strategy of SABAH; b. profiles of the type of customers that the project wishes to target; WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 30 c. description and prices of products that match customer profiles7 ; d. specific guidelines for transparently setting prices and commission for all products at the levels of products, FSAs; e. ways of strengthening the bargaining power of the producers vis-à-vis FSAs; and f. partnerships for increasing outreach and enhancing product quality and sustainability. 2. In order to enhance product quality and sustainability, the project could consider the option of making more effective use of the networks accessed by organizations such AHAN (public sector, present in Quetta) and SABAH (NGO, not presently active in Quetta). Moreover, sustainability prospects for the more active WEs may be improved if the project could finance the establishment of a trade facility center in Quetta through, for example, AHAN or SABAH. The latter’s centers centralize the stitching and distribution functions and are supported by powered stitching machines, specialized seamstresses, a master-cutter and designers. 3. To build capacity of the staff and local partner, a new KFP for training needs to be hired. Similarly the existing training manuals for staff need to be upgraded accordingly. 4. The project empowers FSAs to serve as market access points for WEs who often have little knowledge of the market or how markets work. While the evaluation did not find any evidence of FSAs exploiting this advantage, it is certainly a potential. The project, and future projects of this nature, must guard against this type of exploitation, which could undermine project achievements and threaten sustainability. Since monitoring all groups would not be feasible, random checks on embellisher groups would serve the purpose. A standard set of questions should be asked, to ensure that the WEs are aware of the costs as well as payoffs associated to their work. 5. The media efforts carried out by the project need to be enhanced. Given the precarious situation in Balochistan, a more thorough propagation of the WEE:B success story could contribute substantially to spreading the project’s message. Presently, the project staff is not experienced enough to deal with the local news media. For this reason, the project should hire an experienced person who could propagate the WEE:B story through local newspapers and radio. Furthermore, future media efforts need to collaborate more with USAID’s overall media efforts. 6. While current project beneficiaries are likely to continue benefiting from project results and continued donor funding, the implementing partners will likely require additional time and support to develop the capacity to support further value chain development. Furthermore, the project has been on the ground for only about two years, and has only just begun to enable producers to benefit from market linkages. It will take additional time to enhance and consolidate gains from the project by deepening its impact and broadening its outreach. These reasons suggest the value of an additional phase of the project 7 Basic customer profiles and product descriptions are illustrated in Annex 7. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 31 BIBLIOGRAPHY 1. Food and Agriculture Organization of the United Nations (FAO). (2011). United States Assistance to Agricultural Development in Balochistan Border Areas (USABBA) project, Monitoring and evaluation consultant mission report: annex B: baseline report. 2. Global Gender Gap Report 2011, World Economic Forum. Retrieved from http://www3.weforum.org/docs/WEF_GenderGap_Report_2011.pdf 3. Government of Pakistan, Ministry of Labour and Manpower, Labour Market Information and Analysis Unit. (April 2009). Pakistan Employment Trends for Women 2009 (Series No.5). Islamabad, Pakistan. Retrieved from http://www.lmis.gov.pk/publications/PETFW.pdf 4. Haroon, J. & Khan, A. J. (2007). Trends in regional human development indices. Social Policy and Development Center (SPDC) Research Report #73. Islamabad, Pakistan. Retrieved from http://www.spdc.org.pk/Publications/Research%20Reports/RR-73.pdf 5. Hussmans, R. (n.d.). A labor force survey module on informal employment (including employment in the informal sector) as a tool for enhancing the international comparability of data. Retrieved from http://www.ilo.org/dyn/infoecon/docs/429/F473048202/labour%20force%20survey%20infor mal%20employment%202003.pdf 6. Hussmans, R.; Tyson, L.D. & Zahidi, S. (2011). The global gender gap report 2011. World Economic Forum. Retrieved from http://reports.weforum.org/global-gender-gap-2011/ 7. ILO Country Office for Pakistan. (2011). Searching for the invisible workers: a statistical study of home-based workers in Pakistan. Towards Gender Parity in Pakistan (TGP) Project. 8. Kazi, S. & Raza, B. (1989). Women in the informal sector: home-based workers in Karachi. The Pakistan Development Review, 28(4), 777-788. Retrieved from http://www.pide.org.pk/pdf/PDR/1989/Volume4/777-788.pdf 9. Labor Force Survey (LFS). (various years). Federal Board of Statistics (FBS), Pakistan. Retrieved from http://www.ilo.org/dyn/lfsurvey/lfsurvey.list?p_lang=en&p_country=PK 10. Management Systems International (MSI). (April 2008). Report on evaluation of the access to contemporary markets for homebound women embroiderers project. Retrieved from http://www.usaid.gov/pk/downloads/eg/MEDA.pdf 11. Mennonite Economic Development Associates (MEDA). (2009). Women’s economic empowerment: integrating women into sustainable value chains in Balochistan, project implementation plan. 12. Mennonite Economic Development Associates (MEDA). (September 2011a). Marketing strategy: women’s economic empowerment Balochistan (WEE:B) project. 13. Mennonite Economic Development Associates (MEDA). (September 2011b). Sales agent model, women’s economic development. Retrieved from http://www.usaid.gov/pk/downloads/eg/MEDA.pdf 14. Pakistan Social and Living Standards Survey, 2010-11. (2011). Government of Pakistan, Statistics Division, Federal Bureau of Statistics. Islamabad, Pakistan. Retrieved from http://www.pbs.gov.pk/content/pakistan-social-and-living-standards-measurement-survey￾pslm-2010-11-provincial-district-0 15. SAARC Business Association of Home Based Workers (SABAH). (2010). Exploratory Research. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 32 16. Sen, Amartya. (1990). More than 100 million women are missing. New York Review of Books. Retrieved from http://www.nybooks.com/articles/archives/1990/dec/20/more-than￾100-million-women-are-missing/?pagination=false 17. Sudarshan, R. M.& Sinha, S. (January 2011). Making home-based work visible: a review of evidence from south Asia. Women in Informal Employment: Globalizing and Organizing (WIEGO) Urban Policies Research Report No. 10. Retrieved from http://www.inclusivecities.org/research/RR10_Sudashan%20revert2.pdf 18. Sustainable Development Policy Institute (SDPI). (n.d.). Pakistan: country gender profile, final report, Nov. 2007-July 2008. Islamabad, Pakistan. Retrieved from http://www.jica.go.jp/pakistan/english/office/others/pdf/CGP_01.pdf 19. Water, Environment and Sanitation Society (WESS). (October, 2011). Women’s Economic Empowerment: Balochistan (WEE:B) – Targets vs achievements report, 14th October 2011. WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 33 ANNEXES Annex 1: Complete List of Products Produced by WEGs List of Products (WEE:B Project) S.No Name of products 1 Pathani Suit (small & large size) 2 Balochi Suit (small & large size) 3 Simple Suit (small & large size) 4 Shawls 5 Dopatas 6 Decoration Pieces 7 Hand Fan 8 Mirrors 9 Zartar Paties 10 Balochi Paties 11 Ribbon Paties 12 Pouch 13 Bags 14 Mobile Covers 15 Key Chain 16 Door bells 17 File Covers 18 Photo frame 19 Bangles 20 Cushion set 21 Bed Sheets 22 Wool Frocks 23 Koreshia Table Mates 24 Koreshia Sweaters 25 Wall Hanging 26 Zartar Gala 27 Balochi Gala WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 34 Annex 2: List of Project Documents Reviewed for the Evaluation 1. Integrating women into a sustainable value chain in Balochistan, draft – baseline survey report. (January 2010). Mennonite Economic Development Associates (MEDA). 2. Women’s economic empowerment: integrating women into sustainable value chains in Balochistan – project implementation plan – June 2009. (June 2009). Mennonite Economic Development Associates (MEDA). 3. Women’s economic empowerment: integrating women into sustainable value chains in Balochistan” – annual assessment report. – August 2011. (August 2011). Mennonite Economic Development Associates (MEDA). 4. Women’s economic empowerment: integrating women into sustainable value chains in Balochistan” – Work Plan 2011. (2011). Mennonite Economic Development Associates (MEDA) & Water, Environment and Sanitation Society (WESS). 5. Women’s Economic Empowerment: Balochistan (WEEB) – Marketing strategy. Mennonite Economic Development Associates (MEDA) & Water, Environment and Sanitation Society (WESS). 6. Women’s Economic Empowerment: Balochistan (WEEB) – consolidated sale and order form, April 2010 – July 2011. (July 2011). Water, Environment and Sanitation Society (WESS). 7. Women’s Economic Empowerment: Balochistan (WEEB)” – Profile of buyers from local markets (Quetta). Water, Environment and Sanitation Society (WESS). 8. “Economic Growth Weekly Report 11-18 Nov, 2011” - Reported to USAID 9. Women’s Economic Empowerment: Balochistan (WEE:B) – Targets vs achievements report, 14th October 2011. (October, 2011). Water, Environment and Sanitation Society (WESS). WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 35 Annex 3: Survey Questionnaire Evaluation of Women’s Economic Empowerment in Balochistan MSI Questionnaire 2011 English (WEE-B beneficiaries) WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 36 PERSONAL INFORMATION Name of respondent [should be the senior most registered member in the household]: Respondent ID: Age: 1. 15 – 24 2. 25 - 34 3. 35 – 44 4. 45 – 54 5. 55 – 64 6. 65 – 74 7. 75 and above Education: 1. Illiterate 2. Literate but no formal education 3. Primary 4. Middle 5. Matric 6. Intermediate 7. Graduate 8. Post-graduate 9. Professional (Doctor, Engineer, Teacher, Lawyer etc.) Location (City/Village): 1. City 2. Village District: 1. Quetta 2. Loralai 3. Zhob Tehsil: Language’s: [spoken by respondent] 1. Urdu 2. Punjabi 3. Pushtu 4. Sindhi 5. Balochi 6. Sraiki 7. Other: Location Name (City/Village): Address: Start of interview: End of interview: Date of interview (DD/MM/YYYY): Name of interviewer: Remark: WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 37 Section–A Respondent Background D1 What is your current marital status? 1. Single 2. Married 3. Divorced 4. Widow D2 Who was the main earner in the household during the last 12 months? 1. You   [go to D4] 2. Someone else [go to D3] D3 Specify the main earner’s name, relation with you and occupation? [referred below to the code list for relation and occupation] Name: Relation: Occupation: WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 38 D4 What was the respondent’s primary source of income during the past 12 months? 1. Knitting 2. Poultry raising (eggs or meat) 3. Livestock raising or marketing 4. Shop keeping 5. Formal salary/wages 6. Casual labor/piece work 7. Begging/gifts 8. Wool spinning 9. Wool Shearing 10. Embroidery 11. Selling processed agricultural products 12. Rug making 13. Kitchen gardening 14. Others [specify]: Relationship to household Main earner 1 = Household head 2 = Spouse 3 = Child/step child 4 = Parent 5 = In-law 6 = In-law 7 = Other relative (brother, sister, Cousin, etc.) 8 = Worker Other:_________________ Main Earner Occupation 1 = Agriculture 2 = Livestock 3 = poultry 4 = Petty trade 5 = Labor 6 = Govt. Service 7 = Private Service 8 = Own Business Other:______________ WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 39 Section–B Change in income attributable to the project Q1. How many years have you have been producing embroidery products? Years: Q2. How many years you worked with this project? Years: Q3. Who are the other members of the family who are also doing the same work? Number of members: WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 40 Q4. S # Products These questions are about your embroidery work since Jan 2010. These questions are about your embroidery work in all 12 months of 2009, before the WEE-B project started. Q4a. How many products did you make? Q4b. How many days (elapsed) did it take you to make the last product you made? Q4c. How much money did you receive for selling the last product you sold? (Rs.) Q4d. How much do the inputs cost for the last products you made? (Rs.) Q4e. How much commission did you pay for the last product you sold? (Rs.) Q4f. Did you make more, less, or about the same number of products in 2009? 1. More 2. Less 3. Same 4. Do not remem ber Q4g. How many products did you make in 2009? 4.Do not remember Q4h. Did you receive more, less or about the same amount of money for each product now than in 2009? 1. More 2. Less 3. Same 4. Do not remember Q4i. How much did you receive for each product in 2009? (Rs.) 1 Pathani Suit 2 Balochi Suit 3 Simple suit 4 Shawls 5 Dopatas 6 Decoration pieces 7 Hand fan 8 Mirrors 9 Zartar paties 10 Balochi paties 11 Ribbon paties 12 Pouch 13 Mobile covers 14 Key chains 15 Door bells 16 File covers WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 41 17 Photo frame 18 Bangles 19 Cushion set 20 Bed sheets 21 Wool frocks 22 Koreshia table mates 23 Koreshia sweaters 24 Wall hanging 25 Zartar gala 26 Balochi gala 27 Bags WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 42 Q5. What was the gross value of your sales since Jan 2010? Value of sales (Rs.): Q6. On annual basis is your earning more or less from embroidery since the project started? 1. More [go to Q7] 2. Less [go to Q8] 3. About the same [go to Q9] 4. Don’t know [go to Q9] Q7. What were the two most important reasons you earned more? [Multiple responses are allowed] 1. Made more pieces 2. Received higher prices for what you sold 3. Made more expensive items 4. Improved quality of your items 5. Spent more on inputs 6. Selling in different places 7. Other [Specify]: Q8. What were the two most important reasons you earned less? [Multiple responses are allowed] 1. Made fewer pieces 2. Received low price for what you sold 3. Made less expensive pieces 4. Reduce quality of your items 5. Spent less on inputs 6. Selling in different places 7. Other [Specify]: WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 43 Q9. How many days in the last one month did you work on embroidery? Number of working days in a month: Q10. How many hours per day do you spend on embroidery work? Hours: Q11. Compared to 2009, has the number of women in the household who are making and selling embroidery increased, decreased or remained the same? 1. Increased [go to Q12] 2. Decreased [go to Q13] 3. Remained same [go to Q14] Q12. What were the two most important reasons for increase? 1. 2. Q13. What were the two most important reasons for decrease? 1. 2. Q14. How do you get inputs or supplies, now and in 2009 before the WEE:B project started? Now 2009 Get them yourself 1 1 Family members get them 2 2 Sales agent 3 3 Q15. How do you sell your products, now and in 2009 before the WEE-B project started? Now 2009 Sell yourself 1 1 WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 44 Sell through community organization member 2 2 Sell through a Sales agent 3 3 Q16. Which members of your household decide how the money you earned from embroidery is used? Mainly men 1 Mainly women 2 Both men and women 3 Q17. How do you use the income you earn from embroidery work? 1. Food 2. Clothing 3. Health 4. Education 5. Reinvestment in business 6. Other [specify]: Q18. Compared to 2009, … 1. Yes (go to Q19) 2. No Q19. Has it increased, decreased or stayed the same? Q20.To what extent was income from embroidery responsible for the change? 1. Increased 2. Decreased 3. Same [if increased in any Q19 go to Q20] completely partly Not at all WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 45 Q18a. Has the respect you get within the household changed? 1 2 3 Q18b. Has the respect you get within the community changed? 1 2 3 Q21. Has the quality of inputs you use in embroidery increased or decreased as compared to before project? 1. Increased [go to Q22] 2. Decreased [go to Q23] Q22. If increased, what were the two most important reasons you use higher quality inputs now? 1. 2. Q23. If decreased, what were the two most important reasons you use lower quality inputs now? 1. 2. Section–C (Respondent’s perception of the project and the market) Q24. In 2009 and before, where did you sell most of your embroidered products? 1. In village 2. Outside village 3. Other [specify] Q25. Where do you sell most of your products now? 1. In village 2. Outside village 3. Other [specify]: WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 46 Section–D Experience with trainings Q26. Type of training Q26a. Have you received this kind of training? [Yes/No - If no go to Q27] [If yes continue] Q26b. Did the WEE.B project provide the training? [Yes/No – if no go to Q26d] Q26c. How effective was the training in building your skills? 1. Very effective 2. Effective 3. Not effective 4. Don’t know Q26d. Did you receive the training through an institute? [Yes/No] [Yes go to 26e] [no go to 26f] Q26e. How effective was the training in building your skills? 1. Very effective 2. Effective 3. Not effective 4. Don’t know Q26f. How, if at all, did the training affect your income from embroidery? 1. Increased 2. Decreased 3. No change 4. Don’t know Embroidery skills (quality) Awareness of market Demand (designs) Q27. If you never got any skill training, are you willing to get any skill training? 1. Yes [go to Q28] 2. No [go to Q29] Q28. Where would you like to attend this training? 1. In your area 2. Somewhere else Q29. How satisfied are you with the WEE.B project? Very satisfied Satisfied Unsatisfied Don’t know WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 47 Q30. Do you know who is funding this project? 1. Yes [If yes go to Q32] 2. No [if no go to 33] Q31. Do you know the name of the donor? 1. USAID 2. FAO 3. WESS 4. MEDA 5. Government 6. Other. 7. Don’t Know Q32. What was your total household income from all sources in the past 12 months? [instruct the enumerators to ask the ranges below rather than asking for a single number] 1. <3,0000 2. 30,001—70,000 3. 70,001—10,0000 4. 10,0001—15,0000 5. 15,0001—30,0000 6. >30,0000 7. Don’t Know 8. No Response WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 48 Annex 4: Questions for FSA Group Discussion FSA Selection, Training and Roles 1. Who selected you as FSA? And why? 2. How did the Project train you? How was it helpful to you and your group? 3. What are the most important things you do as an FSA for: a. The producers in your group? b. The shopkeepers? c. yourself and your family? Project Benefits 4. What are the benefits of the project, in terms of: a. The skills of the producers? b. The quality of the inputs? c. The quality of the products? d. The incomes of the producers? e. Your income? f. The sales of shopkeepers? Understanding with Family 5. What help do you get from your family? 6. Do you retain your income? How do you spend it? 7. What do your family members and neighbors think of your work? 8. Are you satisfied with your role as an FSA? Group Dynamics 9. Has anyone in their group faced any competition in terms of others asking to be FSAs? 10. What would you do if someone else in the group wanted your position? Sustainability 11. What would happen if the WESS project comes to an end? Would you still continue doing your work in a similar manner? Knowledge of US Assistance 12. Do you know who is funding the WESS project? WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 49 Annex 5: Questions for Shop Keepers 1. What do you mostly order from the FSAs?   2. How was demand for embroidered Shawls, Chaddars and Joray before January 2010? How do you see it changing in the future? 3. Are tourists interested in buying traditional embroidery garments? 4. How would you rate the quality of the products sold by FSAs on the following parameters? (options: Best, Good, Average, Poor) a. Design: b. Color Combination: c. Thread Quality: d. Cloth Quality: 5. How would you compare the quality of the products sold by FSAs to those by others? (options: Better, Worse) a. Design: b. Color Combination: c. Thread Quality: d. Cloth Quality: 6. What do customers say about the products made by FSAs? Have you received complaints? WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 50 Annex 6: Description of Selected WEE:B Products Introduction Embroidery is the art and craft of decorating fabric or other materials with needle and thread or yarn. It may include materials such as metal strips, pearls, beads, quills, and sequins. A basic characteristic of embroidery is that it entails techniques or stitches of the earliest work, such as chain stitch, buttonhole or blanket stitch, running stitch, satin stitch and cross stitch – all which remain the fundamental techniques of hand embroidery. Below is description of a few hand-embroidered products commonly used in Balochistan: Chader Product: A chader is a rectangular piece of cloth usually made from Japanese, Thai, or Korean fabric with embroidered flowers on its borders. Handmade chaders with Balochi, Afghani, and Uzbaki Tanka (stitches) with cross stitch and ribbons are popular among women and are in high demand. The price of a chader depends on the type, design, and amount of embroidery work and typically ranges from PKR 500 to 5,000. Users: Chaders are popular among women from all walks of life and ages (and they usually start wearing it from age 12). In Balochistan, almost 95% of women use chaders to cover their head and body. Men (usually older men) use chaders (to put it on their shoulders) as a symbol of Baloch and Pashtun culture. Balochi Dress Product: Balochi and Brahvi dresses are quite similar and have few variations. Wide loose shalwar (similar to loose trouser) and knee-long shirts are worn by many women. Female dresses typically consist of a shirt with a front pocket. The shirt usually has embroidery work with embedded small round mirror pieces or with different varieties of tankas including Bugti, Haft Ranga, Zanjeeri, Fish and Mosam etc with large ‘Dopatta’ or ‘Chader’. The upper part of the dress and sleeves is decorated with needlework, a form of artistry that is specific to the clothing of the Baloch women. Balochi dress is made from resham (silk fabric) and is expensive because of its embroidery and embedded mirror work. Users: The dress is usually used by Balochi and Brahvi women and is a prominent feature of Balochi culture. These dresses are also prominently featured in Iranian culture and society. Often the dress contains round or square pieces of glass to further enhance the presentation. Women cover their hair with a scarf, called sarig in the local dialect. These customs are unique to the people and the art of this needlework on women's clothing is considered as a symbol of high status for Balochi women. The prices of these dresses range typically range from PKR 5,000 to 100,000. Balochi/ Pathani Gala/patti (simple suit) Product: Balochi/Pathani Galas, Patties( embroidered stripe) are in fashion and demand as compared to fully embroidered Balochi dresses. Women prefer to use embroidered stripes adorned with beautiful and colorful thread Tankaas, embellishment on the neckline of plain simple suit. Embellishment work on simple suit includes Balochi, Afghan, Uzbek, and beads pattis (Stripes) WOMEN ECONOMIC EMPOWERMENT: BALOCHISTAN – EVALUATION REPORT 51 Users: These embroidered stripes are common these days and considered trendy and fashionable when used with plain suits among local, visitors (tourist) and foreign women. These are used by women of every age, class and profession. The dresses are cheaper as compared to full Balochi dress and range from PKR 250 to 1,200. Embellished Bags and Bangles Product: Many of these handmade bags and bangles incorporate colorful antique fabrics into their designs. This is a clever and creative way to reuse old fabrics or to do it on different colorful fabrics like jamawaar, cotton, khaadi etc. Many of these are magnificent works of art. Tankaas on these bags are usually ooni (woolen) tanks or mostly Qoreshia work. Users: Bangles and bags are popular among women particularly young girls from age 13 to 30. They are available in market with a variety of designs and styles and are used as unique items for gifts. Prices of these bags and Bangals vary from place to place: prices of bags range from PKR 500 to 2,000, and bangals range from Rs 150 up to 300. Pathani Frock Type of Embroidery Product: The Pashtun dress of Afghanistan and Pakistan is typically made from light linens, jorget, jamawar, banarsi, kimkhaab and velvet, and are loose fitting for ease of movement. The Pashtun dress includes shalwar kameez, which is differently made for males and females. Males usually wear kufi, Kandahari cap, turban or pakul as traditional headgear. Women typically wear solid-coloured trousers, a long qemis shirt/frock with a belt and a fancy dopatta. More elaborate and fancier dresses are detailed with gold threading, gold beads, and come in many different colors on silk fabrics. These dresses are usually worn for special occasions and weddings, and are common among rural women. Afghan/Pashtun dress is also notable for its embroidery. Embroidery styles tend to be associated with geographic regions and ethnic groups. Whether from Balochistan, Kandahar, or Kabul, regional associations are made. Styles generally are distinguishable by the fiber content of the fabric (plain weave cottons, pile woven velvets, or synthetic satin weaves) as well as the kind of thread (cotton, silk or gold metallic threads); a variety of embroidery techniques and the complexity of their execution; the floral and geometric motifs; and the design placement of the embroidery. Three such embroidery styles are the gold stitched embroidery or chirma dozi, known for the unique kind of metallic thread and braid used; tashamaar dozi, recognizable by the intricate counted stitch technique; and silk stitched flower embroidery or gul dozi, distinctive because of the rich use of colored threads. Users: Dress also differentiates the age and generational status of the wearer. For example, though all females wear pants, overdress, and head and foot coverings, more costly materials and surface design embellishments are added to women's dowries. The decorative focus is on pants cuffs, dress bodices, and head covering borders as female age and gain more status when they become engaged, marry, and become mothers. Afghan and Pashtun dresses are considered a symbol of pride, modesty, and dignity among women and their prices range from PKR 6,000 and 50,000. 52 Annex 7: WEE:B Budget and Expenditure as of 30th June, 2011 TABLE 11: PROJECT BUDGET AND EXPENSES Budget item Budget Expenditure Allocated budget through June, 2011 Budget utilization through Year 1 Year 2 Year 3 June, 2011 Total Budget Up to March 31 April - June 2010 July - Sept 2010 Oct - Dec 2010 Jan - Mar 2011 April - June 2011 Total expenditure through June, 2011 Total personnel costs 19,669 82,602 86,931 189,197 28,354 28,674 20,018 26,195 17,530 25,048 145,818 145,736 100% Total partner costs 60,000 235,000 215,000 510,000 35,544 47,955 20,935 21,371 36,784 76,306 238,895 402,500 59% Total travel 26,000 26,000 23,000 75,000 3,038 4,396 1,907 6,102 4,004 4,372 23,818 63,500 38% Training 4,500 18,000 18,000 40,500 1,049 1,155 56 -56 39 0 2,242 31,500 7% Total equipment cost 64,000 - - 64,000 1,886 1,053 - - - 20,161 23,100 64,000 36% Total support technical services 20,000 90,000 90,000 200,000 860 35,966 2,458 25,585 32,273 7,385 104,527 155,000 67% Total general operating expenses 3,000 17,325 18,379 38,703 15,135 -1,715 985 3,855 10,669 221 29,149 29,515 99% Totals 197,169 468,927 451,310 1,117,400 85,866 117,483 46,359 83,050 101,298 133,493 567,549 891,751 64% 53 Annex 8: Targets Versus Achievements as of 30th December 2011. Activity no Area/Intervention Project target Achievements Percentage Achieved Outcome 1: Increased production capacity and quality of produce made by women embellishers in targeted districts of Northern Balochistan 1.1.1 Women Embellishers Identification 5,500 7,306 133% 1.1.2 Registered WEs 5,000 5,096 102% 1.1.3 Identify and orient FSAs 200 220 110% FSA registration 200 207 104% 1.1.4 Group formation 200 226 113% Orientation session with WE and FSAs 0 370 FGDs 0 177 Follow up meetings with group 0 2,352 1.2.1 Number of FSAs participated in improved production skills in sub- sector embellished garments i.e. designing and tracing etc. 200 226 113% Number of designing and tracing training events organized 20 23 115% 1.3.1 Number of FSAs participated in product development and design training 200 215 108% 54 Activity no Area/Intervention Project target Achievements Percentage Achieved Number of product development and design training organized 20 19 95% 1.3.2 Number of FSAs motivated to closely work with WEs to produce various designs of products; following the market trends 200 345 173% Number of sessions conducted to motivate FSAs to work with embellishers 20 37 185% 1.3.3 Number of FSAs trained to assist WEs to produce consumers' demanded quantity and designs of the products 200 224 112% Number of training events conducted to facilitate FSAs to assist WEs in producing demand driven products 20 35 175% 1.4.1 Number of FSAs particiapted in session on the use of high quality inputs along-with improved production practices 200 244 122% 55 Activity no Area/Intervention Project target Achievements Percentage Achieved Number of sessions conducted to motivate FSAs on the use of high quality inputs along-with improved production practices 10 35 350% 1.4.2 Number of FSAs provided with update index of quality input suppliers 200 222 111% Number of session codnucted to distribute input index 7 15 214% Outcome 2: Women producers linked with sustainable markets and support services 2.1.1 Facilitate FSAs in exploring potential local markets; (No. of participants in exposure visit ) 200 229 115% 2.1.2 Facilitate and organize exposure visits (No. of visits Local) 20 56 280% 2.1.3 Number of FSAs facilitated to participation in exhibitions 10 77 770% Number of exibitions identified in which FSAs participated 0 11 2.1.3 a Number of FSAs particpated in WESS exibitions 0 69 Number of events organized 0 3 56 Activity no Area/Intervention Project target Achievements Percentage Achieved 2.1.4 Organize buyers-sellers meetings 40 265 663% 2.1.5 Organize and facilitate stakeholders' workshop 10 10 100% 2.1.6 Facilitate FSAs to explore possible nation wide high end markets (no. of participants in visit) 40 42 105% 2.1.7 Facilitate and organize exposure visits (nationwide) no. of visits 5 12 240% 2.1.8 Facilitate FSAs participation in exhibitions (nationwide) 5 5 100% 2.2.1 Aware the FSAs/ WEs about existing Business Development Services and Providers 200 205 103% 2.2.2 Conduct meetings with FSAs to make them aware about BDSs service providers 10 84 840% 2.2.3 Organize and facilitate stakeholders' workshop i.e. for FSAs with BDS providers 10 10 100% 2.2.4 Number of FSAs participated in session on ways and means to access BDSs 200 227 114% Number of session organized 10 17 170% 57 Activity no Area/Intervention Project target Achievements Percentage Achieved 2.2.5 Distribute material related to BDSs among FSAs 4 98 2450% Outcome 3: Developed capacity of local and national level partner organizations/ institutions to implement market driven Embellished Garments Value Chain Development Project 3.1.1 Conduct series of meetings with potential Balochistan based organizations for implementation of project 8 11 138% 3.1.2 Conduct meetings with national level experienced organization/ institution having greater understanding of entrepreneurship development 1 2 200% 3.1.3 Establish partnerships with local and national level organization for the effective implementation of the project 1 1 100% 3.2.1 Conduct Training Needs Assessment (TNA) for the partners 2 2 100% 3.2.2 Number of project staff members participated in VC development trainings 14 25 179% Number of VC trainings/ events organized 8 2 25% 58 Activity no Area/Intervention Project target Achievements Percentage Achieved 3.2.3 Extend technical assistance and mentoring to the partners during the course of project implementation 24 27 113% 59 Annex 9: Change in Production, Before and After the Project TABLE 12 : CHANGE IN PRODUCTION BY ITEM Name of Product Average Prod. (Before) Average Prod. (After) Difference 1 Pathani Suit (small & large size) 0.09 10.72 10.63 2 Balochi Suit (small & large size) 0.37 1.99 1.63*** 3 Simple Suit (small & large size) 1.17 6.52 5.35*** 4 Shawls 0.67 4.54 3.87*** 5 Dopatas 0.96 51 50.04* 6 Decoration Pieces 0.06 3.48 3.42* 7 Hand Fan 0 2.98 2.98 8 Mirrors 0 0.63 0.63 9 Zartar Paties 0.07 1.89 1.83*** 10 Balochi Paties 0.35 9.3 8.95** 11 Ribbon Paties 0.33 8.35 8.02** 12 Pouch 0.1 12.47 12.37*** 13 Bags 0.24 7.23 6.99*** 14 Mobile Covers 0.05 5.34 5.29** 15 Key Chain 0 0.42 0.42* 16 Door Bells 0 1.28 1.28** 17 File Covers 0.05 0.37 0.32** 18 Photo Frame 0.48 12.7 12.22** 19 Bangles 0.34 4.68 4.35** 20 Cushion Set 0.11 0.94 0.84** 21 Bed Sheets 0.29 0.76 0.47 22 Wool Frocks 0.03 0.03 0 23 Koreshia Table Mats 1.48 0.06 -1.42 24 Koreshia Sweaters 0 0.08 0.08 25 Wall Hanging 0.06 1.27 1.21** 60 26 Zartar Gala 0.31 10.03 9.72*** 27 Balochi Gala 0.12 4.77 4.65* Note: For each product, the average was calculated at a unit basis for the sample of 104 respondents, also taking into account those respondents who are not producing that particular product. ***Significance at 1 % level **Significance at 5 % level *Significance at 10 % level 61 Annex 10: Email from FAO Regarding WEE:B’s Reporting and Branding Performance From: Asif Rasheed [mailto:asif.rasheed@fao.org] Sent: Monday, December 12, 2011 1:24 PM To: Imran Khan Cc: ahmed.essa@fao.org; Tariq Husain; thusain@edcpk.com Subject: RE: Evaluation of the WEE:B Project - Queries for FAO Dear Mr. Khan, 1. Reporting as per the donor’s requirement i.e. weekly, quarterly and annual is being furnished by the WEE:B component which is timely, regular and satisfactory. 2. Approval is sought for the significant events whereas the branding guidelines of USAID have also been provided to the concerned. Best Regards, Asif From: Imran Khan [mailto:ikhan@msi-pak.com] Sent: 11 December 2011 5:25 PM To: asif.rasheed@fao.org Cc: ahmed.essa@fao.org; Tariq Husain; thusain@edcpk.com Subject: Evaluation of the WEE:B Project - Queries for FAO Importance: High Dear Mr. Rasheed, This in reference to the ongoing evaluation of the Women Economic Empowerment: Balochistan (WEE:B) Project. The MSI-IMEC has been authorized by the USAID to carry out this evaluation. The reporting performance of the project as well as their adherence to branding guidelines are two of the criterions upon which the project will be evaluated. 62 In this regard, we were told by the Project that they report on weekly, monthly and annual basis to MEDA, which then reports to the FAO, and that all banners are approved by the FAO before they are sent for printing. We had two queries in this regard and they are as follows: 1‐ Are you are satisfied with the punctuality as well as quality of the reporting? And do you have any recommendations for improvement. 2‐ Are the banners approved at the FAO office before being sent for printing? And also is this approval procedure also followed for other media and communication related activities such as newspaper ads etc? We look forward to your reply, please feel free to call me for any clarifications. Regards, Imran Khan Economist MSI-IMEC +92-321-9555035 63 Annex 11: Email from FAO Regarding Coordination Between WEE:B and US-ABBA From: Ahmed J. Essa [mailto:ahmed.essa@fao.org] Sent: Thursday, December 15, 2011 12:03 AM To: Imran Khan Cc: Tariq Husain; thusain@edcpk.com Subject: RE: Coordination between US-ABBA and WEE:B Dear Imran Sahib, Sorry for the delayed response but I was upto my eyes during the past couple of days due to so many events taking place at the same time. The Management of FAO is extremely satisfied with the coordination that both the teams have at the field level. I must mention some of those: a. Monthly Progress Review Meetings at the district level b. Monthly or sometime quarterly review meeting at the Project Head Office level (Quetta) c. Coordination meetings at the Country Level (Islamabad) d. Annual Progress Review/Workplanning Consultations Besides, we always invite WESS/MEDA staff in the training events that are arranged for the project staff by the international consultant at Quetta. Warm regards, Ahmed From: Imran Khan [mailto:ikhan@msi-pak.com] Sent: Sunday, December 11, 2011 8:12 PM To: ahmed.essa@fao.org 64 Cc: Tariq Husain; thusain@edcpk.com Subject: Coordination between US-ABBA and WEE:B Importance: High Dear Mr. Essa, This in reference to the ongoing evaluation of the Women Economic Empowerment: Balochistan (WEE:B) Project. The MSI-IMEC has been authorized by the USAID to carry out this evaluation. The coordination of the WEE:B Project with the US-ABBA project is one of the factors being considered in the evaluation. We just wanted to know if you are satisfied with the level of coordination between these two projects, and if you have any suggestions for improvement? Regards, Imran Khan Economist MSI-IMEC +92-321-9555035 Haroon, Jamal, and Amir Jahan Khan. Trends in Regional Human Development Indices. Islamabad, Pakistan: Social Policy and Development Centre (SPDC), 2007. Print. 65