CORAD DAP PROGRAM END-OF-PROJECT EVALUATION Qualitative Assessment Report Africare Care International CRS World Vision Submitted by: Priti Biswas, Consultant, TANGO Sierra Leone April 2006 Table of Contents Acknowledgements........................................................................................................3 Acronyms.......................................................................................................................4 Introduction....................................................................................................................5 Evaluation Objectives....................................................................................................5 Evaluation Methodology................................................................................................6 Sample of Villages.....................................................................................................6 Qualitative Assessment Tools....................................................................................6 Assessment Findings......................................................................................................7 Objective 1: Health ........................................................................................................8 Access to Health Services..........................................................................................8 Improved Health and Nutrition Practices ..................................................................8 Decision-making and Governance in Health .............................................................9 Objective 2: Food Security ..........................................................................................10 Farm Productivity ....................................................................................................10 Reducing Pre and Post-harvest Losses ....................................................................11 Re-establishing Market Linkages ............................................................................11 Village-level Capacity-building for Agriculture......................................................12 Decision-making and Governance in Agriculture ...................................................12 Maintaining a Safety Net .........................................................................................13 The CORAD Approach............................................................................................13 Monitoring and Evaluation ......................................................................................13 Gender......................................................................................................................14 Lessons Learned and Recommendations.....................................................................14 The Community Approach: the Need to Guarantee Social Inclusion.....................14 VDC Governance: Reinforcing Local Capacity ......................................................14 Farmer Field Schools...............................................................................................15 Targeting..................................................................................................................15 Post-harvest Losses..................................................................................................16 Gender Training.......................................................................................................16 Environment.............................................................................................................17 Monitoring and Evaluation ......................................................................................17 Conclusion ...................................................................................................................17 List of Individuals and Groups Interviewed ................................................................19 Documents Consulted ..................................................................................................20 2 Acknowledgements I would like to express my earnest gratitude to Kumba Koedoyoma, (World Vision), Ali K. Guteh (World Vision), Tabma Nelson (World Vision), Gibril Bangura (CARE), Tamba Dalton Kassoh (CARE), Veronica Smith (Africare), Joseph Juana (CRS) for all their hard work and dedication to the objectives of this mission. My very special thanks go to Myles Harrison (World Vision) and Fred Goba (World Vision) for their constant help and support throughout my time in Sierra Leone. And I am especially appreciative of all the people whom I met and consulted during this assignment for their time and critical reflection on the program. Most importantly, I acknowledge the contribution of the many community members from the villages that generously received me, gave freely of their time, and openly shared and discussed their invaluable experience and knowledge with the team. 3 Acronyms 4 Introduction The Consortium for Rehabilitation and Development (CORAD) has been implementing a 3- year Development Relief Program (DRP), originally referred to as the Transition Assistance Program (TRP) since March 2004. The goal of the program is “to contribute to the restoration of livelihoods and the food security of rural households in 29 focal chiefdoms in the five districts of Koinadugu, Kono, Kailahun, Tonkolili and Bonthe”(CORAD 2003). The DRP was designed to reestablish livelihoods of poor and vulnerable households in some of Sierra Leone’s worst civil war affected areas and do so in ways that will facilitate a transition from relief to development programs. This approach includes the establishment of long-term food security interventions that encompass availability, access and utilization but which retain a targeted food assistance strategy. Such initiatives were designed to promote social and economic conditions that enable individuals to gain access to food, either through producing it themselves or by earning the income to purchase it. The DRP has two program objectives designed to contribute to the accomplishment of this goal through improving health status and reestablishing livelihood activities associated with agriculture. Under the first program objective, CORAD is implementing a series of health￾focused interventions that will contribute to improve the health status of 15,800 rural food insecure households through: improved access to comprehensive maternal and child care services (IR 1.1); improved household knowledge related to improved health and nutrition practices (IR 1.2); strengthened capacity of village-based organizations (IR 1.3); and the reestablishment of linkages between village-based health organizations and policy-makers at district and national levels (IR 1.4). Under the second program objective, the DRP contributes to improved food supply and access to food by 37,400 rural food insecure households. This result will be achieved through six distinct but interrelated strategies: restoration of production on in-land valley rice farms, tree farms or upland farms (IR 2.1); rehabilitation of community-managed drying, storage and processing facilities (IR 2.2); improved access to local markets (IR 2.3); capacity enhancement for agricultural-focused village-based organizations (IR 2.4); the reestablishment of linkages between village-level agricultural groups and decision-makers at chiefdom and district levels (IR 2.5) and; addressing the immediate food and shelter needs of vulnerable households and people in institutions (IR 2.6). During the first quarter of the project, a baseline survey was conducted to gather information at project and program level. Since the project has implemented the second year of planned activities, the evaluation assesses the progress and impact of the program to date. This assessment thus provides a qualitative analysis of local livelihoods to complement the quantitative survey that has been completed. Employing qualitative instruments, the assessment has compiled and compared a variety of perspectives from a wide range of project stakeholders, including program beneficiaries in villages, CORAD implementing partner personnel at several levels, Ministry of Health and Ministry of Agriculture staff, the representative of USAID, and project implementers. Evaluation Objectives The four key objectives of this evaluation are: (a) Provide a comprehensive, independent and external assessment of the DRP design and planning processes. At this level, the evaluation will specifically accomplish two tasks: (1) assess the relevance of program strategies to beneficiary needs and priorities as well as government programs for targeted areas and; (2) assess the degree of participation by key stakeholders in the planning of the DRP. 5 (b). Provide a comprehensive, independent and external assessment of the DRP implementation and performance management operations in order to assess: (1) program targeting criteria and processes; (2) program effectiveness by documenting key achievements to date; and (3) the efficiency of program operations including the effectiveness of program-level units (CCU and M&E). (c). Provide a comprehensive, independent and external assessment of the DRP program end and exit strategies in order to assess: (1) behavioral changes and impacts created by the program; and (2) the sustainability thereof (key outputs, behavioral changes and impact). (d). Document key lessons learned from DRP planning, implementation and performance management, and program exit strategies and generate recommendations that will inform the planning and management processes of MYAP and other future similar programs. Evaluation Methodology This qualitative assessment was designed to collect in-depth information regarding the impacts of DRP interventions on the targeted population of poor and vulnerable households in rural Sierra Leone. This assessment focuses not only on the targeted outputs of the last two years, but also additional factors that hinder the progress and well-being of these vulnerable communities. It offers insights into the village-level reality of war-torn poor households, community empowerment dynamics, food insecurity, and local coping mechanisms. The categories of information that were collected included, at the project beneficiary level, the contextual information on the political, social, economic, and natural environment in which households make decisions, the process, reasons, motivation behind participation in the program for different individuals and households, and the impacts of the program on health and food security for participating households. For other stakeholders, the information included the process, strengths, limitations, lessons learnt and future directions for the program. Sample of Villages Qualitative interviewing was carried out in ten project villages within four districts — Tonkolili, Koinadugu, Kailahun and Kono. The sample was designed to cover a range of factors that might influence project success, including the geographic location, ethnic composition, the number of households in the villages, and the relative success in the adoption of project interventions. There were also differences in road access and remoteness among the villages. Nine out of these ten villages were randomly selected from the number of villages that were surveyed during the quantitative survey. At least two villages were selected from each CORAD partners implementing areas. Qualitative Assessment Tools The overall methodology adopted a participatory approach designed to elicit local reflections on the direct and indirect impacts of the project. In doing so, a number of PRA tools were employed including focus group discussions, key informant interviews, and in-depth interviews with different stakeholders. In addition, seasonal calendars, well-being analysis, and transect walks were also applied in each village. The transect walks identified the spatial organization of the village and its relationship to the fields, water bodies, and other resources 6 Table 1.1 List of villages in qualitative assessment Districts Village (implementing partner) Tonkolili Rogbesseh (CARE) Matham (CARE) Koinadugu Helemakona (CARE) Kamaron (CRS) Badala (CRS) Kailahun Manowulo (CRS) Darujanction (Africare) Banda Jume (Africare) Kono Manjama (World Vision) Condama (World Vision) as well as Farmer Field School (FFS) demonstration plots, community health centres, growth monitoring centers, water wells, reestablished village markets, and other infrastructural features. During the focus group discussions, seasonal calendars were constructed to identify temporal variation in food availability and accessibility, income, and labor demands, disease incidence, and other factors that significantly affected the local community. The well-being analysis provided a critical village profile on socio-economic stratification and general characteristics about the background of program beneficiaries. Focus group interviews with women and men were carried out separately and in mixed groups to investigate the level of participation, representation, and intended and unintended impacts of the project. In all the visited villages, the receptiveness of the local population was very high and participation was widespread. Careful analysis of the interview texts and the observational data was oriented towards persistent thematic trends and patterns in response. Assessment Findings The CORAD DRP was a pioneer program designed to improve the health status and food security in the areas severely affected from over ten years of war. The implementing consortium was formed with four international NGOs working in–country, and much effort was invested in developing a common framework for taking the program forward. There are three key challenging features of this program which need to be restated. First, the prolonged and brutal warfare introduced overwhelming livelihoods shocks and traumatized the collective psyche of the population. The physical structure and infrastructure of rural society were destroyed and remain largely absent, and investment in human capital through skill￾building and education was paralyzed. The war made the country a “fragile state.” The second challenging factor is that the CORAD approach represents the first time all four international NGOs in the country have come together to form an effective and efficient partnership approach that builds upon on each other’s strengths. The third challenging factor is the shift of intervention designs from one of emergency relief to that of creating a supporting foundation for sustainable development. Despite difficult odds, the program has made a significance contribution in the area of its intended objectives of health and food security over the last two years. Nonetheless, there are issues and concerns that require further discussion in order to improve the overall performance of the program throughout its lifetime. 7 Objective 1: Health Access to Health Services Access to health care in the programme area—especially with regard to reproductive health for women—has been limited in two major ways. First, the GOSL health structure extends only as far as district towns; and due to difficult roads and the absence of communication, these facilities are seldom sought out by village inhabitants. Second, women’s body and reproductive health have traditionally been matters of cultural taboo and superstition. In each village, there are women known locally as sowie who perform FGM on young girls and attend births. The skills have been passed through the matrimonial line for generations. Births have traditionally occurred in a secret place locally known as Bondo bush, which is a taboo word itself and must not be uttered in the presence of a man. Any form of birth-related complication is regarded with extreme superstition. For example, a woman who has a prolonged and difficult labour is believed to have been unfaithful toward her husband. Such practices have resulted in the national maternal mortality being one of the highest in the world (WHO 2003). The programme targets sowies in order to train them in safe delivery practices and to provide them with TBA kits that can be used after the training. In some places, TBA huts are also built with the help from local community in order to improve the birthing environment. There is no doubt among the men and women in the visited villages that women have gained greater access to better and safer delivery services through the training of TBAs,. A number of TBAs are also working as blue flag volunteers who demonstrate the preparation of ORS, and some TBAs also have apprentices who benefit from on-the-job training. Immunisation outreach activities have been successful as well. The number of children (less than five years of age) immunised over the last two years has been very significant, and all the interviewed mothers stated that their children had received immunisations. Though this success rate cannot be solely attributed to the DRP since there were other parties involved in these activities (e.g. GOSL and UNICEF), DRP has contributed significantly toward this achievement. Access to health care in the DRP refers to both the provision of services and the utilization of services. There is a great deal of knowledge and awareness about the importance of health services utilization (e.g. clinics); however, the actual utilization rates of these services are somewhat limited. The constraint to utilization is likely due to the difficulty of physical access, since these clinics are often distant to the villages, and transport facilities are also very limited. In some areas, village health volunteers accompany people to the clinics, especially in the hearth models villages where there are two village health workers are assigned to promote growth monitoring. Improved Health and Nutrition Practices The program has targeted mothers to improve their knowledge and skills in a range of health￾related areas including immediate and exclusive breastfeeding (IEBF), effective weaning, rehabilitation of moderately malnourished children, importance of early referrals, diarrhoea prevention and management, malaria prevention, HIV/STD prevention, family planning methods, schistosomiasis prevention, the value of diet diversification, food processing, safety, and storage, and food and health hygiene and safe water. The project has effectively applied a positive deviant model and a hearth model, both of which are based on a demonstration learning. Every month the nutritional assessment of children under five is monitored, and the results assign each child to one of three categories. In the positive deviant model, one of the 8 best practicing mothers is identified and others are encouraged to learn from discussions among themselves. This approach has proved successful for the moderately malnourished children. The Hearth model is demonstrates food preparation by utilising locally available nutritious ingredients. Mothers with identified severely and moderately malnourished children meet with a volunteer mother who demonstrates how to prepare nutritious food for their children to eat. This process continues for 12 days while the mothers learn improved methods of food preparation, and on the 13th day, the mothers prepare food on their own. This particular process is also supported by two village health workers, one male and one female. This particular model has been very effective, and children do recover quickly from severe or moderate malnourishment. Despite the success of such approaches, however, there were a small number of severely malnourished children wearing red wrist bands observed during the visit. Women in the visited villages showed a substantial understanding of and awareness for the prevention of diarrhoea, malaria, and STDs, Family Planning Methods as well as other health￾related issues. Almost everyone knew the health messages that have been communicated to them by the program. Despite the high level of awareness, several respondents complained about an increased incidence of diarrhoea over the last five months among adults as well as children. This was due to the drying up of the water wells, which was forcing people to drink unclean stream water. A very high level of awareness of the value of exclusive breast feeding and diet diversity was demonstrated by the women’s group members. Overall improved cleanliness was also mentioned by numerous male and female respondents. It was observed that several children wearing red bands were carried around by their older siblings, especially girls. While mothers work in the fields, these older sisters assume the responsibility of child care. Thus, while mothers have the knowledge about a nutritious diet, they are not always present to prepare the food. Also, although women were aware of STD/AIDS, some stated that their husbands had no knowledge of preventative measures or refused to take them. Village-level Capacity-building for Health The major achievement in health capacity building is the presence of trained TBAs in every village. In the words of one of the TBAs, “…before this program, I was anxious about each birth, I had to guess how long the labor was going to last, and what if something goes wrong, now I know how long the pain may last and where to go should there be any problem. I feel so much more confident.” [TBA in Kamaron Village] In addition to the TBAs, there are a number of community health promoters and village level volunteers, but the health sub-committees within the village development committees are largely absent or inactive. In most visited areas, members of the village development committees did not acknowledge the existence of a community health action plan. So whereas in some villages improved health practices, (such as the use of plate racks and cloth pegs etc) are in place, the community–level capacity for health planning needs to be reinforced. The linking of district medical health care services with community and village development sub￾committees has not yet been achieved. Decision-making and Governance in Health The village development committees mentioned that they have monthly or bi-monthly meetings. However, there was no evidence of a clear action plan in the visited areas and no initiatives for overall community health improvements. A number of TBA huts can be found in the program working areas, which were built with help from other projects and community 9 contributions. And in general, the health program activities appear to have worked well where hardware support was available from other programs. The GOSL officials on the health steering committees have limited information about and understanding of health issues. The community health workers expressed the hope that the MoH would one day absorb them into the system; however, no such plan was identified at this stage. Overall, improved health governance was the objective that showed the least progress. This is understandable given this stage in project implementation, since changes in governance are likely to take longer. Objective 2: Food Security Farm Productivity Farm production and productivity have always been at the heart of Sierra Leone’s economy. Inland swamplands have been cultivated in rice since 1920s (Encyclopedia 1974). The other critical cash crops for export have been coco, coffee, cola beans, and oil palm. Due to the extended conflict, the most productive lands were abandoned. The DRP, through its farm productivity component, has focused on two major interventions—first, to bring previously cultivated inland valley rice land back into production and, second, to recuperate the essential tree crops that provided for agricultural exports. During field visits, farmers were very keen to show the progress they have made, which is indeed remarkable. Large areas of inland valley land are now under cultivation and productive. The FFS initiative has helped advance this work, and farmers are particularly receptive to the input support offered by the project. In terms of tree production, farmers in some areas have taken loans ranging between 150,000 and 750,000 LN to invest in their cacao and coffee farms. This is relatively large loan size and a clear indication that some farmers in some of the DRP intervention areas are getting back on their feet. It suggests that DRP interventions have reduced the slide into poverty and accelerated the recovery process. As one of the respondents describes the impact of the program, “… this program has made a big difference in our lives; we received tools, seeds, and even food so that we can work on our farm. We take food with us while we are working in our field, cook and eat it there. We also bring some food back for our children. It has been really good to work together…without the program we could not have done this, we would have suffered a lot...’ [farmer in Darujanction village] According to many respondents, inland valley rice production has increased and will continue to do so as more and more people return back to the rural areas and re-establish their rural household economies. FFS groups are cultivating rice and cassava and have adopted improved techniques of producing and preserving seeds. Women are using better techniques for keeping vegetable seeds. In one of the respondent’s words, “…before joining the group, I used to leave the fruits to rot on the tree and fall in order to collect seed. But now, I know, I should harvest the healthy fruit from the plant, let it ripen, then collect and dry the seeds to preserve them until ready for use…“[ woman farmer in Matam village]. Qualitative interviews with female groups in the visited areas revealed that vegetable production has been severely limited by grasshopper infestation. Although some have harvested small quantities of vegetables to share among the group members and consume in the household, many report no production. To avoid grasshopper damage in one of the visited villages, women moved their vegetable plots two and half miles away from their last year’s location, making it harder to water and care for their plants. Such pest invasion is a major constraint to the sustained adoption of vegetable production. In other areas, the lack of a 10 reliable local market for vegetable products has significantly limited commercial-scale vegetable production. In general, this component has been successful. Communities in the rural intervention areas have been provided with food rations while they rebuild their lives. Although years of conflict and neglect had left the productive farmland unusable, now people in the area have begun to take charge of their own farming and food production. Reducing Pre and Post-harvest Losses It is commonly estimated that farmers in Africa lose up to 30 percent of the post-harvest product. This project component was designed to reduce loss by building and rehabilitating community drying floors and storage facilities in the DRP working villages. The targeted number of drying floors and community storage facilities has not been built or rebuilt due to the unavailability and price of exported building material (such as cement). Although there is no standard set of dimensions for the drying floors, the actual size seems to vary between 10- 20 m2 , depending on the amount of food allocated under FFW/CFA norms. The drying floors are used seasonally and were not in use at the time of the visit. Villagers report that the drying floors only accommodate 5-8 bags of rice, and they are used primarily by the households located in closer proximity. Although the technology and the need of the drying floor itself is not in question, the limited capacity of the standard unit does raise an issue of access to a wide range of beneficiaries. It would appear that expansion of this program component would seek to assure broader access to the floors. Community storage facilities have also not been an effective intervention, primarily because of inconvenient location. In most communities, the siting of the facilities is decided by the town chief and elders. In some places, the storage facilities were built by agencies to serve the emergency food distribution operations, thus location is decided for logistical effectiveness rather than community need. In general, although the VDC committees acknowledge the presence of such facilities, they are not located in convenient places for many villagers to use. Moreover, poorer people in the village are reluctant to use these facilities because this will give information to their neighbours about how much food they have in store, which they prefer not to do. Re-establishing Market Linkages Under this initiative, the program partners have sought to rehabilitate regular weekly market facilities and provide for village to market roads. In the visited villages, market facilities were rebuilt using locally produced materials from palms, trees, and bamboo. These facilities are well-utilized and much appreciated by villagers. Farm to market road-building is on-going and appears to be progressing well. According to staff there is clear, widespread satisfaction with this project component of the program. The roads effectively open larger markets to a range of agricultural products and increases food security for all segments of the population. Under the current scheme, however, the efficiency of the road building is to some extent irregular. One visited project, for instance, had a 6.5 mile farm-to-market road stretch that had been started in August 2004, but only one mile had been completed. Between 120 and 137 people (including 75 women) are currently reported to be working on the road. In-depth discussion with the group revealed that the village inhabitants from 6 adjacent villages came together to rebuild the road and decided to work one day a week on a voluntary basis since all of them have other activities to do during the week. So they come every Friday to work on the road. The DRP field worker met with this team and drew up a MOU, which, while encouraging the workers, has not made much difference in terms of the work schedule. The village volunteers are continuing to come every 11 Friday, and although they wish to continue the road-building with or without any support from an external agency, there is no plan for completion. Overall, there is a need for more sharing of best practices among the different organizations engaged in road-building. The effective road building strategies in those places where the roads have been built and are successfully operational could be shared across the Consortium. Village-level Capacity-building for Agriculture Under this component, each of the four CORAD members have formed farmers groups such as the Farmer Field Schools and Women’s Agricultural Groups. In all the visited areas, the FFSs are working well. They have monthly meetings to discuss farm issues; they cooperate in the planting of cassava and rice and other agricultural activities; and they share harvest. Farmer field days have been highly successful according to the organisers and participants, and these activities are highly appreciated and praised by MOA. Farmers in the DRP working areas have also adopted new techniques to preserve selected seeds from their own harvest. Most FFS have both male and female members. The total number of members averages between 15 and 30 depending on the size of the village. Many of the FFSs have produced an action plan comprised of a list of tasks they wish to do in the near future such as, clearing lands for cultivation, planting, etc. As one of the farmer puts it, “now we know the benefit of working together in a group, if you have any problem you are not too worried because you know you are not alone…” [FFS member in Badala village] Grasshopper infestation severely affected the levels of production for cassava and rice among the FFS members. The visited demonstration cassava and vegetable plots were filled with grasshoppers, and this problem has occurred two years in a row. The Consortium is aware of this problem and has taken steps to introduce environmentally-sensitive preventative measures. Several FFSs have several members from the same households—often both husband and wife and other close family members. Some FFS members also belong to several groups (mixed FFSs and female FFSs, for example). Although it might be the case in rural society that men will not allow their wives to join the FFSs alone, the inclusion of more than one household member does restrict access to the FFS for other households that could benefit from the project. This situation can also result in “double-counting” problems which can distort project achievement targets . Decision-making and Governance in Agriculture The Village Development Committees (VDCs) have been at the centre of most interventions in the DRP. In many cases, the program field staff facilitated the formation of the VDCs before any intervention activities were implemented. Similar traditional groups have always been a part of the informal political structure in the villages, usually under a single leader. For instance, in the southern districts these groups have been known as “Gbotima,” with around 30-40 members. Since the VDCs have been the primary entry point and the key channel for the implementation of project activities, these are critical village-level institutions for the DRP program. However, in several visited villages, the VDCs do not appear to be very effective. The committees are intended to have 12 members, but the actual number varies between 10 and 22 participants. The composition of the committees also varies from village to village, and the effective representation of the wider community is not always guaranteed. In some VDCs there are several members from the same (e.g. husband and wife, mother and son). In some cases, several villages have formed one VDC. In one instance, the team 12 documented a committee formed with members from eight villages, twelve of the twenty members from a single market village and one member each from the other villages. The nearest village was a half-mile from the market town while the furthest was three and a half miles away. This composition raises question about the effectiveness and motivation of the committees. Although, many VDCs have specific action plans they do not have a community level monitoring system to assess progress and effectiveness of these plans. Maintaining a Safety Net After 10 years of war, the area known as the ”breadbasket” of Sierra Leone was devastated, and many people lost their livelihoods and became extremely vulnerable and desperate. For many households, the key economic contributors either died or were disabled. Culturally, families look after vulnerable members and almost always accommodate them within their limited means. During this time of extreme stress, however, cultural coping strategies are over-extended, and the DRP has helped provide a subsistence base for the most vulnerable through food aid. The VGF program has targeted the blind, wounded war veterans and amputees, war widows, aged and chronically ill, and the physically disabled. On average, five individuals are supported via DRP in each working village. The food rations support includes bulgur wheat, lentils and vegetable oil. Although the quantities are small, this food has made a big difference, and families often share the food among the members. Although the VGD ration itself is standardized the level of VGD assistance varies from one area to another depending on specific needs and vulnerabilities. In some areas VGD consists of twelve monthly rations, while in other areas, it covers only the food shortage months. It is not clear however, what criteria are used to determine the period of support, since the beneficiaries appear to be chronically vulnerable in all of these areas. The CORAD Approach As stated earlier, this is the first time these four CORAD members have worked together. The approach has worked very well and the positive impact is felt across all the member organisations. It has been most effective at the central and higher levels of management. The collaboration has provided opportunities to draw upon complementary institutional strengths, and it is felt to be highly cost effective. Through the CORAD, it has been easier to deal with other key stakeholders including donors and GOSL, and CORAD has achieved a strong reputation and a very high level of acceptance. Despite the very effective level of coordination and sharing among CORAD members at the higher levels, some field staff expressed a desire for more systematic sharing of experiences and information at the field level. Field exchange visits would be a mechanism for the mutual support for partners as they struggle through the complex aspects of project intervention and implementation. Overall, DRP has provided all four CORAD partners the invaluable transitional opportunity for staff to shift the institutional mindset away from a relief-oriented distributional approach to a more sustainable development one. Monitoring and Evaluation World Vision has played the central role in providing M&E services for CORAD during the implementation of the DRP. A team of three members in coordination with the CCU, oversees the timely and consistent collecting of progress data. Three of the four CORAD partners also have their in-house M&E staff person who serves as the point of contact for the 13 core CORAD team. The M&E team has very competently developed indicator formats, conducted training workshops to build staff capacity, and prepared quarterly and yearly reports for all relevant parties, including donors, as part of the contractual agreements. In addition, the team also ensures that projects comply with environmental guidelines. The M&E unit has effectively delivered its contractual obligations and produced regular and timely reports. However, there were concerns from the field that indicator formats were not user-friendly. The key monitoring tools had become non-standardized, and each partner organization began to use somewhat different tools for a number of activities. Another limiting factor was that every partner organization did not have an M&E staff person to consult. Although there is an effective quantitative monitoring system in place to track the completion of activities, a more systematic and regular qualitative monitoring component integrated into the current M&E unit would provide better insights into project impacts in a timely fashion. Projects may meet or surpass targets, but the true nature of impacts is best understood with a qualitative monitoring process. The FFSs provide an excellent mechanism for developing a local-level participatory monitoring system that can identify problematic issues and better inform project decision-making as well. Gender The programme has made special efforts to include women as participants and target beneficiaries, so that they also can reap the project intervention opportunities. In many cases, women stated that they were never before members of committees until the DRP interventions were introduced. Despite unequal power relations (as some men still tell women to keep quiet in the meetings!), the effective participation of women is a great achievement for the project. However, in some cases, the targeting of women may have led to an exploitation of their labour and increased their workload. There was evidence of men recruiting female members in order to assign them the more menial tasks. Lessons Learned and Recommendations The Community Approach: the Need to Guarantee Social Inclusion The DRP has relied heavily on a “community approach,” and it is important to unpack the concept of “community.” Communities in this region are not homogenous; rather, they are characterized by different levels of interest and stratified power relationships. Without an in￾depth understanding of village heterogeneity, the impacts of DRP will be limited. It is critical to understand the needs of different groups within the community and for implementation to go beyond the “community gate keepers.” Such a perspective will enhance staff capacity in the field and encourage the critical thinking necessary to ensure project flexibility. The process of social inclusion is one that requires constant vigilance by the project leaders and the institutionalisation of mechanisms that insure the representation and participation of all community segments. VDC Governance: Reinforcing Local Capacity A reinforced effort should be invested in the VDCs in order to build their capacity and to enable them to take charge of their own development as effective community-based organisations. It is particularly critical to introduce greater transparency within the committees with regards to how members are selected, the length of term, and the expected 14 level of accountability towards the community constituency. The VDCs have to establish their own legitimacy as community leaders. The effective representation of youth, women, and the elderly should be a focus of attention. Though it may be culturally challenging, the inclusion of youth and other marginalised groups is very important in order to reduce the vulnerability of poor people in the region. The age limit for membership should be reduced for younger persons, for example, change the upper age limit from 35 to 20 years. For long term institution building, the VDC should have constitutions that clearly spellout the different roles and responsibilities for different members. Having several household members in the same group should be prohibited, and several villages should not be encouraged to form one committee. The DRP should have as extensive a coverage as possible. Since membership in the FFS and women’s group is limited the unit of membership should be the household and not the individuals. This would correct the situation in which household contributes more than one member to a given FFS. Farmer Field Schools A system should be developed or adapted to monitor FFS action plans. Farmers might implement a participatory monitoring and evaluation (PM&E) system in which they meet regularly to discuss project progress and document the results of these meetings. Also, at the end of each season, farmers can meet to evaluate progress of the action plan for that year, and then develop a subsequent, modified action plan for the next time period. The FFS membership regulations should be reviewed. Though it may be challenging for some cases to include women without their husbands being a part of the FFS as well, no more than one member from each household should be encouraged to join the same FFS. Such initiatives may require motivation and awareness-building among male members. In order for the FFS to increase production and sustain their improved agricultural practices, further capacity-building is needed, particularly in pest management (e.g. grasshoppers). There are appropriate technologies for pest control available worldwide, and a low-cost, simple technological alternative should be introduced. For instance, in Asia, nets are used to catch grasshoppers from the rice fields, and similar techniques could be tested in Sierra Leone on a pilot basis. Stronger linkages between research institutes and the FFSs could help identify the alternative technologies for pest management. Finally, most villages in the DRP working area cultivate crops like cacao, coffee, palm oil, and cola beans. In many places, the market linkages are weak, and farmers do not have an outlet for their production. More market knowledge and improved market organization are needed to improve the effectiveness of the feeder roads. Targeting DRP targeting strategies should be revised to include adolescents to include younger members of the community. While only less then a quarter of the rural births are registered in Sierra Leone (UNDP 2005) and it is often difficult to ascertain age, adolescent boys or girls can easily be identified. In addition, the DRP should be sensitive to wealth and asset differences when deciding upon potential interventions. FFW activities should target the poorest only, especially those individuals who sell their labour to survive during all or part of the year and have no other means of income. 15 Health hygiene and health prevention interventions should target male members of the household as well as females, because women’s groups alone can not make the necessary prevention decisions with regards to such health issues as STD/HIV prevention. Well-being categories from one village Better off (oiolah): business people, large-scale vegetable production, owned land, durable assets like bicycles, sell imported goods like cigarettes, sell rice by the bag, sometimes help the poor people in the village by lending food/money and do not charge interest Moderate poor (abarbar): 4-6 months of food supplies from their own sources, 2-3 acres of land, raise small livestock, 4-6 months of food insecurity Very poor (fishertan): hand–to-mouth existence, no land, sell labour sometimes, dependence on the forest for their livelihood: palm oil, firewood, bush food Extremely poor (ormoneh): clinically ill, blind or handicap, beggars, dependence on God and charity Post-harvest Losses The project should revisit the drying floor intervention and assess whether the standard size will in fact meet program goals. It is possible that DRP promote locally available mats (bamboo mats, palm leaf mats) as an alternative to drying floor structures in order to assure maximum community participation. This alternative might promote appropriate post-harvest practices and provide and income-generating activity (mats) for some households. Most importantly, the major post-harvest loss appears to be caused by the burden of loan. For poor and very poor households, one of the key coping mechanisms during the hungry period is to take a food credit either in cash or in food. In the northern part of the country, it is known as ‘belah’ (meaning ‘imposed on you’). The effective interest on these loans is very high. For example, a household that borrows one bushel (25 kg) of rice during the hungry period has to return up to four times that amount only four to five months later (400% interest in 5 months). If a household fails to repay its debt, the borrower is taken to court and ordered to make repayment. The program should consider interventions that allow for interest-free loans or small ”food credits” (different from IGA micro-credit intervention) made available to poorer households during the hungry season, so that their harvest is not jeopardised by debt. Gender Training To assure household food security, access to food is just as critical as its availability. In rural areas, the cultural food distribution favours adult male members of the household. Men are served first with the biggest and the best portion of food. It is important to raise awareness among men and women on sharing food more equally. Male members should also be included in health-related training. Staff capacity-building on gender sensitivity and gender responsive programming are important project investments that yield positive impacts. 16 Environment In the DRP working area, widespread deforestation is occurring. Increasingly, charcoal production is becoming a livelihood option for many poor people, resulting in great pressure on the woodlands. This is both an environmental concern and a food security issue, since the forests provide multiple products to rural families livelihoods systems. Many households have a seasonal dependence on the forest products, such as , forest foods like bush yam, bush banana, palm heart, etc. Although the agricultural component of DRP has improved food security among beneficiary households, the use of forest products continuous to be an important safety net and critical coping mechanism in the face of variable access and availability of food for rural communities. Monitoring and Evaluation The M&E system should extend its scope, especially in CORAD. There are two key roles for M&E coordination unit—the first, a structured and standardized information gathering system for monitoring progress. To achieve this, the M&E team needs to develop data gathering and analysis tools, train individual M&E staff within each organization, and set up a reporting protocol. The second role would be to create an action research component to equip the program with enhanced, in-depth knowledge about the field realities, document best practices, and disseminate learning across organizations. The teams members either independently or with the program team can systematically collect information, process and analyze it to offer insights into constraints that hinder program effectiveness and limit impacts. For instance, the team may collect case studies of successful and failed interventions, analyze the varying success, and identify strategies for improving project performance Each partner should have an M&E person within the organization, and the field staff should participate in the development of appropriate monitoring tools. The core M&E team should be available to provide evaluation expertise to the partners. Conclusion The qualitative assessment provides ample evidence that the activities of the project are consistent with the felt needs of this war-struck population in the process of rehabilitation. In its two main objectives—health and food and nutritional security—the DRP has implemented a set of interventions that is responsive and significant in terms of measurable impact. The CORAD partnership has demonstrated both certain strength through “economies of scale” and the value of combining the comparative advantages of different NGO partners in a complementary fashion. The partnership has succeeded in navigating the complex path that winds from emergency relief aid to a sustainable development program. The design and the implementation of the program are relevant and effective in terms of development impacts on poor communities and are recognized as such by the beneficiaries. At the same time, the DRP has faced difficult issues that confront most development efforts. In its community focus, CORAD has experienced the challenge of identifying the institutional mechanisms that can guarantee the representation of all social groups in the development process. The program has sought to “localize” the development decision-making through group formation strategies and yet assure that all stakeholders, particularly the most marginalized, can participate. Also, the program provides clear examples of the importance of making relevant information available to community actors on a sustainable and regular basis—information on production technologies, markets, health practices, and nutrition. The program has underlined another major development lesson by building local capacity. The different project interventions have taken local resources (TBAs) and local knowledge (foods, forms of organization) and mobilized them in the effort to create a development dynamic in 17 small communities. These activities, as expected, have not yielded perfect results, but they have created new opportunities and contributed significantly to a process of positive change. 18 List of Individuals and Groups Interviewed Name Position Organization Antonia Powell Health Program Manager CRS, SL A. Tom Roberts, PH.D Agricultural Economist World Vision Alieu Badara Sesay Food aid monitor, M&E Alie K. Guteh Field Officer World Vision Ben A. Massaquoi National Extension Coordinator Ministry of Agriculture, Govt. of SL Dorance M. Y. Cooper Administrative/ Program Assistant Food for Peace USAID, SL Edward Joseph Benya Reintegration Specialist USAID, SL Gibrial Bangura M&E Officer CARE Fred Goba M&E Officer World Vision, FT Finbarr Sweeney Project Manager, DRP CARE Ibrahim Jalloh Area Manager, North CRS John Perry Chief of Party CARE Joseph Juana Senior Program Officer for Health CRS, SL Joseph Senesie Health Manager World Vision, SL Joseph Mustapha Regional Coordinator World Vision Kumba Keodoyoma SRN/SCM Reproductive Supervisor World Vision Marion Almeida Project Coordinator, DRP Africare Myles Harrison National Director World Vision Nick Webber Country Director CARE Nicole S. Balliette Head of Programs CRS, SL Paul Davies Acting Country Representative Africare, FT Tamba Nelson Agric Extension Officer World Vision Tabma Dalton Kassoh On Job Trainee CARE Solomon Kassibo NGO Coordinator Sylvetta Scott Program Manager, Nutrition and Health Ministry of Health, GoSL Veronica Smith M&E Supervisor Africare Mausa Kamara Health Field Agent CRS Joana Saido Village Health Worker Manowulo Village Maita Vandy Village Health Worker Manowulo Village Rogbesseh village community, FFS, VDC, Women’s group, TBA Matham village community, FFS, VDC, Women’s group, TBA Helemakona village community, FFS, VDC, Women’s group, TBA Kamaron village community, FFS, VDC, Women’s group, TBA Badala village community, FFS, VDC, Women’s group, TBA Manowulo village community, FFS, VDC, Women’s group, TBA Darujanction village community, FFS, VDC, Women’s group, TBA Banda Jume village community, FFS, VDC, Women’s group, TBA Manjama village community, FFS, VDC, Women’s group, TBA Condama village community, FFS, VDC, Women’s group, TBA 19 20 Documents Consulted CORAD (2003). Transitional Assistance Program (TAP) Proposal 2004-2006. CORAD (2005). P.L. 480 Title II, Sierra Leone/ CARE International on behalf of the Consortium for Rehabilitation and development (CORAD), Fiscal Year 2005 (FY2) Result Report Encyclopedia (1974). Encyclopedia of World Geography: Man and his World Today. London, Octopus Book Ltd. GOSL (2006). Sierra Leone Government Minute Paper, Report on Evaluation Mission on the Development relief program Implemented by CARE - Sierra Leone UNDP (2005). Human Development Report: International Cooperation at a crossroads, Aid, Trade and Security in an Unequal World. Human Development Report. New York, United Nations. WHO (2003). The World Health Report 2003 - shaping the future. Geneva, World Health Organization.