FINAL EVALUATION of HEALTH ALLIANCE INTERNATIONAL’S CENTRAL MOZAMBIQUE CHILD SURVIVAL AND MATERNAL CARE PROJECT Prepared by: Kerry Selvester, Team leader Dr. Jagrati Jani, FINNIDA Mr. Davissone, Provincial Health Directorate – Manica Mr. Congara, Provincial Health Directorate – Sofala Mr. David, Oral and School Health Programme – Ministry of Health Prepared for: Health Alliance International 1107 NE 45th Street Seattle, Washington 98105 and Child Survival and Health Grants Program Office of Health, Infectious Disease, and Nutrition Bureau for Global Health United States Agency for International Development Cooperative Agreement Number: FAO-A-00-98-00054-00 November 2002 Health Alliance International Child Survival Programme Final Evaluation 2002 1 Table of Contents Executive Summary 1 A: Assessment of Results and Impact of the Programme 6 1. Background 6 2. Evaluation Methodology 8 2.1 Approach 8 2.2 External Evaluation Team Members 8 2.3 Programme 9 3. NGO Approach to Project Implementation 9 4. Project Emphasis 9 5. Project Achievements 10 5.1 Summary Table of Indicators 10 5.2 Narrative of Project Achievements 14 6. Crosscutting Approaches 27 6.1 Community Awareness 27 6.2 Youth Groups 28 6.3 Training Strategy 29 6.4 Advocacy 30 6.5 Information Management 30 6.6. Sustainability Strategy 31 7. Programme Management 31 7.1 Planning 31 7.2 Staff 32 8. Conclusions and Recommendations 33 8.1 Conclusions 33 8.2 Lessons Learnt 34 8.3 Recommendations 35 B: Results Highlights 40 References Attachment A: Poverty Mapping Attachment B: List of Studies Undertaken by HAI Attachment C: List of Interviews Attachment D: Terms of Reference Health Alliance International Child Survival Programme Final Evaluation 2002 2 Glossary of Terms and Abbreviations Cantos Joviais Adolescent drop-in centres CLC Community Leaders Council CS Child Survival DIP Detailed Implementation Plan DPS Provincial Health Directorate VCT Voluntary Counselling and Testing Centres HAI Health Alliance International HF Health Facility HIS Health Information System MTCT Mother to Child Transmission IEC Information, Education, Communication (material) IFPRI International Food Policy Research Institute INE National Statistics Institute KAP Knowledge, Attitude and Practice (Survey) MCH Maternal and Child Health MFP Ministry of Finance and Planning MOH Ministry of Health NGO Non Governmental Organisation NHS National Health Service PAC Programme for cultural Activists PVO Private Voluntary Organisation SESP Provincial IEC Section STD Sexually Transmitted Disease STI Sexually Transmitted Infection TBA Traditional Birth Attendant Health Alliance International Child Survival Programme Final Evaluation 2002 3 Final Evaluation Report. Health Alliance International Child Survival Grant 1998-2002 Executive Summary Health Alliance International (HAI) was awarded a Child Survival grant from the USAID Bureau for Humanitarian Response to carry out maternal health, malaria control and prevention, and STI/HIV/AIDS education and prevention in Manica and Sofala Provinces in Mozambique from 1998- 2002. HAI’s partner in Mozambique is the Ministry of Health. HAI works directly with the Provincial Health Directorate for each province (Direcção Provincial de Saúde, or DPS). This partnership represents the essential strength of the HAI programme as the approach builds both ownership and sustainability of activities. The programme goal is to bring about sustainable reduction in infant, antenatal and maternal mortality and morbidity in the two provinces. HAI has achieved remarkable success in reaching the targets set in the Detailed Implementation Plan (DIP) for the Child Survival Project. However, there are still severe health problems in the central region to be tackled, further exacerbated by HIV/AIDS epidemic that has changed the epidemiological landscape. One of the principle achievements of CS grant has been the strengthening of the health structures from provincial to health post levels through the extensive capacity building programme in an extremely difficult operating environment. Over the period of the grant HAI has responded to the rapid changes in the HIV/AIDS field by supporting important provincial initiatives that were in accordance with the overall objectives of the organisation, namely in the areas of the Mother to Child Transmission (MTCT) and the Voluntary Counselling and Testing Centres (VCT). HAI should continue with the approach of working through the National Health Services (NHS) and work to the organisation’s strengths in terms of health worker training, operational research and innovation in cutting edge areas of policy (Mother to Child Transmission and antiretroviral treatment). All aspects highlighted by HAI for the next-phase Child Survival grant are considered to be priority areas for intervention by the provincial authorities and are more focused than the previous Child Survival Programme. HAI needs to consolidate its internal structure in order to clarify roles of personnel within the organisation and their responsibilities towards partners and donors. The current reporting culture in HAI is not conducive to reflective learning and new mechanisms need to be devised to encourage organised reflection that can stimulate staff members to develop critical analysis of their programmes in a supportive environment. Lessons Learnt: HAI will work to its strength as an organisation and have a more focused operational approach. HAI should disseminate their operational approach by writing an issue paper highlighting the successes and constraints of working through the local health institutions Health Alliance International Child Survival Programme Final Evaluation 2002 4 HAI will avoid moving into areas where they do not have the resources or vocation to adequately address the problems. HAI has learnt the need to embrace new challenges when they are deemed to be of overwhelming importance and urgency, even if they were not necessarily foreseen at the beginning of an operating period, for example, HIV/AIDS work. HAI needs to consider national level representation to strengthen their advocacy and operational base. HAI has used a considerable amount of resources on health worker training using an in-service training model. It is time to consider a more innovative approach to the capacity building needs of the NHS. Recommendations. Detailed programmatic recommendations were made to improve the performance of the organisation in the extension period of the Child Survival Grant for each of the component parts. These included: Maternal and Newborn Care · Introduce the rapid `strip` test in peripheral health facilities (CS Extension) · Put emphasis on the early treatment of partners for syphilis and promote the use of condoms during pregnancy · Carry out an investigation to understand the cultural acceptability of increased involvement of men in reproductive health issues. · Develop a strategy to overcome the supply problems in the syphilis treatment programme · Carry out a study on risk factors for stillborn and neo-natal deaths. STI/HIV/AIDS Cantos Joviais · Support the transformation of the Cantos into Adolescent Friendly Clinics and maintain the Cantos within the health facilities. · Identify sources of IEC material and provide the youth workers with the information in order that they can become autonomous in requesting material. · Develop a fund raising strategy with the members of the Canto · Develop a membership sustainability plan · Suggest a policy of `bring your sister´ to the Cantos in order to address the gender imbalance Mother to Child Transmission · Consider moving the Mother to Child Transmission programme from the central and main provincial hospitals to city maternity facilities. · Circulate more information to health personnel about the MTCT and clarify the coverage of the programme (MOH) · Introduce in Beira group therapy sessions for mothers enrolled in the MTCT VCT · Expand the VCT using the NHS as base for the centres · Develop an in-service training strategy for counsellors · Contribute and use more fully the national web site. Anti-AIDS School Clubs · The Ministry of Education should explore financing possibilities for the Anti-AIDS Clubs. · HAI should act as a facilitator for the clubs. Health Alliance International Child Survival Programme Final Evaluation 2002 5 Malaria ß Expand the bednet programme to include all traders in each location ß Develop a strategy for the reduction of the price of the bednet for pregnant women and children ß Immediately carry out a study of possible supply strategies. ß Increase the involvement of the DDS-DPS in the implementation of the bednet programme Further recommendations were made concerning strengthening partnerships: ß HAI should maintain and improve the co-ordination mechanisms instituted with the DPS ß HAI should initiate, in FY03, a joint planning process with the DPS including sharing of information on the financial resources available for each joint activity ß DPS should institute a participatory planning process with HAI for activities to be undertaken in FY03-04 within the strategic planning process. ß DPS should review the job of descriptions of HAI personnel (to be provided) and share with relevant heads of section within the DPS ß Evaluate the capacity of the CLCs in the light of the demands from programme components and critically examine which of the roles are most suited to the CLCs ß Discuss the official recognition of the CLCs. Some recommendations were made to improve the internal management of the programme ß Organise a retreat for all staff to discuss the HAI programme strategy for the next five years ß Prepare a new organisational chart that reflects the new programme ß Develop job descriptions with staff to reflect new roles and responsibilities ß Share the new organisational chart and job descriptions with all partners, especially the DPS ß Re-examine the role of the Country Coordinator; consider a more strategic role and less administrative role ß Consider HAI representation at national level ß Rationalise internal reporting mechanisms and institute mechanisms for reflection and analysis ß As soon as possible carry out an organisation-wide training on HIV/AIDS ß Write the story of HAI achievements in words, photographs or video ß USAID should actively support initiatives by PVOs to introduce reflective learning into the monitoring cycle ß USAID should examine the impact of the emphasis on results-based monitoring on the analytical quality of programming of PVOs. Recommendations to improve training strategies were made: ß Consolidation of training material in all components (MCH/STI/HIV/Malaria) to enhance the curriculum of initial training courses in the Institute for Health Sciences/Training Centres ß Carry out a training of monitors course in the Institutions/Training Centres ß Provide the libraries of the training institutes with copies of all training material used over the past four years. Make a list of the materials available to the student health workers ß Facilitate the final fieldwork of at least 10 student health workers per year ß Continue to plan all in-service training with the in-service training department of the DPS. Health Alliance International Child Survival Programme Final Evaluation 2002 6 A: Assessment of Results and Impact of the Programme 1. Background In 1998, Health Alliance International (HAI) was awarded a Child Survival grant from the USAID Bureau for Humanitarian Response to carry out maternal health, malaria control and prevention, and STI/HIV/AIDS education and prevention in Manica and Sofala Provinces in Mozambique. The baseline data for this programme included knowledge, attitudes, and practices (KAP) baseline survey of populations in both provinces. The population of Manica is estimated as 1,207,000 inhabitants with an estimated 20 inhabitants per km², Sofala is estimated to have a population of 1,516,000 inhabitants, with an estimated 22 inhabitants per km² [Projection: INE. 2002]. Of the provincial populations, 18% in Manica and 28% in Sofala live in the capital cities of Chimoio and Beira, and an estimated 35% in addition live in urban or periurban areas along the Beira Corridor (a transportation route through the centre of the two provinces that connects the port city of Beira with Zimbabwe and the interior). An estimated 85% of the population in the two provinces has access to government-sponsored health services, based on the proportion of pregnant women who deliver having had at least one prenatal care visit. The remaining 15% live in very remote rural areas in the far north and far south of the region. The following map shows both the incidence rate of poverty (population below the national poverty line) by district and the number of poor people living in the areas. It is possible to see from the following map that Sofala Province has some of the poorest districts in the country with both high poverty incidence rates and high numbers of poor people living in the cities and along the Beira corridor that links the provinces of Manica and Sofala [MFP. 2002] Health Alliance International Child Survival Programme Final Evaluation 2002 7 Map 1: Rate of Incidence of Poverty and Number of Poor Inhabitants in Mozambique by District. Source: Mapeamento da Pobreza: MFP/IFPRI . 2002 Legend: Indicencia da Pobreza: Poverty Incidence # 1 point = 10,000 poor people ## # # # # ## # # # # # ## # # # # # # # ## # # # # # # # # # # # # # # # # # # # # # # # # # ### # # # # # # # # # # # # # # # # # # # # ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## # # # # # # # ### # # # ## # ## # # # # # # # # # # # # # # # # ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## # # # # # # # # # # # # ## # # ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # ### # # # # # # # # # # # # # # # # # # # # ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## # # # # # # # # # # # # # ## ## # # ### # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## ## # ### # # ## # # ## # # # # # # # # # # # # # ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## # # # # ## # # # # # ### # # # # # # # # # # # # # # # # # ## # # # # # # # # # # # # # # # # # # # ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ### # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## ### # # # # # ## # # # ##### #### # # # # # ###### # ## Incidência da pobreza 0.18 - 0.48 0.48 - 0.62 0.62 - 0.74 0.74 - 0.85 0.85 - 0.98 # 1 ponto = 10.000 pessoas pobres Health Alliance International Child Survival Programme Final Evaluation 2002 8 2. Evaluation Methodology 2.1 Approach. The evaluation team adopted an inclusive approach to the final evaluation exercise, involving programme staff, partners and beneficiaries in the discussions and development of the evaluation plan. Extensive consultation of programme documentation was studied by all members of the evaluation team. The information was used to prepare discussion topics for the initial workshop with HAI staff. The workshop was held at the beginning of the fieldwork with the staff members of the HAI, in order to discuss how the evaluation was to be conducted: specifically, to identify the aspects of the work that needed to be covered in the programme in order reflect the depth and breadth of the Child Survival Programme; to identify with the team how they viewed their work and the problems faced in activity implementation; and finally to identify the stakeholders that needed to be interviewed in order to cover all aspects of the Child Survival Programme. A short presentation was made of the preliminary results of the final KAP study. This was followed by a lively discussion about the meaning of the results. The workshop was useful in terms of introducing the evaluation team to the extensive programme and to raise some issues that the team wanted investigated during the evaluation process and in helping to guide the evaluators through the process. It was followed by a discussion about the interview guides. The workshop was followed by a period of eight days of fieldwork in the provinces of Manica and Sofala. (See programme for details.) The visits covered all areas of the programme, namely, Maternal and Newborn Care component; Malaria component; and STI/HIV/AIDS component. Interviews and group meetings were held with health workers, target groups and community member. The interviews were standardised through the use of interview guidelines that identified key topics to be covered. (See attached list of people interviewed.) Members of the HAI staff accompanied the visits and were interviewed by the evaluation team. The evaluation was carried out in a spirit of co-operation and willingness to discuss openly the successes and the constraints faced by the partners and the HAI staff. A presentation was made of the preliminary results in the provincial Directorate of Health in Sofala. The Provincial Medical Officer was present, plus heads of all key health departments. HAI staff from the Manica and Sofala offices attended the presentation. Dr. Steve Gloyd was present at the final presentation. 2.2 External Evaluation Team Members Kerry Selvester Team leader Dr. Jagrati Jani FINNIDA Mr. Davissone Provincial Health Directorate – Manica Mr. Congara Provincial Health Directorate – Sofala Mr. David Oral and School Health Programme – Ministry of Health Health Alliance International Child Survival Programme Final Evaluation 2002 9 2.3 Programme 19th of August Document consultation 20th of August Workshop with HAI staff 21st of August Continuation of Workshop 22nd of August Meetings with Provincial Health Staff in Manica 23rd of August Three district visits: -Barue (Malaria) -Manica (HIV/AIDS) -Gondola (MCH) 24th of August Document consultation Meeting with NGOs 25th of August Travel to Sofala 26th of August Meeting with Provincial Health Staff – Sofala 27th of August Three district visits -Nhamatanda (Malaria) -Dondo (MCH) -Beira (HIV/AIDS) 28th of August Final interviews and preparation of presentation 29th of August Presentation of preliminary findings 3. NGO Approach to Project Implementation HAI’s partner in Mozambique is the Ministry of Health. HAI work directly with the Provincial Health Directorate for each province (Direcção Provincial de Saúde, or DPS). This partnership represents the essential strength of the HAI programme as the approach builds both ownership and sustainability of activities. The Child Survival (CS) Programme builds on experience and current activities of HAI’s existing PVOII programme, also working in both provinces and funded by the USAID mission. The focus of HAI’s CS Programme is the improvement of service delivery for the population with access to care, and increasing community awareness and demand for services in those areas. The programme goal is to bring about sustainable reduction in infant, antenatal and maternal mortality and morbidity in the two provinces. The intervention areas are Maternal and Newborn Care (40% of programme effort); STI/HIV/AIDS Prevention (35%); and Malaria Control (25%). 4. Project Emphasis Overview The maternal health care intervention aims to improve the quality of care available to pregnant women, including STD services, through improved training and service delivery capacity in health units throughout the provinces. The programme’s STI/HIV/AIDS prevention efforts include health worker training, work with high–risk groups, support to the Voluntary Counselling and Testing Centres (VCT) and support to the new initiative for the prevention of mother to child transmission (pMTCT) through a hospital based delivery system. Malaria control activities focused primarily on appropriate case management of children and pregnant women, on assessment of chloroquine resistance and treatment￾seeking patterns and on testing the feasibility of the commercial distribution of insecticide-impregnated bednets in selected districts. Community awareness work was carried out through health education and advocacy activities with Community Leaders Councils (CLCs), theatre groups including the established PAC (Program of Cultural Activists) theatre troops, school clubs and adolescent drop-in Health Alliance International Child Survival Programme Final Evaluation 2002 10 centres (Cantos Joviais) and work with high risk (HIV/AIDS) groups such as sex workers and health workers. Changes mainly due to advances in the HIV-AIDS pandemic in science and health policy, led to changes in the emphasis of the programme in the STI/HIV/AIDS component with the addition of VCT and MTCT centres as important parts of the response to the problem. 5. Project Achievements 5.1.Summary Table of Indicators The following objectives and indicators were established at the beginning of the Child Survival Programme and are set out in the Detailed Implementation Plan (DIP). The table below summarises the achievements against indicators. Section 5.2. presents a narrative commentary on the achievements of HAI in relation to the stipulated objectives. Global Objective: Reduce Maternal, peri-natal and infant mortality in two provinces (Manica and Sofala) in Mozambique. Prevention of STI/HIV/AIDS (35% of effort) Objectives Improve the understanding, skills and behaviour for the protection and prevention of HIV in communities, women and pupils in the programme area Indicator Measurement method Information Baseline study Estimate : mid-term evaluation Information: Post intervention study 50% of women in the communities in the project area can identify three practical ways of preventing HIV (1) -Baseline and post￾intervention survey. – PAC reports -HAI – HIS 17% (survey) NA 33% (survey) 65% 2 or more 93% 1 or more 65% of the pupils from 12-17 years old can indicate 3 ways of preventing HIV -Mini-survey (club members ) -Monthly reports of STI/AIDS NA NA 60% (GTZ survey) 40 of CLCs have HIV prevention plans for their communities (25 in Manica and 15 in Sofala) -Monthly reports from SESP -HAI HIS 0 10 Total 7 Manica 3 Sofala 63 Total 45 Manica 18 Sofala 70% of the members of OMES reporting that they always or generally use condoms with causal partners -Mini survey of the members of OMES -Reports from OMES -Reports :head of HAI programme -HAI HIS NA NA 95% (OMES survey ) 50% of pupils participating in school anti-AIDS clubs reduce high risk behaviour due to fear of catching AIDS -Mini survey of participants in Anti￾AIDS clubs in schools –Reports from partners -HAI HIS -HAI reports NA NA 60% (GTZ survey) (1) 91% condoms, 37% faithful to partner, 27% reduce # of partners, 25% avoid relations with prostitutes, 11% don’t share razors, 9% avoid blood transfusions, 5% abstinence. Health Alliance International Child Survival Programme Final Evaluation 2002 11 Promote skills and practices for the prevention of HIV in health workers in the Health Units in the programme areas Indicator Measurement method Information Baseline study Estimate : mid-term evaluation Information: Post intervention study 90% of Health Units have at least one member trained and active in the teaching of the appropriate use of condoms -STI/HIV/AIDS Training reports – Monthly reports from SESP + STI/ HIV/ AIDS sections -Reports from the heads of Health Posts -HAI HIS 0% 68% Total (HAI HIS) 55% Sofala (HAI HIS) 91% Manica (HAI HIS) 74% Total (HAI HIS) 66% Sofala (HAI HIS) 100% Manica (HAI HIS) The Health Units have condoms available during at least 80% of the twice-monthly supervision visits. -Supervision visit reports -Reports from the heads of the health posts -HAI HIS 25% (HAI HIS) 100% (HAI HIS) 100% (HAI HIS) The Health Units with maternities have adequate stocks of latex gloves in at least 80% of the twice￾monthly supervision visits. -Supervision visits reports -Reports from the heads of health posts -Reports from the SESP sections of SESP, STI/HIV/AIDS, MCH 60% (HAI HIS) 91% (HAI HIS) 100% (HAI HIS) Maternal and Newborn Care (40% of effort) Objective Improve community knowledge about the use of antenatal, post-natal and newborn health care in the two provinces in Mozambique Indicator Measurement method Information Baseline study Estimate : mid-term evaluation Information: Post intervention study Increase the proportion of pregnant women that seek antenatal care in the first or second trimester of pregnancy from 40% to 60% -Monthly and annual reports from the MCH section - Household survey. Baseline and final - PAC reports - -HAI HIS 40% (HIS) 88% (survey) 49% (HIS) 51% (HIS) 89% (survey) Increase the proportion of pregnant women that seek post-natal care from 30% in Manica and 25% in Sofala to 50% in both provinces -Monthly reports from the MCH section –Household survey (baseline and final) -PAC reports 30% Manica (HIS) 25% Sofala (HIS) 72% Manica (HIS) 38% Sofala (HIS) 86% Manica (HIS) 42% Sofala (HIS) Health Alliance International Child Survival Programme Final Evaluation 2002 12 Indicator Measurement method Information Baseline study Estimate : mid-term evaluation Information: Post intervention study -HAI HIS Increase the proportion of mothers that know 3 danger signs for obstetric emergencies from 38% to 60% -Household survey (baseline and final) -HAI HIS 38% (survey) NA 16% (survey) Increase to 60% the proportion of CLCs that have an emergency obstetric transport plans -Reports for head of Health Post -Supervision Reports -HAI HIS -Training Reports - CLCs and TBAs 5% (HAI HIS) 3% (HAI HIS) 61% (HAI HIS) Increase the proportion of women who have given birth with the help of midwives, MCH nurses or trained TBAs from 55% in Manica and 30% in Sofala to 65% in Manica and 50% in Sofala -Monthly and annual reports from MCH ( district) -Household Survey (baseline and final) -HAI HIS 55% Manica (HIS) 30% Sofala (HIS) 88% (survey) 57% Manica (HIS) 41% Sofala (HIS) 46% Manica (HIS) 46% Sofala (HIS) 89% (survey) Improve the provision of antenatal care, post-natal care, care during labour, and newborn care in the programme areas. Indicator Measurement method Information Baseline study Estimate : mid-term evaluation Information: Post intervention study Increase the percentage of pregnant women in antenatal care that receive syphilis screening and correct treatment from 40% in Manica and 20% in Sofala to 80% in the two provinces. -DPS HIS -Monthly MCH report -MCH supervision reports -MCH training reports -HAI HIS -HF records -Laboratory records 40% Manica (HIS) 20% Sofala (HIS) 84% Manica (HIS) 81% Sofala (HIS) 81% Manica (HIS) 84% Sofala (HIS) Increase the proportion of Health Facilities that have plans for obstetric emergencies to 80%. -Reports de SMI -Training reports -HAI HIS 0% 5% (HAI HIS) 25% (HAI HIS) Increase to 80% the number of health personnel who can describe the key elements for antenatal, post￾natal, safe birth care, including emergency transport plans. -Training reports -Supervision Reports 0% 56% (HAI HIS) 100% (HAI HIS) Health Alliance International Child Survival Programme Final Evaluation 2002 13 Malaria Control (25%of effort) Objectives Improve the case management and prevention of malaria in children under 5 years old. Indicator Measurement method Information Baseline study Estimate : mid-term evaluation Information: Post intervention study Carry out a study on the levels of resistance to malaria drugs -Resistance study report. 0 2 done (1 Manica, 1 Sofala) Total of 3 carried out (2 Manica, 1 Sofala) Carry out a study on health seeking behaviour in mother and infants in four locations -Study report 0 Report completed Report completed Increase from 40% to 60% the proportion of pregnant women that are treated correctly under the NHS protocols -Household survey (baseline and final) -Training report from MCH -HAI HIS 40% (survey) NA 39% (survey) Increase from 50% to 70% the percentage of children with malarial symptoms that were correctly treated in the last two weeks in the two programme areas -HAI HIS -Special studies in two locations -PAC reports -Supervision reports 50% (survey) NA 86% treated in the health facilities 64% receive proper treatment (survey ) Implementation of a study for impregnated bednets in two locations in the programme areas. Indicator Measurement method Information Baseline study Estimate : mid-term evaluation Information: Post intervention study Carry out a study on the feasibility, costs and sustainability of the use of impregnated bednets in central region of Mozambique -Report of the study in for the pilot project 0 Plans to start the project Report completed. Replication of programme underway Increase by 10% to 70% the proportion of children who use impregnated bednets in the two focus areas. -Pilot Project reports -monthly reports from SESP -HAI HIS -PAC reports 13% (survey) 1% of total have impregnated bednets (survey) NA 25% (survey) 17% of total have impregnated bednets (survey) Health Alliance International Child Survival Programme Final Evaluation 2002 14 5.2. Narrative of Programme Achievements 5.2.1 Maternal and Newborn Care (40% of programme effort) Objective 1: Improve community knowledge about the use of antenatal care, post-natal care, secure birth practices and care of the newborn in two provinces in Mozambique. Lack of access to maternities continues to plague the Maternal and Newborn Care component of the Child Survival Programme The percentage of women seeking early pre-natal care increased over the programme period but did not reach the stipulated targets. From Health Information System data the percentage now stands at 51% (previously 40%). Indications from the field evaluation point to the distances women need to travel to reach health posts and the attitude of health workers who do not encourage first trimester visits as being the principle reasons for lack of compliance to early antenatal attendance. However, post-natal visits have increased and, in the case of Manica Province, surpassed targets set in the DIP for the programme (86% in Manica and 41% in Sofala). Emphasis during training sessions was put on this aspect of the maternal health care component. Training modules were developed that explained the importance of the post-natal visits. Despite the increased awareness and compliance in relation to post-natal visits, the number of women taking up family planning options remains low in both provinces. A disappointing result in terms of awareness of danger signs was found in the final KAP study with only 16% of women able to correctly identify three danger signs. The evaluation team considers that the result does not, however, reflect the true levels of knowledge of the women concerning danger signs in pregnancy, as 40% of women were able to identify at least two danger signs. Question formulation in the KAP study needs to be revised in order to measure this indicator adequately. The evaluation team considers that the results obtained are positive and demonstrate that the increased emphasis on education is beginning to increase awareness. The nurses interviewed during the evaluation consider that more effort is needed to ensure that all women can recognise the major danger signs. However, they are of the opinion that probably the greatest risk factor for women during pregnancy is linked to the distance that women need to travel to obtain emergency obstetric help, and not necessarily failure to recognise danger signs. One of the targets set by HAI for the Child Survival Programme was to guarantee that 60% of the CLCs had functional emergency transport plans. It was found that the target was reached and surpassed in both provinces. In discussions with the CLCs in Manica District (Manica) and Dondo District (Sofala) it was clear that the transport plans were functional, although far from ideal considering the distances covered by the populations to reach the nearest health posts. The transport available in the district is generally ox-carts or bicycles. There were cases where women were transported on homemade stretchers in areas where there were no other forms of transport. In Gondola (Manica) the Traditional Birth Attendants decried the fact that the numerous private mini-buses (chapas) were not helpful to pregnant women with difficulties and refused to take them if they were unable to pay. The CLCs stated that the transportation of the sick had been carried out before this initiative, but now it was more organised and there was a certain level of commitment from the members of the transport group. It was interesting to note that in Manica District two women were owners of ox-carts and members of the transport plan. The ownership of the means of production by women is not common in Health Alliance International Child Survival Programme Final Evaluation 2002 15 Manica and may have been due to the death of the male head of the household. The women showed a high level of commitment to the transport plan and claimed to have transported women during 2002. The statistics from the health information system (HIS) show a very pessimistic picture in terms of assisted births in the province of Manica with a decrease from 50% to 46%. Sofala shows an increase in assisted births from 30% to 46%. Even with the increase in Sofala the coverage of assisted births is very low. The survey results show 88% in the baseline and 89% in the Final KAP. The apparent contradiction probably occurs due to different sampling techniques (the KAP study is carried out in women living relatively closely to the health units). There may be some under reporting within the HIS by TBAs due to lack of incentives to maintain monitoring records. If women are over reporting institutionalised births this may reflect a desire on their part for this service, while the HIS statistics reflect the reality of the problems of the long distances that women need to travel in order to give birth in health units. Information from the evaluation visits support this view as mothers interviewed stated that they preferred to give birth in the maternities. One of the reasons noted by the nurses is due to the strongly held belief that a protracted labour indicates infidelity during the pregnancy and can lead to confrontation with in-laws during labour. Women also confirmed that they felt that maternities offered greater security for themselves and the newborn baby. Other constraints identified by the women interviewed were the illegal charges levied by the nurses in the maternities and the distances to the nearest maternity. Nurses highlighted the problems of reference health units and the lack of emergency obstetric intervention in the majority of peripheral health units. The lack of institutionalised births is a serious threat to women’s health and in the follow on Child Survival Programme, HAI will concentrate on the improvement of the skills of the elementary midwives and the MCH nurses in the peripheral health posts. HAI will also look at culturally acceptable waiting houses near the health centres for women who travel long distances to reach the health posts. Traditional Birth Attendants. Capacity building of Traditional Birth Attendants may not be the solution to high maternal mortality rates in Manica and Sofala HAI has been working with the health sector on increasing the skill levels of Traditional Birth Attendants for a number of years. However, numerous surveys have failed to prove that neonatal, peri￾natal or maternal mortality rates have been influenced positively by the resources spent on TBA training. During the evaluation the team met a number of TBAs who had received refresher courses during the last year, but they were not able to recall the material that had been covered during the training; still they insisted on the need for more courses. Most of the TBAs interviewed were able to identify danger signs during pregnancy, and were able to clearly state when pregnant women needed to be transferred for obstetric help. The MCH nurses that head the programmes have mixed views on the importance of the TBA programme. They recognise that the shortage of health posts and the distances travelled by women to reach health care make higher coverage of institutional births a goal that is difficult to achieve, therefore they need to rely on TBAs. On the other hand they recognise that women would prefer to give birth in the health centres. In the face of this reality both provincial MCH heads of department reiterated the need to continue with training and re-training, and clarification of the national policy in relation to TBAs. The question of renewable materials for the TBAs is a particular Health Alliance International Child Survival Programme Final Evaluation 2002 16 sticking point as the MCH nurses see these materials as being essential for the application of hygienic birthing practices taught during the training courses. At present the MOH is not providing these materials on a regular basis resulting in a breach in communication between the TBAs and the nurses in the peripheral health units. The materials in question are mostly soap and bleach. These items may not require external assistance to supply them to the TBAs, if the TBAs feel that they are essential for good practice. In the new phase of the Child Survival Programme HAI will not work directly with TBA training but concentrate on health worker training. The MOH needs to make a policy decision about the role of the TBAs in order to avoid using scarce resources on programmes that do not have a positive impact on the maternal and neonatal mortality statistics. Training MCH nurses benefit from extensive in-service training, and this is reflected in programme quality. HAI has financed extensive training, using and adapting nationally produced manuals and teaching materials. All training programmes have been carried out as part of the in-service training department of the two Provincial Health Directorates. HAI has adopted a system that involves carrying out training sessions, followed by supervision visits specifically to observe and support the application of the training to work practice. Further discussion of the efficacy of the training strategy can be found in the section of crosscutting issues. Both Health Directorates are appreciative of the training exercises financed by HAI. In the follow-on CS Programme, extra emphasis will be placed on training of staff in the peripheral health posts in the area of MCH practice. HAI has recognised the need to train health staff involved in some of the new initiatives now offered in the NHS; in particular training has been financed to support the capacity building of nurses and doctors involved in the pMCT programme in the central hospital in Beira and the Provincial Hospital in Manica. Heads of various programmes in the two directorates requested training in programme management. Heads of programmes are generally technical staff with professional training and have no management knowledge or experience. HAI should consider sponsoring a health management course at provincial levels. A further request was made by health staff for more training in research methodologies as they have found that the participation in studies sponsored by HAI has been professionally rewarding. This is encouraging as it indicates the commitment of some health workers to improving programme design and impact and is a welcome sign in a setting that is generally beset by a demoralised workforce. Objective 2: Increase the provision of pre-natal care, safe births, post-natal care and care of the newborn in the areas of the programme. Syphilis screening and treatment make huge strides in Manica and Sofala but stillbirth rates continue to be high. In general this objective of the programme was achieved and has proven to be extremely successful in increasing the capacity of nursing staff to respond to maternal and child health needs. The targets set for increasing syphilis screening during pregnancy was achieved in both provinces with an impressive 81% of screening and treatment carried out in Manica and 84% in Sofala. The nurses interviewed were Health Alliance International Child Survival Programme Final Evaluation 2002 17 extremely positive about the service offered and confirmed that the women were keen to take up the (free) treatment and generally completed the three doses. The nurses also stated that partners were compliant and were generally treated. Counter intuitively, men preferred to be treated within the antenatal clinic rather than the general clinic. Nurses were of the opinion that this was because of the time spent in the queues in the other areas of the service. Men paid 5,000 mt for each penicillin shot but this did not seem to be a deterrent. Some anomalies in service delivery were found that decreased the efficacy of the service, for example, where women needed to take prescriptions to pharmacies and return for treatment. It was concluded that the system is best served when testing and treatment were done at the same site at the same time (screening, treatment and follow-up). If it is possible, continue to have men come to the antenatal clinics; this could result in less missed opportunities and better counselling for the couple. Stock rupture of penicillin for syphilis treatment occurred both in early 2001 and 2002. In addition, there is a serious problem with the supply of syringes and needles (disposable). This has caused significant fluctuations in the testing and treatment of couples. This is extremely serious and could have a negative impact on the newly revived screening service. Even though the syphilis screening programme is considered an overwhelming success, there has as yet been little impact on the number of stillbirths in the two provinces that remains alarmingly high. Although health workers met during the evaluation were of the opinion that this could be due to high re-infection rates, this idea is not supported by work carried out in other countries that demonstrate very low re-infection rates after completed treatment. 1 One of the issues identified by the HAI team is the lack of testing facilities in the peripheral health posts. In order to overcome this problem, testing using a rapid strip test will be introduced in these health units in the second phase of CS Programme. It may also be necessary to carry out a further detailed study to identify risk factors for still births given the changing disease patterns linked to high rates of HIV infection. 5.2.2 STI/HIV/AIDS prevention The landscape of HIV/AIDS intervention in Mozambique has changed considerably during the four￾year period under consideration (1998-2002). Initially HAI was involved mainly in education, awareness raising, and community mobilisation. However as policy towards voluntary testing and medical intervention changed, HAI adapted their programme to the new environment. Objective 1: Improve the understanding, skills and behaviour for the protection and prevention of HIV in communities, women and pupils in the programme areas. Multi-faceted approach to community mobilisation pays dividends as the dialogue on HIV/AIDS prevention enters into the common discourse The CS Programme identified a number of indicators to measure raised awareness in key high-risk groups. HAI aimed to ensure that at least 50% of women were able to identify three ways of preventing HIV infection. The results of the final KAP study are misleading as only 33% of women were able to identify three ways of preventing HIV. However 65% of women could name two or more prevention strategies and 93% one or more measures. It is encouraging that the most frequent answer to the question was condom use (91%) followed by fidelity (37%). A quarter of the responses 1 Information provided by Dr Gloyd during the oral presentation of the evaluation results. Health Alliance International Child Survival Programme Final Evaluation 2002 18 identified reduction of the number of partners (27%) and avoidance of prostitutes (25%) as ways of avoiding HIV infection. Other answers included avoid sharing razors (11%), avoid blood transfusions (9%) and practising abstinence (5%). The results of the KAP are generally positive and can be attributed to the intensive work carried out through the varied awareness raising interventions, including health worker training, use of the media (especially radio spots and theatre), community theatre and youth focused work. The KAP does not give any indication of actual behaviour, as it measures knowledge and not practice; it is therefore difficult to judge whether people are in fact reducing high-risk behaviour. Another high-risk group identified by the CS Programme was youth (school pupils and adolescents). The target for the youth group was that 65% of the group could name three ways to prevent HIV infection. In the final KAP 60% of pupils were able to identify three ways of preventing HIV infection. This successful result was mirrored in the conversations held with both the school children in the Anti￾AIDS clubs and the adolescents in the drop-in centres in the Health Facilities. All the children stated that they had no problem talking about HIV/AIDS, that their messages wee well received and that awareness among their peer group was generally high. Some of the adolescents who had been involved in the programme for a number of years commented on the changes in awareness and willingness to engage with the subject of AIDS. When asked about the reasons for the changes they were clear that this was due to the number of deaths experienced by communities: `Every family has someone, or knows someone who has died of AIDS` [ Member of Canto jovial - Beira ]. Cantos Joviais – Adolescent Drop-in Centres So many adolescent boys talking about safe sex – the Cantos must be getting something right One of the ways in which youth was targeted in terms of education and awareness-raising was through the drop-in centres in the health facilities. The `Cantos Joviais` are spaces provided by the health centres for use by and for adolescents. The Cantos Joviais are part of a countrywide initiative to encourage adolescents to undertake peer education, self-education and seek treatment for STIs. The Cantos have been particularly successful in breaking down barriers between the health service and youth groups. The ease of relationships between the nurse service providers and the adolescents participating in the activities was impressive and augers well for increased openness and improved health-seeking behaviour by these young people. Both the health workers and the youth groups were interested in the Cantos transformation into Adolescent Friendly Clinics. At national level this policy has been adopted and is beginning to be implemented. In Manica steps have been taken to implement this new policy with the appointment of personnel at provincial level who will head the Adolescent Health Programme. HAI will continue to work with the DPS on the implementation of the Adolescent Friendly Clinics. The Cantos have not as yet fulfilled their potential in terms of STI-treatment- seeking behaviour, with the exception of demand for condoms. Take-up of the service for treatment of STI is still low; for example, the Canto in Dondo–Sofala has only treated 96 cases of STI since January 2002. One of the problems appears to be linked to the lack of permanently placed health personnel in the Cantos. This problem may be diminished when the national policy for provision of Adolescent Friendly Services is implemented and more emphasis is placed on reproductive health rather than sexual health. The management of the physical space of the Cantos needs to be decided upon on a case-to-case basis. In general terms the ideal would be for the service to be within the grounds of the health facility but Health Alliance International Child Survival Programme Final Evaluation 2002 19 not in a place that interrupts the normal business of the health centre. In rural health posts this is generally feasible where health centres have space for the Cantos. In city health posts this may not be possible. However, the physical link to the health services is considered to be extremely important as it allows for freedom of communication between health service providers and the adolescents. Problems mentioned by the youth (also focused upon in the UNICEF review of the Cantos) were the dropout levels in the centres, lack of gender balance, and the lack of financial support for the members. We would like to address these questions separately. Dropout rates: The strength and weakness of youth clubs is the dynamic membership. Youth clubs are characterised by change and renewal. This should be taken into account when discussing the success or failure of the Cantos. As the adolescents grow up and take on new responsibilities, they will necessarily leave the group, taking with them the knowledge and skills gained during membership. This is both a blessing and a curse. A blessing as they will hopefully use the knowledge gained during membership of the group in their adult lives and a curse because the club loses a resource. The key to the success of youth clubs is the renewal of membership, continuity provided by philosophy of the club and a culture of peer education of older members to new recruits. These issues should be tackled by the organisations supporting the Cantos. Continuity can be provided by regular recruitment drives, commitment of each member to train and pass on knowledge to at least two other members and a written institutional memory that provides a historical thread that can lead new members. Solutions can be found through insistence in the training of the youths on their roles as trainers. An institutional memory needs to be created in the Cantos. This could take the form of a newssheet (hand-written or computer generated) that documents events, education programmes and the successes of the group. These should be distributed but master copies kept in a folder for new members to read. The Cantos could be encouraged to programme regular (twice yearly) recruitment drives where they, not only put on community plays, music events, but also actively recruit new members for the club. Possible timing for the recruitment drives could be the beginning of the long holidays, using the 1 st of December (International AIDS Day) and a mid-year event (Independence Day or 1 st of June – International Children’s Day). Youth clubs often find it easier to work towards `events` as this mobilises energy with a specific short-term goal. Gender Balance The Cantos continue to fail to attract female members at the same rate as males. The majority of the members are pre-adolescent and adolescent boys. The UNICEF review claims that many of the members are not sexually active. On one hand the lack of gender balance is a concern as girls continue to be one of the highest risk groups for HIV infection. However, the adherence of adolescent males must be seen as a very positive aspect of the clubs. Adolescent boys are a particularly difficult group to reach in terms of health education and often do not have any contact with the health services unless they are sick. Women tend to have more contact with health services due to MCH services and caring roles for sick family members. If work carried out through the Cantos can break down some of the barriers that young men have about discussing reproductive health and sexual behaviour, even before they are sexually active, this should be a cause for celebration. Some suggestions about trying to attract more girls into the Cantos: ß Keep the Cantos in the health facilities as this provides legitimacy to the clubs and a measure of protection for girls that may not be present in community-based youth clubs. Health Alliance International Child Survival Programme Final Evaluation 2002 20 ß Ask the male members of the club to persuade their parents to let their sisters participate in the club, discuss with the boys that they will need to ensure the protection of their sisters. ß Consider setting aside some `girl` only sessions where the girls can come to the centre to discuss problems, receive training, and participate in club activities. Financial Rewards. Trained members of the Cantos have been paid a small subsidy through UNICEF funding (200,000 Mt – approximately $9 a month). At the time of the evaluation they had not received the subsidy for over two months. They confirmed that they were continuing to work on the development of the clubs even though they have not been paid; they were hopeful and thought that HAI would resolve the problem. The question of payment is complex from the point of view of the organisations supporting the Cantos and the adolescents. The organisations supporting the clubs would like to encourage the voluntary nature of the clubs, whereas the adolescents have a number of real economic pressures and they see working in the clubs as a way of resolving them (school material, transport costs, entertainment, etc.). There is no easy solution to these problems. One approach that may be considered is the encouragement of the entrepreneurial skills of the adolescents and for the organisations to help each Canto to develop a fundraising strategy. The strategy should include a number of elements and will depend on the local context. In general the youth should be encouraged to look at: ß Sponsorships (from the private sector, local NGOs, international NGOs, in exchange for community theatre presentations, educational talks) ß Income-generating activities: income-generating activities should be confined to activities directly linked to the aims of the club and not involve high investment costs (either in terms of capital or time). Possibilities: making HIV/AIDS emblems, making tee-shirts, putting on music and theatre shows ß Affiliation with established youth groups (Rotary –Interact, Scouts, Church youth groups) ß Staging once a year sponsored events – runs, cycle races, sponsored clean-up campaigns. The youth groups should be encouraged to become part of the AIDS networks, to make contacts with other governmental and non-governmental bodies. HAI has strong links with the youth members and with the national AIDS initiatives; they should review their role and begin to think as facilitators – connecting the youth with potential long-term partners, and not implementers. One of the surprising findings of the evaluation team, in all areas of education and awareness-raising, was the lack of IEC materials. This was true in the Cantos, the school Anti-AIDS Clubs, and the counselling offices. There does not appear to be a plausible explanation for the lack of materials given the number of organisations (Government and Non Government) who are working in this field, but in each of the places visited there were very little materials available. The adolescents were apt at giving talks on HIV transmission and prevention, family planning, secure sex, and use of condoms, and these talks formed the basis of their educational programmes. However, in order to maintain interest they need to have regular updates on information with attractive IEC materials. The youth complained about the lack of videos, reference material, new posters, and music. Although the development of material is expensive, it is not possible to consider that the educational campaigns will be successful if the market is not flooded with suitable IEC materials. Health Alliance International Child Survival Programme Final Evaluation 2002 21 Anti-AIDS School Clubs Pupils relaxed and discussed high-risk sexual behaviour in front of teachers and the evaluation team. Are we beginning to see the safe-sex generation? The indicator for the success of the Anti-AIDS school clubs required that 50% of students participating in clubs reduce high-risk behaviour. The data from the GTZ Report on School Clubs suggests that 60% of the participating members have reduced high-risk behaviour. Meetings held by the evaluation team with Anti-AIDS school clubs confirmed this finding in terms of the knowledge of the pupils of high-risk behaviour. Some examples given by the pupils of high-risk behaviour: ß Under-age sex when young girls go to bars and get drunk ß Having sex with other pupils ß Having lots of boyfriends or girlfriends ß Taking drugs (sharing needles) (Massange Primary School – Beira 2002 ) At present only six schools have been actively involved in the HAI-supported Anti-AIDS clubs. Both pupils and teachers involved in the clubs were able to express clearly the types of high-risk behaviour and ways to avoid infection. They do not seem to be ashamed or reticent to talk about sex and said they are not worried about using the model penis for demonstrations for condom use. The teachers in the Beira school visited also stated that there has been a reduction in the number of pupil pregnancies since the beginning of the Club work. It was not possible to verify this statement. The clubs are involved in peer-to-peer education, community theatre and at home awareness raising. The pupils stated that there were no problems in terms of talking about these issues, even at home. Initially parents required assurance from the school that the children were in fact participating in the club activities (to explain absences from home), but in the last year the teachers have not experienced any difficulties with pupil participation. There are a number of issues related to teacher remuneration that were raised during the evaluation. Teachers are not at present paid for running the clubs, and given their extremely low salaries, this is a considerable disincentive for continued teacher participation (they do not have time for other lucrative activities). The teachers also complained that they receive very little regular support for the activities; materials are in short supply, including costumes for the plays and instruments for the music sessions. Until teachers are paid a living wage and there is an improvement in their very difficult working conditions (between 60-70 children in a class), it is unrealistic to consider that public health messages can necessarily be transmitted via these professionals without some sort of remuneration package. It is not feasible or advisable for the Clubs to be encouraged in income-generation activities; therefore, organisations working in the field of HIV/AIDS need to find ways of supporting these initiatives. The Ministry of Education should be encouraged to find funds nationally for the Anti-AIDS clubs that could include top-ups for teachers who are committed to these initiatives. Health Alliance International Child Survival Programme Final Evaluation 2002 22 Local HIV/AIDS Prevention Plans CLCs adapt traditional culture to prevent AIDS proliferation The third group that was targeted by HAI in their community mobilisation work is the CLCs. The target set was to have 40 CLCs with clear local HIV/AIDS prevention plans. In total 63 CLCs have HIV/AIDS prevention plans (45 in Manica and 18 in Sofala). The work carried out with the CLCs is very important, and of the CLCs visited all had clear ideas about HIV/AIDS and preventive behaviour. The local prevention plans are not necessarily blueprints for action, but rather identify some issues that can be tackled in a culturally sensitive way to prevent the spread of HIV/AIDS. One example given to the group was the decision by the community in the Tabaco area of Manica District to modify traditional practices on the death of a husband. Traditionally after the death of a husband, the widow would be taken into the home of the husband’s brother, who would initiate a sexual relationship with the widow. This practice has been discussed in the community and has been modified in the case of a suspected AIDS-related death. Now the village leaders are recommending that the widow will be taken into the home of an older female member of the husband’s family. The widow is therefore under the protection of the extended family but does not have sexual relations with any other males of the family. They stated that if the widow accepts this arrangement she would be kept in the family. If, however, she wants to continue to have sexual relations she will be asked to leave and return to her father’s home. We were not able to explore the subtleties of this situation, or if in reality this is the case. In other countries the death of a husband through AIDS results in destitution for the widows. Another interesting area that was raised by the CLC group was linked to nutritional aspects of the care of people living with HIV/AIDS. They stated that the disease was very `hungry` and ate all of the food given to the person with HIV; therefore the person needed to eat more and more food of good quality to feed the illness and the body of the person. Once again we were not able to explore further implications of this view of the feeding of people living with HIV/AIDS and whether this would lead to preferential treatment of the person or the `starvation` of the illness that was feeding on the person’s food. Further investigation of these issues and others raised through the Local AIDS Prevention Plans could prove fruitful for community intervention strategies. In order for the Plans to be made into action plans there needs to be considerable work, both in terms of understanding more about the beliefs and views of people in relation to HIV/AIDS and the practical implications on behaviour. Once again the lack of good IEC material is a huge handicap for active CLCs who do not have sufficient information to guide positive behaviour in the community. Single Mothers Associations (Sex workers) Sex workers identify with health: `We are health workers, we educate in the fight against HIV/AIDS` (Sex workers – Beira) HAI has worked with two single mothers groups active in HIV/AIDS prevention work, one in Beira and another in Manica. The target was for 70% of the women in the single mothers group (sex workers) reporting that they use condoms. In the final survey 95% of women reported using condoms. This is a very high compliance rate and probably reflects more a desire to use condoms in casual sexual relations than the reality of use. On the positive side the desire to use condoms is a step towards HIV prevention. In both of the single mother groups they reported the ease with which they are now able to carry out their nocturnal educational activities, in sharp contrast to when they began their activities. In the beginning they were chased out of bars and nightclubs, threatened with violence and Health Alliance International Child Survival Programme Final Evaluation 2002 23 could not suggest use of condoms with clients. Now the bar owners and clients welcome them and the up-take of condoms out-strips the quantity they are able to supply. During discussions with the groups of women they requested that more female condoms be made available. When asked about their reasons they stated that women would then not have to rely on male compliance, that women could control the safety of the sexual encounter, and that they could protect themselves from broken condoms. One of the positive aspects of the work with the single mothers groups is the identification that the women feel with the health sector. They refer to themselves as health workers. The breaking down of the traditional barriers that distance health from the community was one of the most interesting and positive added values` aspects of the HAI work under the CS grant. Objective 2. Promote skills and practices for the prevention of HIV in health workers in the Health Units in the programme areas Health staff learn about bio-security at work and put the theory into practice HAI aims to strengthen the NHS in order to improve service delivery. As part of this approach HAI have spent considerable resources training health workers and strengthening supply systems in order that health workers can work in a secure working environment. One of the targets of the CS programme was that 90% of Health Units would have one member of staff trained to demonstrate the use of condoms and is actively using the knowledge in the health centre. Although the target was not fully met in Sofala where only 66% of the health facilities were covered, an impressive 100% of Health facilities in Manica were covered during the programme period. Sofala still needs extra resources if it to meet the target of 90% coverage. Health worker protection was measured through the provision of plastic gloves. This was achieved in all health units in both provinces. A recently introduced law gives the right to health workers to retroviral treatment in the case of proven contamination in the work place. New perspectives of health worker training on bio-security and implications of the implementation of the retroviral law will be undertaken by the STD sector of the provincial directorates. Other achievements i. Mother to Child Transmission Programme (MTCT) The beginning of the pilot MTCT service in the provincial hospital of Manica and the central hospital in Beira has been welcomed by patients and by practitioners. The MTCT service receives pregnant women who are referred by the Voluntary Counselling and Testing Centres (VCT). The women will receive a single dose of an antiretroviral drug during labour in order to cut the rate of mother to child transmission. All people interviewed in relation to the programme were extremely positive. Health workers feel that at last they can offer some help for HIV-positive mothers. In Manica the Provincial Office for the Co-ordination of HIV/AIDS Prevention Programmes has taken the initiative to offer women enrolled in the pilot project weekly group therapy sessions. The sessions aim to provide women with information about living positively with HIV (life-style information, nutrition information, etc), provide emotional support for the women, and provide further information about the pilot study they are participating in. Health Alliance International Child Survival Programme Final Evaluation 2002 24 Given the importance of the pilot MTCT, more work should be carried out to clarify for all participating health practitioners the protocol of the study, the objectives of the MTCT service, the scope of the service, and ways to explain the service to the pregnant women. The nurses in the study are not clear about all of the possible implications of the treatment. It is extremely important that the nurses who are recommending women to enrol can respond to all of the possible queries that they will face from the women. The Provincial Health Directorates should define a clear strategy about the public information in relation to this initiative as information is already available in the health centres (as far away as the satellite town of Dondo), but the information is partial and may lead to unnecessary confusion for both families seeking the treatment and the health practitioners that need to refer women to the programme. MCH nurses working in the antenatal clinics requested that they receive more IEC material to help with their talks to the women. At present they give the talks before the antenatal sessions, but this is not conducive to clarifying doubts or fears of the women. If the programme moves forward to include more maternities, thought should be given to the IEC needs of the programme and delivery methods. MCH nurses consider that counselling should be as far as possible an individual activity with the patient. The Director of the Eduardo Mondlane Health Centre in Manica suggests that there is greater adherence to treatment norms including partner contacts when the sessions are held in private where the importance for the individual of the partner testing can be emphasised. He feels that the health practitioners should spend less time on public talks and more time on individual counselling. Health workers working on the pilot of the Mother to Child Transmission programme should be given training in counselling skills in order that they can support the women and their partners. The Prevention of Mother to Child Transmission programme will cause considerable extra work for the maternities in the central hospital in Beira and the provincial hospital in Manica. Consideration should be given to identifying alternative maternities in the provincial capitals that could be used as a base for MTCT programme, thereby allowing the main hospitals to treat obstetric emergencies. ii. Voluntary Counselling and Testing Centres (VCT) HAI has supported the opening of VCTs in the two provinces and are proving to be very popular (up to 300 clients per month). The Chimoio VCT that was inaugurated in October 2001 has tested 23,722 people. The majority of the centres are in health centres, where they have been provided private spaces for the service. Both provinces have conducted extensive (and expensive) publicity campaigns on the radio, as well as using the community networks and this has resulted in a growing number of clients. Concerns were initially expressed about placing the VCTs in the health facilities, as the lack of anonymity may have been a barrier to use. However, this has not proven to be the case, and it appears that the busy health centres provide a secure environment for the service. Each VCT has three counsellors who received short training courses. The majority of the counsellors are health workers as this is seen as an advantage in terms of the testing and understanding of the medical aspects of the service. The counsellors feel that they need more skills in counselling as they are facing situations that were not covered in the original training course. The counsellors are under considerable stress as up to a third of the clients test positive, and they have to be in a position to help people to understand the implications of their situation. The counsellors are confronted with many ethical questions that are hard for them to handle given their low levels of training and experience. For example they cite cases of couples where one partner is positive and the other negative, they do not Health Alliance International Child Survival Programme Final Evaluation 2002 25 have enough training to manage this situation. They are also worried about some of the technical aspects of the work. One example from the Chimoio VCT concern inconclusive tests. The counsellors wanted advice about what to do if one test is positive but the other test inconclusive; they are recommended to ask the client to come back in three months, but they have found that the same problem persists. They do not have sufficient professional training to resolve this issue. A careful study should be undertaken to look at all the issues concerning the counsellors in order to begin to design an in-service training course to address the main concerns of the VCT staff. The DPS needs to consider the psychological needs of the counsellors. GTZ sponsored a retreat for the staff from the VCT, to rest and discuss the problems they are facing. These initiatives should be encouraged and become part of a regular system for the VCT staff. The VCT waiting rooms were clean and well appointed. Rehabilitation of the facilities has been carried out to provide a welcoming environment. The counsellors in two of the VCTs were worried about the lack of ventilation in the counselling rooms. They are particularly concerned about TB contamination due to the high numbers of HIV positive clients. Once again the team found a shocking lack of IEC materials suitable for VCT waiting rooms. The counsellors also commented on the fact that they do not have materials to give to the clients after the counselling sessions. In particular all health workers requested more copies of the Living Positively booklets (Vida Positiva). There is an urgent need to stock the VCT with IEC materials. The VCT programme should be expanded and supported by HAI in the next phase of the CS. The training needs of the counsellors should be co-ordinated with the national programme. The need for IEC materials at all levels of attention and implementation in the field of HIV/AIDS should be addressed by the National AIDS programme as a matter of urgency. Concerns about the effect of the increased numbers of clients due to the MTCT programme led to suggestions of specific counselling services for women referred through the antenatal clinics. 5.2.3 Malaria Prevention Programme: Objective 1: Improve the case management and prevention of malaria in children under five years old. HAI proves the usefulness of operational research in policy formulation In order to achieve this objective HAI worked intensively with health workers training to improve case management and at policy level to improve treatment protocols. HAI carried out three drug resistance studies that were used by the National Malaria Programme as part of the discussion to change the treatment protocols. A further study was carried out to look at health-seeking behaviours of mothers with feverish children. The resulting study provides important information about the classification of fevers and explains to a large extent the complexity of health-seeking behaviours. Despite considerable investment in re-training of all health staff in peripheral health posts, the results of the final KAP survey do not show any reported improvement in malaria case management for pregnant women. This may be due to the fact that the KAP study has certain intrinsic limitations in measuring appropriate case management. The improvement in case management of infants showed an Health Alliance International Child Survival Programme Final Evaluation 2002 26 increase from 50% to 64%, with 86% of mothers seeking treatment from the health facilities for feverish infants. Additional testing of KAP questions should be carried out to ensure more accurate measuring of case management of key disease indicators. In general HAI achieved the goals set under this objective with both increased awareness of mothers to the dangers of high fevers and the improvement in the case management by the health workers. Objective 2- Carry out a study for the introduction of impregnated bednets programme in the programme areas. Traders welcome bednets and CLCs. HAI needs to learn from other rural development projects the pit-falls of sustainable commercial initiatives. HAI is involved in implementing, with the Provincial Health Services, pilot bednet initiatives. A study was carried out in Sofala that led to the expansion of the initiative to Manica. The initiative is based on the subsidised selling of bednets through local traders or through the CLCs. The programme set as a target an ambitious increase in bednet use by under-five-year-old children (from 10% too 70%) in four target districts in the two provinces. This was not achieved during the life of the programme and the final KAP study indicated that only 25% of children sleep under bednets and only 17% have impregnated bednets. However, in contrast to the findings of the KAP survey the evaluation team found that both the commercial traders and the CLCs involved in the programme are extremely enthusiastic about the programme. Traders in both Manica and Sofala are selling the bednets and creating sustainable schemes for the continuation of the sales. Some of the mechanisms that are being tried by the traders include: ß Exchange of bednets for agricultural products ß Offering credit and re-payment terms to clients (Nhazonia) ß Mobile trading (going into the remote rural areas away from the district centre – Nhamatanda)) ß Setting up wholesaler business to supply other traders in the area The programme is also testing the possibility of the CLCs selling of bednets as part of an income￾generating scheme. The CLCs did not present any difficulties to the evaluation team and were pleased with the margin of profit made from the sale of the nets (between $5-$7 dollars per net). As a small income-generating scheme linked to health mobilisation activities the use of the CLCs has some merit. However, certain aspects of income generation by community structures need to be followed carefully by the programme staff in order to avoid problems with the control and use of the funds raised under this initiative. Some considerations: ß Control and use of funds. CLCs do not have legal status and some of the CLCs cover remote rural areas that do not have banking facilities. Mechanisms for the control of the funds raised by the CLCs should be discussed with the members in order to guarantee transparency in the use of the funds. Example could be taken for the village saving schemes where there is joint control over the cash box (e.g. CARE – Vilanculos) ß Inclusion of women from the CLCs in the scheme ß Legality of the CLC as traders. If CLCs are selling bednets for profit, they should be registered as traders and subject to fiscal control. This should be discussed with the Provincial Health Directorate and the Provincial Directorate for Commerce and the District Administration. HAI Health Alliance International Child Survival Programme Final Evaluation 2002 27 should seek advice from Population Services International, who has been encouraging community vendors within their Social Marketing Programmes. Although the DPS in each province is a collaborating partner in the bednet scheme, there is a need to increase the integration of the district level health authorities in the programme. The District health staff was not involved in the decision-making about the sale of the bednets and the vehicles for commercialisation. HAI is concerned about the price of the bednets and the ability of the poorest families to support the cost. In the second phase of the bednet programme a further subsidy will be introduced for pregnant women and children. This will add another layer of complexity to the commercialisation process that will have to be carefully monitored in order to ensure that there is a workable and sustainable system in place at the end of the programme period. At present the HAI bednet initiative has been concerned mainly with demand issues, ensuring that communities are willing to buy and use the bednets and that there are people who are willing to sell them. However, one of the critical aspects of all commercialisation programmes lies in the supply lines. HAI is at present facilitating the purchase of the bednets and delivery to the districts. To ensure sustainability of the initiative it is essential that in the next phase of the programme that the supply aspect of the programme is resolved. Many rural development programmes have failed in terms of sustainability due to a lack of attention to the supply side of the economic equation. It would be useful for the programme manager to consult with other agencies that have worked on rural water programmes or the commercialisation of agricultural in-puts using traders or community-based associations. 6. Crosscutting approaches 6.1 Community Awareness Great strides have been made in the past four years in raising community awareness of the importance of the major issues tackled by the Child Survival Programme and bringing health services closer to responding to the needs of communities. Community awareness building has taken place on a number of fronts that have been either innovative or building on solid previous experience. Challenges still remain in the public health sector due to the absolute poverty of the majority of the population in the areas (See poverty maps in the first section of the report), the paucity of resources, both human and structural to provide services for the populations and an under-motivated and under￾paid work force. HAI has carefully worked within these difficult constraints and tried to maximise the benefits of the health service. “Cordless Telephones” (description of the CLC by head of CLC in Tabaco, Manica District) Successful initiatives have built on the Community Leaders Councils (CLC) and this has resulted in solid communication lines between the community representatives and the peripheral health units. One of the leaders of a CLC in Manica District referred to them as “telefones sem fio,” or cordless telephones. Both the communities and the health sector appreciate the exchange of information, ideas and initiatives. Concrete examples of this partnership can be seen through the development and use of transport plans for obstetric emergencies, the extensive mobilisation work carried out by the CLCs, interest in the HIV/AIDS issues, the beginning of culturally acceptable plans for combating HIV/AIDS in the community, and finally the beginnings of the initiative for the selling of bednets through this community council. Health Alliance International Child Survival Programme Final Evaluation 2002 28 CLCs have also been included in wider planning initiatives in Manica Province, where they were invited to be part of the strategic planning process, airing views on the needs of the communities and ways of tackling health problems faced by the communities. It is clear that in both of the provinces the CLCs are considered to be integral and useful parts of the health network by health practitioners and the communities. A national policy for the development of health and community linkages is under discussion in the MOH. The successful implementation in Sofala and Manica of the CLCs is one of the examples that is under consideration. CLCs in Sofala are calling for official recognition by the authorities of their role. The demands for recognition range from a simple request for identification badges to the setting up of legally recognised associations. The officialisation of the CLCs needs to take into consideration the implementation of the governmental decree on recognition, roles and responsibilities of traditional authorities in relation to the public sector (Decree 15/2000), that is now in vigour in the country. One of the dangers that may exist for the future of the CLCs is the overloading of this organisation with too many tasks that are not necessarily within their mandate. As few functional community structures exist in the rural areas, there is a tendency to look at the CLCs as vehicles for many different initiatives. As tasks increase so does time spent on the tasks, and this will begin to increase the demands for financial reward, which could seriously undermine the CLCs. HAI aims to work, in the next phase, with the CLCs on a number of issues. It would be pertinent to debate the advantages and disadvantages of each of the new initiatives in the light of the overall tasks of the CLCs and decide whether they are the most appropriate vehicles for this particular programme component. We would recommend that this subject is fully debated within the organisation, with the DPS and the CLCs in order to decide on a policy direction. In the opinion of the authors of the report the CLCs are at their best when carrying out the following functions: - mobilising community participation (sporadic) - capitalising on and organising initiative already existing in the community – for example transport plans - raising culturally sensitive issues and finding local solutions – for example HIV/AIDS plans - participation in planning processes It is very important to include the CLC in the health planning process. In Manica the CLCs are included in the bi-annual and annual meetings in order to listen to the community point of view and plan activities. The CLCs were also involved in the strategic planning exercise. More care should be taken when introducing revenue-raising initiatives – such as bednet sales as these often exclude women’s participation due to lack of numeracy literacy and may cause conflict with the CLC. 6.2 Youth groups It is notable that HAI initiatives through the CS Programme have worked tirelessly with youth groups, especially on issues linked to HIV/AIDS, although there appears to be some negative feeling about the sustainability of the initiatives due to high drop-out rates, low treatment take up rates for STD, and the never-ending questions of financial remuneration. The other side is the excellent mobilisation that has Health Alliance International Child Survival Programme Final Evaluation 2002 29 occurred with young people openly talking about sexuality, unwanted pregnancies, and feeling comfortable to spread messages into communities, at home and to peers. The changes in attitudes noted by the youths themselves can only be commended. Behaviour change is a long-term project and cannot be judged within a narrow timeframe. There are successes – enthusiasm and belief in the health issues they are promoting, self-confidence in expressing views on HIV/AIDS, drugs, and delinquency are very positive steps. The high adherence to the programme by young men is also extremely positive as they are seen by communities as one of the most vulnerable groups in these times of high unemployment, violence, and criminality. Challenges are how to make the school clubs, the cantos, renewable (sustainable). This will occur through the passing on of information and skills within the group to new members, and the constant renovation of the groups through recruitment drives. Possibilities of financial sustainability could be looked at if it is felt that outside funding is not available for these initiatives. The fundraising should be kept as close as possible to aims and objectives of the group – for example, one should encourage the improvement in the quality of the theatre and music promotion so that they can be sponsored to give concerts and shows, they could become involved in the production of tee-shirts with HIV/AIDS slogans for commemorative days and organisations working in HIV/AIDS. They could look at beadwork and the production of ribbons and other HIV/AIDS emblems. The groups should not be encouraged to look at other income-generating activities, such as chicken-rearing, vegetable-growing or carpentry (suggestions by the youth themselves) as these require high levels of inputs, commitment, and financial management that are not available in the groups and will not add value to their work. 6.3 Training Strategy Extensive training has been carried out with health workers as the primary providers of health care to communities. Health workers have been involved in all of the community initiatives supported by HAI; this is extremely important as this approach has created fertile ground for partnerships between the health service and the community, and has not created conflict in this resource-poor environment. The focus of the training has been to encourage increased use of the health facilities, improved treatment and services offered, and increased bio-security for health workers. HAI has consistently worked with the in-service training departments of the provincial health directorates and responded, within the stipulated areas of action, to requests for in-service training. One of the questions raised by partners in the health service was that,although generally HAI supported requests for the financing of training, HAI did not provide information in advance about the amount of money available for training in any given area, which resulted in a lack of prioritisation. If HAI was able to stipulate the funds available for training in the different areas of interventions, the heads of department would be able to improve their annual action plans. HAI has strengthened, not only the training programme, but also supervision of the trained staff, providing support for the implementation of the newly acquired skills. This approach is extremely important. However, one of the principle constraints to improved health services, the low technical level of the staff, continues to have a negative impact on the population. HAI needs to look at the training Health Alliance International Child Survival Programme Final Evaluation 2002 30 component of their programme through a different lens and begin to consider intervening at the beginning of the professional life of the health workers and not in mid-career. Working to improve the curriculum, teaching, and reference materials for student health workers will have a positive impact on the quality of health care in the two provinces. HAI has accumulated considerable training experience and, with the provincial directorates, developed teaching materials. Future training interventions should contemplate working directly with the provincial Training Institutions to identify how HAI can support initial training courses. The Training Institute in Beira trains both basic and diploma-level health workers, and the Training Centre in Manica trains basic level health workers. Possible areas for interventions could include refresher courses for teachers, provision of up-to-date teaching materials for the Institutes, training of teachers in the use of the materials, and provide supervision for final-year students in fieldwork. 6.4 Advocacy HAI has produced high-quality and convincing research over the life of the programme (See Attachment B for a complete list of studies). The work has been highly appreciated at provincial and district level. There are calls from the health workers at all levels of the health service for HAI to continue to support operational research. Health workers have actively participated in the research and this is an essential part of the capacity building programme supported by HAI. However, HAI continues to have less than optimal influence on policy-making (centralised) due to the geographic location of the management of the HAI programme, and therefore should consider central level representation to strengthen influence on important policy debates. HAI needs to consider institutionalising the operational research capacities through the strengthening of the Provincial Directorates investigative capacity. HAI staff has made considerable efforts to keep abreast of the changes in the HIV/AIDS field and are linked to the main national networks, however, once again the lack of central representation means that they have to rely on network partners for up-to-date information on policy debates. One of the major issues for HAI in the next few years will be access to the joint funding initiatives through the SWAP processes supported by many donors (Sector Wide Approach to Funding). In order to represent adequately the value of the HAI programme, the organisation will need to raise their profile at national level 6.5 Information Management HAI regularly produces a bewildering number of reports. Staff members are expected to produce weekly, monthly, quarterly, six monthly and yearly reports on activities. In addition donor-specific reports and regular written updates to the two provincial health directorates are produced. Further reports are written on specific activities undertaken, for example, training courses and seminars. HAI also produces reports on studies carried out and regular presentations of results to provincial, national and international forums. A very impressive paper trail. HAI shares information with partners and although generally report writing is not a joint enterprise, all information is shared. HAI does not, however, share financial information with the Health Directorates. This omission has a negative impact on the planning and prioritising of activities by the Health Alliance International Child Survival Programme Final Evaluation 2002 31 partners in the health service. Information provided about the amount of funding available for specific activities (in particular training and supervision) would improve provincial level planning. There is a need to rationalise the reporting culture in HAI to encourage a more critical review of project implementation. Progress against planned activities should be charted using matrices to save time and make the information more accessible for evaluation. Reports, on the other hand, should be used as discussion platforms, identifying successes and constraints. The move from producing list reports to producing analytical reviews is not easy. One approach could be to hold quarterly meeting where each staff member is asked to identify (1-2) important aspects of their component, either successes to be shared or constraints to be overcome. These aspects would then be discussed by the team in order to make links between components, reaffirm approaches, and look for value-added improvements that can be made in each component to achieve the global goal of the programme. Team members must feel confident that they can share difficulties and problems with implementational success. The exercise will not work if team members feel competitive or need to hide problems. The HAI programme for the next three years will be significantly enriched if sharing across components can be achieved. The underlying approach, health service strengthening, is shared and many of the components are inter-linked. This presents the organisation with an opportunity to increase shared learning leading to improved programming and ultimately having a higher impact for beneficiaries. One of the constraints to improving the learning culture within the organisation is the demand from donors and HAI Headquarters for activity reports. Results-led (as in activities completed) reporting is time-consuming and leaves little time for more critical or analytical thought. HAI Headquarters should reconsider field office reporting requirements and support lobbying of donors for moves towards more analytical, process-led reporting. 6.6. Sustainability strategy The operational approach of HAI lends itself to sustainability with emphasis on the institutional strengthening of the health service through capacity-building and improvement in health delivery systems. Working closely with the health service is challenging due to the absolute lack of resources available within the public service and the low motivation levels of many of the health staff. This can lead to frustration for both HAI and the NHS, as levels of expectations can be very different. HAI works directly with the heads of programmes in each of the health directorates and conflicts arise over resource sharing, rhythm of work and occasionally policy directions. However, in general both partners feel that the effort is worthwhile and that mutual learning is achieved. HAI should continue to engage with the Health Service directly and use the programme resources to institute long-term changes wherever possible. This can be achieved through introducing into the training strategy support teaching in the Health Science Institutes (initial health worker training courses) and in encouraging more efficient planning processes in the health directorates through sharing financial information with programme planners. HAI should continue to encourage and support operational research that has lasting effects on the quality of the health service offered. The extension to the Child Survival grant will continue to work on the components of the programme started under the first grant. The second CS initiative is more focused and will play to the strengths of the organisation while still responding to the needs of the population in the programme areas. HAI will continue to support community mobilisation work in strict collaboration with the health service and more emphasis will be placed on supporting the HIV/AIDS initiatives of MTCT and VCT. The findings of the evaluation team fully support the broad areas of intervention identified in the extension grant. Health Alliance International Child Survival Programme Final Evaluation 2002 32 7. Programme Management 7.1 Planning The planning process for the identification of the components of the Child Survival grant were discussed and approved by the health authorities before the start of the programme. Throughout the life of the programme the health authorities have approved annual plans. HAI has regular review meetings with both health directorates to discuss progress on the planned activities. As mentioned previously in the report, there was little financial transparency with the health authorities, which led to less than optimal planning exercises with some of the specific departments-sections in the directorates. This is an area that should be improved upon during the next programme cycle. During the life of the programme (1998-2002) significant changes have occurred in relation to HIV/AIDS, both in the global debate and the situation on the ground in Mozambique. HAI was required to react to the situation and respond to the needs of the health sector in terms of VCT and the pilot Mother to Child Transmission programme. This has been covered in the achievement section of the report. Suffice to say that HAI acted with integrity and moved into an area of work that is beneficial to the population and will strengthen the health service response to the epidemic. 7.2 Staff Human Resources and Staff Management. All programme staff are Mozambican with the exception of the HIV/AIDS co-ordinator, the MCH advisor (long-term Mozambican resident) and the Country Coordinator. The staff is committed and competent. The role of the HAI staff has changed over the reporting period. Initially HAI staff was considered technical advisors; however, they now play more of a managerial role. They manage the planning and financing of activities with partners and are less involved with direct capacity building of counter-parts in the health directorates. Some of the staff members are actively involved in training seminars, but increasingly expertise is sought within the health service to carry out the training. This shift needs to be explicit in the staff job descriptions and shared with the health authorities. There is an urgent need to carry out HIV/AIDS training with the HAI staff. The training programme should not concentrate on HAI HIV/AIDS activities outside of the organisation, but be tailored to help staff members to face both personal and professional challenges. Organisation policy on treatment of HIV/AIDS, voluntary testing and employment policy should be thoroughly explored in an open and supportive environment. HAI should seek outside facilitators to carry out the work in order that all members of the team are seen to be confronting the issues. Supervision of Programme staff HAI staff has regular meetings in country and supervision from Headquarters. This has a positive effect on staff cohesion and staff morale. HAI could take more advantage of the high-calibre staff by encouraging reflective learning within the organisation. (See Information Management section above for a discussion of improving the reporting culture in the organisation.) HAI staff have been actively involved in changing the supervisory culture in the health service, encouraging supervision for support and improvement and not for policing. Partners see this as Health Alliance International Child Survival Programme Final Evaluation 2002 33 extremely positive. However, the financial cost of regular supervision continues to be problematic, as the health service does not have sufficient funds to support an adequate supervisory system. Financial Management HAI has instituted good financial management practices, with annual auditing carried out by external auditors. Auditors have raised no issues over the life of the CS Programme. HAI carries out a very lean in-country operation, with low expenditure on expatriate staff and capital investment. The fleet of cars used by HAI is well maintained, but also old and in need of renewal. HAI has office space within the DPS in Sofala and modest main offices in Chimoio. National staff costs are in keeping with other similar NGOs in the region. There was no budget adjustment beyond the 25% allowed within the agreement. Technical, Logistics and Administrative Support The recruitment of a senior advisor for the Beira office will relieve some of the programme constraints caused by the double role of the Country Coordinator as both Coordinator and Technical Advisor. The Country Coordinator (CC) will need to concentrate on strategic and policy issues arising from the new programme components and be less involved in day-to-day administrative questions. Given the new funding sources that have been identified, extra management support should be employed. The CC should take a pivotal role in shaping HAI into a learning organisation with a strong analytical culture. No logistical or administrative problems were found during the evaluation and interviews with staff. The problems of running two offices are largely overcome through regular visits and email-telephone communications. It may be necessary to increase the administrative support to the Beira office as some of the new programme initiatives are implemented. Management Lessons Learnt HAI needs to re-define their organisational structure to take into account the new programmatic challenges. The need to do this has been identified by both partners and HAI staff. Reporting demands are overwhelming the organisation and strangling the critical faculties of the organisation. The negotiation of less onerous reporting requirement needs to be carried out with partners, donors and Headquarters. HAI needs to engage actively with major policy debates in the country and place the organisation in a position to take advantage of new funding initiatives (joint funding through SWAPs for example). It may be necessary to appoint a HAI representative in Maputo to fulfil this role. 8. Conclusions and Recommendation 8.1 Conclusions. HAI has achieved remarkable success in reaching the targets set in the DIP for the Child Survival Programme. However, there are still severe health problems in the central region to be tackled, further Health Alliance International Child Survival Programme Final Evaluation 2002 34 exacerbated by HIV/AIDS epidemic that has changed the epidemiological landscape. One of the principle achievements of CS grant has been the strengthening of the health structures from provincial to health post levels through the extensive capacity building programme in an extremely difficult operating environment. HAI has also, during the four-year period, identified some of the critical medical supply issues and worked with the NHS to minimise the disruption to health services. HAI has chosen an implementation approach that is dependent on the resource-poor NHS. Many of the achievements of the Child Survival Programme are also the achievements of the NHS. In order for this approach to continue to be fruitful, the organisation needs to establish goals and timetables taking into consideration the pace and capacity of the supporting health service. HAI should endeavour to increase integration of planning strategy and activities in order to ensure that the NHS can accompany the rhythm of intervention and take advantage of all the capacity-building efforts. HAI was successful in carrying out the activities stipulated in the DIP for the Child Survival Programme. Some of the outcomes were not as successful as initially hoped for, especially in terms of impact on health-seeking behaviour and reduction of high morbidity-mortality rates. Problems still exist in terms of treatment of STDs, high maternal mortality and neonatal deaths. There continue to be problems in terms of awareness of health problems (obstetric risk) and high-risk behaviour (HIV/AIDS). However, strategies adopted by HAI working with the NHS and communities are beginning to pay dividends. For example, work with sex workers and the adolescent groups shows a high level of commitment of these groups to the programmes and high levels of identification with the health service as a partner in the fight against HIV/AIDS. These programmes should not be abandoned but improved so that the effects of changes in cultural perceptions lead to increased use and access to health care. In the extension CS grant issues linked to these problems have been identified and will form part of the new programme. Over the period of the grant HAI responded to the rapid changes in the HIV/AIDS field by supporting important provincial initiatives that were in accordance with the overall objectives of the organisation, namely in the areas of the Mother to Child Transmission and the Voluntary Counselling and Testing Centres (VCT). These interventions have been taken on by HAI in a thoughtful and inclusive way, moving forward as policy has changed. The importance of these initiatives is reflected in the extension grant for the Child Survival Programme. HAI’s attempt to engage all community stakeholders in the resolution of targeted health problems has not been wholly successful, for example the lack of follow-up on work with “curandeiros” (traditional healers) and church groups. HAI has learnt over this period that they are more suited to working with community awareness strategies strictly linked to health service delivery and this will be the focus of their future work. HAI should take a facilitation role in support of the networks for community based initiatives and not as implementers of these initiatives. HAI should continue with the approach of working through the NHS, and work to the organisation’s strengths in terms of health worker training, operational research, and innovation in cutting edge areas of policy (Mother to Child Transmission and antiretroviral treatment). All aspects highlighted by HAI for the next phase Child Survival grant are considered to be priority areas for intervention by the provincial authorities and are more focused than the previous Child Survival Programme. HAI needs to consolidate its internal structure in order to clarify roles of personnel within the organisation and their responsibilities towards partners and donors. The current reporting culture in HAI is not conducive to reflective learning and is seen as a control measure rather than an analytical Health Alliance International Child Survival Programme Final Evaluation 2002 35 tool. New mechanisms need to be devised to encourage organised reflection that can stimulate staff members to develop critical analysis of their programmes in a supportive environment. This is a challenge and not easily achieved given the pressure on the organisation to fulfil donor reporting requirements, however we feel that it is extremely important that HAI maximises the use of the considerable skills of the staff to improve programming and not merely report on activities carried out. . 8.2 Lessons Learnt : HAI will work to their strength as an organisation and have a more focused operational approach. HAI will concentrate on strengthening the MOH partnership, increasing capacity for operational research, and ensuring the responsiveness of the NHS to community needs as expressed through the various community-based forums. HAI will avoid moving into areas where they do not have the resources or vocation to adequately address the problems. HAI will rather strengthen their role as a facilitator and gradually reduce their implementational role. HAI has learnt the need to embrace new challenges when they are deemed to be of overwhelming importance and urgency, even if they were not necessarily foreseen at the beginning of an operating period. This is especially pertinent in the HIV/AIDS field where national policy, science, and practice dictated significant changes in implementation emphasis. During this operating period it has become increasingly clear that HAI needs to consider national level representation. Policy formulation is still a centralised process in Mozambique and with considerable experience gained from both research and implementation there is a need for HAI to take a more active role in advocating for change. HAI of all the USA-funded NGOs is most committed to institutional strengthening and support of the health systems. This brings considerable challenges in terms of results-led monitoring as the pace of the health directorates to a large extent dictates the pace of the implementation. This requires considerable skill to achieve results and not short circuit institutional lines of communication. HAI has achieved a remarkable degree of success with the set targets and managed to work within the constraints of a resource-poor National Health Service. This achievement should not be underestimated and should be explored in an issues paper. HAI should build on the basis of their experience to further integrate activities with the provincial health authorities. HAI has used a considerable amount of resources on health worker training using and in-service training model. It is time to consider a more innovative approach to the capacity building needs of the NHS. (See Recommendations for specific suggestions.) 8.3 Recommendations. 8.3.1 Programmatic Recommendations All programmatic recommendations concern HAI and the MOH unless otherwise stipulated Health Alliance International Child Survival Programme Final Evaluation 2002 36 Maternal and Newborn Care · Introduce the rapid `strip` test in peripheral health facilities (CS Extension) · Put emphasis on the early treatment of partners for syphilis · Promote the use of condoms during pregnancy · Carry out an investigation to understand the cultural acceptability of increased involvement of men in reproductive health issues (start with Cantos Joviais)and the cultural and practical implications of male presence in antenatal clinics and during labour, in order to develop inclusive reproductive health strategies · Investigate the supply problems linked to syringes and needles and develop strategy to overcome the problem · Carry out a study on risk factors for stillborn and neonatal deaths in the provinces of Manica and Sofala, including syphilis screening during labour, screening of mothers of stillborn babies, investigation of infections, nutrition status, and other risk factors. MOH · Define policy on TBA and clarify policy with all health workers including staff working in peripheral health facilities. STI/HIV/AIDS HAI + MOH Cantos Joviais · Support the transformation of the Cantos into Adolescent Friendly Clinics · Maintain the Cantos within the health facilities but in locations that do not disturb the normal running of the health facility · Provide identification tags for the youth workers from the Cantos · Identify sources of IEC materials and provide the youth workers with the information in order that they can become autonomous in requesting material. · Develop a fundraising strategy with the members of the Cantos that could include: ß Sponsorship (from the private sector, local NGOs, international NGOs, in exchange for community theatre presentations, educational talks) ß Income-generating activities: these activities should be confined to activities directly linked to the aims of the club and not involve high investment costs (either in terms of capital or time). Possibilities: making HIV/AIDS emblems, making tee-shirts, putting on music and theatre shows ß Affiliation with established youth groups (Rotary –Interact, Scouts, Church youth groups) ß Staging once a year sponsored events – runs, cycle races, clean-up campaigns. ß Develop a membership sustainability plan that could include some of the following ideas : o A newssheet (hand-written or computer generated) that documents events, education programmes and the successes of the group. o Regular (twice yearly) recruitment drives where they not only put on community plays, music events, but also actively recruit new members for the club. o Institute a system where each member is responsible for training two new members · Suggest a policy of `bring your sister´ to the Cantos in order to address the gender imbalance Health Alliance International Child Survival Programme Final Evaluation 2002 37 Mother to Child Transmission · Consider the pros and cons of moving the Mother to Child Transmission programme from the central and main provincial hospitals to city maternity facilities to avoid over-crowding at the reference hospitals · Circulate more information to health personnel about the Mother to Child Transmission Programme, in particular to the nurses who are directly or indirectly involved in the programme · Clarify the coverage of the programme (MOH) · Introduce in Beira group therapy sessions (similar to Manica experience) for mothers enrolled in the Mother to Child Transmission Programme VCT · Expand the VCT using the NHS as base for the centres · Develop an in-service training strategy for counsellors, including retreats for rest and recuperation (GTZ) · Contribute to and use more fully the national Web site. Distributing information from the Web site to the VCT (HAI) · Institute a mechanism for providing the VCT staff and other health personnel with updated information from the Web site (HAI) Anti-AIDS School Clubs MOE and DPE · The Ministry of Education should explore financing possibilities for the Anti-AIDS Clubs including top-ups for teachers that are active in the clubs. · HAI should act as a facilitator for the clubs, linking the teachers into the networks of organisations working with HIV/AIDS. Malaria · Expand the bednet programme to include all traders in each location, including more isolated areas. · Develop a strategy for the reduction of the price of the bednets for pregnant women and children, taking into consideration longer-term sustainability issues. · Immediately carry out a study of possible supply strategies, in consultation with NGOs with experience in this field (commercialisation). · Increase the involvement of the DDS-DPS in the implementation of the bednet programme. 8.3.2 Partnership Recommendations HAI and MOH · HAI should maintain and improve the co-ordination mechanisms instituted with the DPS. · HAI should initiate, in FY03, a joint planning process with the DPS including sharing of information on the financial resources available for each joint activity. Health Alliance International Child Survival Programme Final Evaluation 2002 38 · DPS should institute a participatory planning process with HAI for activities to be undertaken in FY03-04 within the strategic planning process. · DPS should review the job of descriptions of HAI personnel (to be provided) and share with relevant heads of section within the DPS. HAI-CLCs · Evaluate the capacity of the CLCs in the light of the demands from programme components and critically examine which of the roles are most suited to the CLCs. · Discuss the official recognition of the CLCs, taking into consideration the Governmental Decree 15/2000 that regulates the recognition of traditional and community based structures. 8.3.3 Managerial Recommendation HAI · Organise a retreat for all staff to discuss the HAI programme strategy for the next five years. · Prepare a new organisational chart that reflects the new programme. · Develop job descriptions with staff to reflect new roles and responsibilities. · Share the new organisational chart and job descriptions with all partners, especially with the DPS. · Re-examine the role of the Country Coordinator; consider more a strategic role and less administrative role. · Seriously consider HAI representation at the national level in order to capitalise on advocacy positions and maintain current on policy debates. · Rationalise internal reporting mechanisms and institute mechanisms for reflection and analysis: Reduce number of activity reports (use a chart for indicating activities completed against plan); use quarterly reports to reflect on progress towards objectives (and not progress towards activities completed); use six monthly reports for team reflection and analysis; use annual reports to re-examine goals, objectives and plans. Develop process indicators to measure how the approach used by HAI is effective in achieving goals. · As soon as possible carry out an organisation-wide training on HIV/AIDS, including personnel policies. 2 · Write the story of HAI achievements in words, photographs or video. In particular an issues paper should be written about HAI approach to institutional strengthening. USAID · USAID should actively support initiatives by PVOs to introduce reflective learning into the monitoring cycle. · USAID should examine the impact of the emphasis on results-based monitoring on the analytical quality of programming of PVOs 8.3.4 Training strategy Recommendations 2 World Vision has a staff training package that they are willing to share with other NGOs. Health Alliance International Child Survival Programme Final Evaluation 2002 39 · Consider consolidation of training materials in all components (Maternal and Newborn Care, STI/HIV/AIDS, Malaria) to enhance the curriculum of initial training courses in the Institute for Health Sciences in Beira and the Training Centre in Chimoio. · Investigate with the directors of the training institutions the possibility of training the teaching monitors in the teaching of the new material · Provide the libraries of the training institutes with copies of all training materials used over the past four years. Make a list of the materials available to the student health workers. · Facilitate the final fieldwork of at least 10 student health workers per year. · Continue to plan all in-service training with the in-service training department of the DPS. Health Alliance International Child Survival Programme Final Evaluation 2002 40 B: Results Highlights Syphilis screening and treatment make huge strides in Manica and Sofala but stillbirth rates continue to be high. The syphilis screening and treatment service offered to pregnant women and their partners have been vastly improved in the provinces of Manica and Sofala. Thousands of women and their partners are treated under the National Health Service syphilis programme every year. HAI will continue to carry out operational research to try and pinpoint the reasons for the continuing unacceptable high rates of stillbirths. Lack of access to maternities continues to plague the MCH component of the Child Survival Programme Long distances still have to be covered by women in Sofala and Manica Provinces in order to access health care. This is one of the factors that lead to high maternal mortality. HAI is committed to studying all risk factors in order to work together with the Ministry of Health to improve maternal health strategies. So many adolescent boys talking about safe sex – the Cantos must be getting something right Instead of boys just talking about sex, boys in Sofala and Manica, are talking about safe sex. This breakthrough is largely due to the excellent work carried out under HAI´s Child Survival and Maternal Care Project. Young men and women have been provided with information on the prevention of STI/HIV/AIDS and they are spreading the word through theatre, lectures and popular games sessions in the Adolescent Friendly Health Centres. Sex workers identify with health: `We are health workers, we educate in the fight against HIV/AIDS` (Sex workers – Beira) Sex workers are the newest recruits in the fight against HIV/AIDS. The sex workers carry out condom promotion campaigns in bars and discotheques. The sex workers consider themselves to be the nocturnal arm of the health service and are committed to their programmes. HAI has supported these women’s groups—training, teaching, counselling and encouraging them to make healthy choices. HAI has worked with this culturally sensitive group and encouraged the National Health Service to accept these very active activists! Traders and Community Leaders Councils welcome treated bednets. HAI needs to learn from other rural development projects about the pit-falls of sustainable commercial initiatives. HAI´s treated bednet programme has the makings of an important breakthrough in malaria control. Malaria continues to be one of the most dangerous and debilitating illnesses that affects the populations in Central Mozambique. HAI is working with local traders and with Community Leaders Councils to sell the subsidised bednets. The HAI team has established that there is demand for the bednets. They now have to guarantee the supply of the bednets in a challenging commercial environment. HAI will learn from other NGOs who work in the area of social marketing to ensure the success of the initiative. Health Alliance International Child Survival Programme Final Evaluation 2002 41 References DPS/HAI/ADPP DTS/HIV/SIDA : Apresentações : Conhecimentos, Atitudes, Percepções e Prácticas Relacionadas as DTS e HIV/SIDA entre as Mulheres em Risco na Cidade de Beira. Daria J., Ramirez LE., Coutinho J., Amós J., Mussa S., . HAI Central Mozambique Child Survival and Maternal Care Project. Application for BHR/PVC Child Survival Grants Program. December 3, 1997. HAI A study of knowledge, attitudes, behaviour and practices regarding maternal care, malaria and AIDS of mothers with children under two years in Central Mozambique. HAI . February 1999. HAI Detailed Implementation Plan for Health Alliance International Central Mozambique Child Survival and Maternal Care Project. March 30, 1999. HAI Mid-term Evaluation. Health Alliance Internatioonal. Child Survival and Maternal Care Project. Armand L. Utshudi, Michael Brian, Naguib Hossumane Gulamo, Anna Cattane. September 2000 HAI Central Mozambiqe Child Survival and Maternal Care Program. Application for BHR/PVC Child Survival Grants program Cost Extension . December 20, 2001 OMES Sex Workers in Manica Respond to AIDS. Mid-term review. December 2001 UNICEF External Review of UNICEF support to HIV/AIDS Prevention for Youth November 2001. USAID Guidance on the Definition and use of the child survival and health programs fund. FY 2002 Update. May 1, 2002 Health Alliance International Child Survival Programme Final Evaluation 2002 42 Attachment A: Poverty Mapping Key to Table below ß The incidence rate of poverty shows the percentage of the population that is found below the absolute poverty line. ß The depth of poverty measures the average distance of the poor from the poverty line. This measurement is useful for the monitoring of the impact of policy on the poor, as it can measure movement towards the poverty line and not merely the numbers above and below the line. ß The distribution of the severity of poverty depends on both the incidence of poverty and the number of the poor in the area. ß The Inequality Index (General Entropy) shows the level of dispersion of wealth of the households in a given area. The higher the number the higher the levels of inequality in the area. Province District Incidence of poverty Depth of poverty Severity of poverty General Entropy (GE(1)) Number of poor Manica Chimoio city 0,67 0,31 0,18 1,34 112715 Manica Barue 0,53 0,20 0,10 0,64 41568 Manica Gondola 0,72 0,31 0,17 0,77 130274 Manica Guro 0,56 0,23 0,13 0,37 25236 Manica Machaze 0,43 0,18 0,10 1,29 32374 Manica Macossa 0,71 0,34 0,21 0,68 9753 Manica Manica 0,68 0,30 0,17 0,77 104541 Manica Mossurize 0,48 0,20 0,11 2,06 57805 Manica Sussundenga 0,69 0,32 0,19 0,62 63300 Manica Tambara 0,66 0,31 0,19 0,50 20257 Province District Incidence of poverty Depth of poverty Severity of poverty General Entropy (GE(1)) Númber of poor Sofala Cidade da Beira 0,53 0,22 0,12 0,58 199557 Sofala Buzi 0,89 0,52 0,35 1,15 121499 Sofala Caia 0,91 0,52 0,34 0,48 50173 Sofala Chemba 0,98 0,70 0,53 0,30 45076 Sofala Cheringoma 0,93 0,58 0,40 0,34 18621 Sofala Chibabava 0,96 0,64 0,47 0,39 66919 Sofala Dondo 0,72 0,34 0,20 0,42 113557 Sofala Gorongosa 0,97 0,70 0,53 0,31 72967 Sofala Machanga 0,94 0,63 0,46 1,85 40076 Sofala Maringue 0,98 0,71 0,55 0,28 51844 Sofala Marromeu 0,84 0,45 0,28 0,35 56673 Sofala Muanza 0,89 0,50 0,33 0,29 13212 0,17 Sofala Nhamatanda 0,93 0,58 0,40 0,38 122396 1,74 Source : Ministry of Planning and Finance. Maputo. 2002. Mapeamento de Vulnerab Health Alliance International Child Survival Programme Final Evaluation 2002 43 Attachment B: List of Studies Undertaken by HAI HAI list of studies and presentations related to Child Survival Programme 1. “Community Leadership in Central Mozambique ” Ethnographic study carried out to better understand community leadership and how to best incorporate community leaders in the health sector, presented at the Mozambique National Medical Conference, 2001. 2. “Treatment seeking behavior for children under 5 with febrile illness” Ethnographic study on treatment seeking behavior for children under 5 with a febrile illness, presented at the Mozambique National Medical Conference, 2001. 3. “Feasibility study of Rapid Syphilis Tests in Antental Care” Final report on pilot implementation of rapid test, presented at the Mozambique National Medical Conference, 2001. 4. “Feasibility Study of Insecticide Treated Mosquito Nets in Central Mozambique” Final report from the first year’s activities related to ITN sales, presented at the National Medical Conference, 2001. 5. “Preference Among Pregnant Women Seeking Antenatal Care Services” Rapid survey carried out among women attending antenatal care services, presented at the National Medical Conference, 2001. 6. “Knowledge, Attitudes and Practices of HIV/AIDS Among Women 15-49 Years in Central Mozambique” Results of project baseline survey, presented at the World AIDS Conference in Durban, 2000. 7. “A randomized trial of Chloroquine and Fansidar Efficacy in Children Under Five with Uncomplicated Malaria in Manica-Mozambique” Resistance study report, presented as thesis project for MPH program at the University of Washington. 8. “Prenatal Syphilis Screening in Mozambique” Presentation made on project activities at CORE Maternal Care workshop in Nairobi, 2000, and BEHAVE Johannesburg workshop, 2002. 9. “Control of Malaria in Pregnancy in Central Mozambique” Presentation made on project activities at the CORE Fresh Air Malaria Workshop, Nairobi, 2001. 10. “A Community Survey on the Impact of CLCs in Central Mozambique” Report on community survey presented as thesis project for MPH program at the University of Washington. 11. “Community Stigma and HIV/AIDS: a study of knowledge, attitudes and practices among women aged 15-49 in Central Mozambique” Survey results, oral presentation at the World AIDS Conference in Barcelona, 2002. Health Alliance International Child Survival Programme Final Evaluation 2002 44 12. “Building health system capacity to provide Voluntary Counseling and Testing (VCT) services in Central Mozambique” Results of programme experience to date, oral presentation at the World AIDS Conference in Barcelona, 2002. 13. “Self perception of risk: a study of knowledge, attitudes and practices among women between ages 15-49 in Central Mozambique” Survey results, oral presentation at the World AIDS Conference in Barcelona, 2002. 14. “Acceptability of Prevention of Mother to Child Transmission of HIV services in Chimoio, Mozambique” KAP survey carried out in preparation of launching pMTCT services. Health Alliance International Child Survival Programme Final Evaluation 2002 45 Attachment C: List of Interviews Provincial Health Director Manica Dr. Dayse Provincial Medical Officer Sofala Dr. Amosse Head of C.H Department Manica Victorino Nhamazou Head of MCH Manica Nurse Esperança Head of MCH Sofala Head of STI Manica Mr. Cavado Head of STI Sofala Nunes Sampaio Head of Adolescent Health Manica District Directors of Health Manica Gondola Barue Dondo Nhamatanda Chimoio Laboratory personnel Manica Mr. Desidereo Sofala Pharmacy Personnel Manica VTC staff Manica Sofala MCH nurses Manica Sofala School teachers Sofala Kubitsirana Manica Faustino Manel João Pedro Atibo Italian Aid Sofala Dr. Luigi Andreoni ADPP Sofala Health Alliance International Child Survival Programme Final Evaluation 2002 46 HAI Staff: Manica Sofala Group Interviews CLCs Manica Gondola Dondo Nhamatanda Traders Nhamatanda Gondola School children Beira Sex workers Beira Health Alliance International Child Survival Programme Final Evaluation 2002 47 Attachment D: Terms of Reference CHILD SURVIVAL PROJECT PARTICIPATORY FINAL EVALUATION I. Introduction The 4-year Child Survival project in Central Mozambique was initiated in October 1998 in both provinces Manica and Sofala. The goal of the Child Survival project is to bring about sustainable reductions in infant, perinatal and maternal mortality and morbidity in Manica and Sofala. The interventions are as follows: maternal and newborn care (40% of project effort); STD/HIV/AIDS prevention (35%); and malaria control (25%). A detailed implementation plan (DIP) was elaborated in late March of 1999, approved by USAID, and is being implemented. A participatory Child Survival midterm evaluation was carried out in August and September of 2000, and measured the accomplishments of the program implementation. The results of this evaluation, which were interpreted by key program stakeholders, resulted in an action plan which has strengthened the efforts of the second half of the Child Survival project. In an effort to measure the accomplishments of the Child Survival project and document best-practices, a final project evaluation is planned for August, 2002. This evaluation, which is strongly recommended by USAID, will follow an abbreviated version of the participatory evaluation experience described by Judi Aubel. It is expected that this collaborative methodology will ensure the relevance of the evaluation experience for project stakeholders, ultimately increasing the utilization of these results and reinforcing the next phase of the Child Survival project. II. Objectives of the evaluation 1- To assess if the program met the goals and objectives described in the DIP. 2- To assess the effectiveness of the technical approach in achieving the goals and objectives described in the DIP. 3- To describe and document overarching best practices and lessons learned from the project. 4- To develop a strategy for communicating these best practices and lessons learned both within the organization and to partners. III. Evaluation team The evaluation team will include 4 or 5 member, including: Team Leader (Kerry Selvester) The Team Leader is at once a facilitator, trainer, advocate and technical specialist, who is committed to involving project staff and stakeholders in a participatory evaluation process. She is an independent consultant, and is responsible for coordinating all of the methodological aspects of the evaluation; participating directly in the data collection; supervising the other team members; facilitating data analysis; organizing the feedback of key evaluation findings and recommendations Health Alliance International Child Survival Programme Final Evaluation 2002 48 to project stakeholders; preparing a first draft of the report; and presenting a final draft to HAI Headquarters. She must be committed to involving program staff in the evaluation process. NGO Representatives (2) (TBA) The NGO Representatives will be a staff member of one of the NGOs currently collaborating with the CS project. Representative of the MOH/Maputo (TBA) The representative from the MOH will be a full member of the team. DPS Representative (TBA) One member of the provincial health office will be on the team from each province (taking part in team activities only for that province). Other evaluation staff and informants include: HAI Headquarters Representative (Dr. Stephen Gloyd) As a key informant, he will be present during some of the data collection process and throughout the analysis of data. CS Program Manager He is responsible for organizing and coordinating the evaluation process; facilitating the communication between the main evaluators and Headquarters; defining evaluation questions and facilitating data and needs required by the evaluation team. Program field staff: MCH, STD, Health Education and Malaria/HIS Advisors (5) They are responsible to facilitate daily data, and to participate in the data collection and analysis Provincial/ District health directorates: Planning officer, MCH, STD provincial and district responsible and Community health department (5) They are responsible for participating in data collection and analysis, and in the discussion of the evaluation results. Partners organizations: ADPP- Hope, PAC, FINNIDA, Provincial AIDS Control Program Office, OMES, GTZ (6) These groups, that are co-implementers of the project activities, will provide their insights to the evaluation team. It is expected that 2 representatives from NGO collaborators will be full members of the team. CS Administrator/logistic (2) Their main task is to assure that the necessary resources are available on time and place, participating in the development of the evaluation timetable with the CS program manager and procuring all needs to ensure the success of the evaluation process. IV. Logistics and material arrangements The CS project manager will work with the CS administrator and logistician to organize all logistical arrangements according to the timetable. CS administrator and logistician will organize and purchase Health Alliance International Child Survival Programme Final Evaluation 2002 49 office materials, photocopies, trips, transport, perdiems, etc. Per diems will be paid to all team members as required, and the team leader is provided a consultant fee for leading the evaluation team. Administrative procedures to organize visits to some districts will be coordinated with the HAI program staff and counterparts. V. The evaluation process Methodology The suggested methods to be discussed during the orientation workshop include: 1. interviews with government partners, NGO partners, community members 2. observations of the development of activities in the health units and communities 3. review of data, register books, progress report, health district records (with program staff) Phase I – The introductory workshop (2 days) A 2-day workshop will be hold at the beginning of the evaluation process to introduce the participatory evaluation techniques and process to the entire evaluation team. A visual framework of the project will also be detailed during this workshop, and all participants will be provided with the CS framework, detailed implementation plan (DIP) and initial results from the CS KPC community survey completed in August, 2002. These survey results will be discussed in relation to the baseline KPC results, and will be used to guide the methodology and instruments for data collection (including questionnaires, observation guidelines, team assignments, etc). All of these materials will be field tested during phase I. In addition, the timeline and site visits will be decided upon and the process for analyzing data, summarizing findings, drawing conclusion/recommendations and reviewing with the stakeholders will be explained. At the conclusion of this workshop a list of all required materials will be delivered to appropriate staff members. Phase II - Fieldwork (6 days) 2.1 Fieldwork teams: data collection techniques and logistics Team members will carry out the data collection. The CS Administrator and Logisticians will facilitate the logistical arrangements. Observations of project activities will take place, and in-depth interviews, etc, will be performed by the team members. 2.2 Analyze information collected The analysis of data will occur within each subgroup under the Team Leader’s supervision at the conclusion of each day of data collection. In addition, at the end of each province visit the team will meet for ½ day to summarize their findings. Analysis will also include the results of the CS KPC survey. Phase III (2 days) 3.1 Summarize fieldwork findings Each field work group will use the evaluation questions to guide their summary of findings. Consequently, each workgroup facilitator will work with the Team Leader and assistant of each field work group to integrate them into one set of evaluation findings. 3.2 Formulate lessons learned Every work group will present what they learned from the evaluation findings and suggest conclusions or lessons from the observation. The entire team will be responsible to study the evaluation and formulate recommendations. Health Alliance International Child Survival Programme Final Evaluation 2002 50 3.3 Share preliminary findings with stakeholders After determining a preliminary set of findings and recommendations, the team will present them in an informal meeting to interested stakeholders for discussion and revision. 3.4 Summarize evaluation findings, best practices and lessons learned The team will then meet to review any revisions needed to improve the conclusions and recommendations of the group. They will also devise a strategy for sharing these best practices and lessons learned with project stakeholders. Phase IV (XX days) 4.1 Preparation of the final evaluation report The team leader will prepare a full draft of the report based on input from the evaluation process and present the draft to headquarters staff and other team members for revision. The full and final report must be presented to HAI headquarters by September XX, 2002. 4.2 Discuss evaluation results with program stakeholders The main results from the evaluation will be presented and discussed with the participation of available members of the evaluation team and key stakeholders (including provincial health directorate staff, some district health directorate staff, and collaborating NGOs). This presentation will take place as soon as possible after the evaluation is completed. 4.3 Implement lessons learned into next phase of project The evaluation findings will be discussed with project staff and stakeholders in a stakeholder workshop to be held at the beginning of the next phase of the CS project, to be held in early November. These results and feedback from stakeholders will be instrumental in developing the implementation strategy for the DIP of this next phase.