Midterm Evaluation
Abaqulusi Child Survival Program
Salvation Army World Service Office
KwaZulu Natal Province, South Africa
COOPERATIVE AGREEMENT # HFP 0 A – 00 – 02 – 0040 – 00
START DATE – OCTOBER 1, 2002
END DATE – SEPTEMBER 30, 2007
Submitted by:
Richard Crespo, Ph.D.
Lead Evaluator
June 27 – July 15, 2005
2
Table of Contents
A. Summary.................................................................................................................................... 3
B. Assessment of the progress ....................................................................................................... 4
1. Technical approach................................................................................................................ 6
a. Exclusive Breastfeeding...................................................................................................... 7
b. Diarrhea Control ................................................................................................................. 9
c. Washing Hands ................................................................................................................. 10
d. Acute Respiratory Infections ............................................................................................ 11
e. Immunizations................................................................................................................... 12
f. HIV/AIDS.......................................................................................................................... 13
g. Revised Measurable Objectives........................................................................................ 17
2. Cross-cutting approaches..................................................................................................... 19
a. Community Mobilization.................................................................................................. 19
b. Communication for Behavior Change .............................................................................. 19
c. Capacity Building Approach............................................................................................. 20
e. Sustainability Strategy ...................................................................................................... 22
C. Program Management ............................................................................................................. 23
1. Staffing.................................................................................................................................. 23
2. Health Information System.................................................................................................. 23
D. Additional Issues as identified by the evaluation team........................................................... 24
E. Conclusions and recommendations ......................................................................................... 24
F. Results Highlights.................................................................................................................... 26
3
A. Summary
The Abaqulusi Child Survival Program (ACSP) is located in northern Kwa-Zulu Natal in the 
Republic of South Africa. The program’s goals are to reduce the morbidity and mortality of 
27,242 children under five years of age and 37,816 women aged 15 – 49 years and to prevent the 
spread of HIV and other sexually transmitted diseases.
This project has five interventions: control of diarrheal disease (20% effort), control of acute 
respiratory infections (10% effort), childhood immunizations (20% effort), exclusive 
breastfeeding (10% effort) and HIV/AIDS (40% effort). 
Solid progress is being made in immunizations, control of diarrheal disease and HIV/AIDS 
prevention among young people. 
The data on ARI are mixed. The midterm LQAS survey indicates negligible change but the 
midterm field survey indicated a high level of behavior change among mothers who are 
monitored by CHWs. The ACSP team will monitor this indicator more closely in its 
surveillance system.
Behavior change in exclusive breastfeeding is potentially complicated if a high proportion of 
mothers work or are in school. This report recommends investigating what percentage of 
mothers are away from their children during the day and distinguishing between mothers who 
can have their infants with them all day and those who do not. Additionally the ACSP team is in 
the process of refining its behavior change strategy regarding benefits and barriers to exclusive 
breastfeeding.
Following is a summary of the ACSP programmatic accomplishments.
1. Training CHWs: They had received very little supervision prior to this project. Through 
training workshops and one-on-one training their skills have improved in following up sick 
children, health communication and growth monitoring.
2. Immunizations: Training the CHWs to monitor immunizations and follow up drop-outs has 
contributed to increased coverage.
3. Work in the schools on HIV: The mentors who facilitate learning sessions weekly in the 
schools have been a great asset. They are volunteers, but have been faithful in their teaching. 
The interactive style has generated an openness to talk about sex and HIV in a healthy way. 
A number of youth have sought counseling from the mentors because they have been abused.
4. The mentors have a strong sense of being part of a team and are enthusiastic about their 
work.
5. The program staff members are known in the community. They feel welcomed in the 
communities and schools and are respected for their contributions.
6. Training pastors in HIV/AIDS: Pastors have welcomed the training; attitudes and actions 
have changed. 
7. Health monitoring system: The project staff has created a system for family registries and 
data reporting. There was no functional system prior to this project.
4
B. Assessment of the progress 
The program is located in the Abaqulusi Municipality of District Council 26 in northern Kwa￾Zulu Natal in the Republic of South Africa. Its goals are to reduce the morbidity and mortality 
of 27,242 children under five years of age and 37,816 women aged 15 – 49 years; to prevent the 
spread of HIV and other STIs; and to provide assistance to orphans and vulnerable children, 
especially those who have been affected by the HIV epidemic.
Background
The program is divided into five supervision areas: Vryheid, Mondlo, Mvunyane, Lowsburg, and 
Hlobane. The five areas are served by 104 Community Health Workers (CHWs), who receive a 
salary from the Department of Health. Three Community Health Facilitators from the project 
staff supervise the CHWs. The Department of Health has the responsibility of assigning two 
additional Community Health Facilitators, but they do not have transport and do no supervision 
in the field.
The ACSP’s principal local partners consist of The Salvation Army/Southern African Territory, 
the South African Departments of Health, Education and Social Welfare and Lethempilo Youth 
Organization. ACSP also coordinates closely with traditional authorities and church groups, 
especially those involved in home-based care. 
Culturally, the area is predominately Zulu. English is spoken only by professionals and people 
living in the major town, Vryheid. Thirty-five percent of the population is peri-urban 
(townships) and 65 percent is rural. Paved roads connect most of the townships with Vryheid. 
A major gap exists between rural and peri-urban areas in terms of water and sanitation services, 
with a much larger proportion of families in rural areas having no water other than surface water, 
and having to walk long distances to obtain that water. Data from the DIP indicate that only 
30.6% of families had access to piped water (including those with communal pipes), and for 
29.1%, surface water (such as lakes, streams and rivers) was their only water source. Only 
24.4% of families had safe sanitation facilities, defined as flush toilets or VIP latrines. The 
infant and child mortality rates for the province are higher than those from the country as a 
whole. The number of tuberculosis cases in the program area is rising sharply, and rising HIV 
prevalence rates are creating a crisis situation as increasing numbers of mothers of young 
children begin to manifest signs of active AIDS.
Summary of Progress
In April 2005 the project staff conducted a mid-term KPC survey. Table 1 on the following page 
presents the findings from that survey.
5
Table 1: Comparison of baseline and mid-term indicators.
Indicator Baseline Mid-term Percent 
Pt. Diff.
Target
Percentage of children age 12-23 months who received a 
measles vaccine
63 79 16 N/A
Percentage of caregivers reporting increased liquids
during child’s illness in the past two weeks
9 28 20 N/A
Percentage of caregivers of children age 0-23 months who 
cite at least two known ways of reducing the risk of HIV
infection
31 46 15 60%
Percentage of children age 12-23 months who are fully
vaccinated (against the five vaccine-preventable diseases) 
before the first birthday
63 75 12 80%
Percentage of caregivers who would allow their child to 
play with a child who was HIV positive
55 67 13 80%
Percentage of caregivers who mention fast/difficult 
breathing as a sign that indicates the need for treatment
7 11 3 N/A
Percentage of mothers of children age 0-23 months who 
wash their hands with soap/ash before food preparation, 
before feeding children, after defecation, and after 
attending to a child who has defecated
1 1 0 50%
Percentage of caregivers who know at least two signs of 
childhood illness that indicate the need for treatment
37 35 -2 70%
Percentage of children age 0-5 months who received only 
breast milk in the last 24 hours
34 31 -3 50%
Percentage of ninth graders who name at least two of the 
three main programmatic ways to prevent HIV 
(abstinence, faithfulness, condoms)
N/A 49
Percentage of ninth grade girls who name at least two of 
the three main programmatic ways to prevent HIV 
(abstinence, faithfulness, condoms)
N/A 45
Percentage of ninth grade boys who name at least two of 
the three main programmatic ways to prevent HIV 
(abstinence, faithfulness, condoms)
N/A 54
Percentage of ninth graders who are virgins 68 72 4 80%
Percentage of ninth grade girls who are virgins 83 80 -3 90%
Percentage of ninth grade boys who are virgins 48 63 15 70%
The data in this table reflect progress in vaccinations, increased liquids during illness, knowledge 
of HIV prevention among caregivers and attitudes about children with HIV. Another 
encouraging result is the increase in the number of ninth grade boys who state that they are 
virgins.
Indicators that do not reflect progress are recognition of fast/difficult breathing, washing hands, 
recognition of signs of illness and exclusive breastfeeding.
6
Regarding exclusive breastfeeding, a potential barrier is that some mothers either work or are in 
school. Refer to Section B.1.a for a discussion of this barrier and recommendations for more 
detailed surveillance on behavior change.
Regarding the indicator on knowing two signs of childhood illness, refer to Section B.1.b., d., 
and g. for a discussion and the recommendation that this indicator differentiate signs of 
dehydration and ARI. Also refer to mid-term survey data on the number of mothers who 
identified wheezing type of cough and stated that they took their child to the clinic for treatment
in Section B.1.d.
The project staff team has laid a solid foundation for improvements in community health. While 
this report identifies a number of challenges, these should not minimize the value of the progress 
that has been made thus far. 
1. Technical approach
The program strategy in the DIP was to work through the Department of Health and local NGOs 
to strengthen the work of the CHWs. The reality at the mid-term is that project staff has mostly 
worked directly with the CHWs. While this approach has contributed to improved skills and the 
quality of their work, it does not bode well for sustainability. 
A number of factors have contributed to this situation. One, the program, has not had stable 
leadership. The first Program Manager was hired from the Department of Health with the 
expectation that she would be instrumental in developing the relationship with the department. 
Unfortunately this did not happen due to a variety of complex factors.
Additionally, there has been high staff turnover. Only one staff member is left who was hired at 
the beginning of the project. This problemis not only a matter of leadership, however. 
Recruitment is difficult because this part of Kwa-Zulu Natal is rural and isolated. People who 
have been recruited with professional training have usually turned down positions for this reason.
A major step forward is the hiring of a new Program Manager, Faye Hannah, who has extensive 
experience in child survival programs.
A second factor is that the Department of Health has not followed through with many of their 
commitments. The two Community Health Facilitators (supervisors) assigned to the project 
were not provided vehicles for supervising CHWs in the field. They are largely confined to desk 
work. Consequently the Department of Health is not involved at the community level in the 
ACSP. Meetings between the ACSP staff and district health officials have been infrequent and 
have not resulted in true partnership for joint planning. Much of the leadership at the district 
level has changed, so the current leadership in the department does not have a good 
understanding of the ACSP project and commitments made during the proposal and DIP stage 
have not been kept.
A third factor is that the local NGOs do not have as much capacity as originally expected to 
support child survival and HIV/AIDS interventions. An example is the NGO Lethempilo Youth 
Organization. It trains and supports home-based care volunteers for people living with AIDS. 
Its leaders are dedicated and hard working, but they function in only one township with a limited 
7
number of volunteers. Project staff has to provide transportation and much of the supervision of 
the volunteers.
A fourth factor is that the Department of Health clinics are not as involved with CHWs as 
expected. Data that are presented later in this report show that referrals between CHWs and the 
clinic are minimal, interaction with the CHWs is sporadic and there is no reporting system 
between CHWs and clinics.
Recommendation: The new Program Manager work with the appropriate Department of Health 
authorities to develop a memorandum of understanding and establish a procedure for mutual 
accountability.
Recommendation: The ACSP staff should engage the clinic staff in creating a procedure for 
meeting regularly with the CHWs for reporting and collaborative planning. It is critical that the 
clinic staff have ownership of this process.
Recommendation: The Program Manager should review the commitments of its NGO partners, 
especially Lethempilo Youth Organization and the Divisional Salvation Army entities. There 
should be mutual agreement about their roles and identification of how ACSP can increase their 
capacity to support child survival and HIV/AIDS interventions in the community.
a. Exclusive Breastfeeding
The LQAS data indicate that there is little change from the baseline in the percentage of mothers 
with infants under six months who breastfeed exclusively. The baseline finding is 34% and the 
mid-term is 31%. The evaluation team followed up this finding in their interviews with CHWs. 
The team interviewed 42 of the 104 CHWs. They were asked to list the things that were easy for 
mothers to do, and then the things that were hard for them to do. The behavior that was most 
frequently mentioned in both items was exclusive breastfeeding. The charts on the following 
page present the most frequent responses to both questions.
8
Charts 1&2: Behaviors that CHWs identified most frequently as easy and hard for mothers to 
do.*
* Note: CHWs were allowed to give multiple responses and to give the same answer to each question.
As seen in the above charts, the CHWs identified exclusive breast feeding as both easy and hard. 
Upon analysis of these findings the project’s field staff stated that the mothers’ responses might 
have been affected by the following factors. One, they regard it as easy because it does not 
require additional equipment (e.g. bottles) and preparation. On the other hand they regard it as 
hard because many mothers work or go to school and cannot breastfeed during this time. School 
is also a factor because of the large number of teenage mothers. At this point the project staff 
does not have data on the percent of mothers who either work or go to school, so it is not 
possible to know how many mothers are affected. 
It will be helpful to know what the percentage is of mothers who exclusively breastfeed and have 
their infants with them all day. This information will give the team a better understanding of the 
EBF adoption rate. Subsequently they can modify their behavior change strategy and messages 
if needed. It is possible that the adoption rate is higher among the mothers who have their 
infants with them all day.
It should be noted that the project team has conducted a Doer/Non-Doer analysis of exclusive 
breastfeeding. The research has been done and the analysis will be completed soon after the 
mid-term evaluation. The findings from this analysis will also contribute to modifying the 
behavior change strategy.
Recommendation: The project team should monitor EBF adoption rates using the LQAS
methodology. When monitoring EBF the project team should add a question about whether the 
55 52
45
0
10
20
30
40
50
60
Percent
Ex-breast
feeding
Immunizations Washing hands
Easy for Mothers to do
N = 42
54
26 23 23
0
10
20
30
40
50
60
Percent
Ex-breast
feeding
HIV testing Home
treatments
Hygene
Hard for Mother to do
N = 42
9
mother is able to have her baby with her throughout the whole day. If the answer is no, then they 
should inquire if it is because of work or school.
Recommendation: The Program Manager should find out from La Leche League or Africa 
Center about their experience and recommendations for mothers who work or are in school. The 
Zulu program staff thinks that expressing milk would not be culturally acceptable. It will be 
important to know if someone has evidence for how to address this barrier.
b. Diarrhea Control
The interventions for control of diarrheal diseases are teaching mothers the signs of dehydration, 
mixing and using home-made oral rehydration solution and taking the child to the clinic if the 
child presents danger signs following IMCI protocols. 
Overall there is good progress in mothers’ knowledge about using ORS and progress in their 
knowledge about taking children to the clinic in cases where the illness continues for three days. 
A challenge is mothers knowing that they should both give ORS and take a child to the clinic 
when diarrhea continues for three days. Following is a presentation and discussion of the mid￾term findings.
Unfortunately the baseline KPC did not specifically ask about signs and behaviors in relation to 
dehydration. The questions in the baseline only asked about recognition of and behaviors related 
to childhood illness. And the mid-term KPC followed the format of the baseline. Consequently 
it is not possible to know how mothers recognize and react specifically to dehydration. 
In order to obtain some indication of mothers’ knowledge the evaluation team asked mothers 
what they would do if their child under five years old had diarrhea continuously for three days. 
Between 25 and 30 mothers were interviewed in each one of the five supervision areas. The total 
interviewed was 121. The question was open-ended thus the answers were given without 
prompting. Upon giving one answer however, mothers were asked if they did anything else. The 
results are presented in the following table.
Table 2: Mothers’ actions when child has diarrhea for three continuous days. N = 121
(Note: mothers could give more than one response)
Actions Percent 
Response
Those who stated give ORT and take to the clinic 23%
Those who stated take to the clinic but did not mention ORT 23%
All who stated they would take to the clinic 46%
All who stated they would give home-mixed ORS 69%
Those who stated give ORT but did not mention taking to the 
clinic
46%
Twenty-three percent of the mothers said they would give ORT and take the child to the clinic. 
This is low, but there is no way to know if this represents progress from the baseline. It is 
however, a good indicator of progress that over two thirds of the mothers knew about giving
10
ORT. An indication that some progress is being made is that that nearly half of the mothers did 
know that they should take their child to the clinic. 
A challenge for the program team is to make sure that the CHWs are communicating both the 
use of ORT and taking the child to the clinic for danger signs.
c. Washing Hands
This behavior is addressed separately from diarrhea control because it is considered a critical 
intervention area in this project. The behavior change message in the DIP was that mothers wash 
their hand before food preparation, before feeding children and after defecation. The finding 
from the mid-term survey is that one percent of caregivers report washing their hands after 
defecation, after attending to a child who has defecated, before feeding children, and before 
preparing meals with soap. 
The evaluation team inquired about the barriers to hand washing. Mothers and Community 
Health Workers were asked to give the reasons why “some mothers” do not wash their hands. 
The question was asked about other mothers so that the respondent would not need to feel 
defensive. The following table presents the findings.
Table 3: Reasons why mothers do not wash their hands with soap*
Reason Mothers 
(N=121)
CHW (N=42)
It is not a habit 67% 61%
They think it is not important 25% 42%
Lack of soap 9% 23%
Water not readily available 0 16%
*Note: Mothers and CHWs could give more than one answer.
It is important to note that mothers and CHWs have similar perceptions about the barriers to 
hand washing. It is a matter of habit, not the lack of resources. These findings will help the staff 
team in crafting their behavior change messages and strategies for promoting behavior change.
During the data analysis workshop the evaluation team discussed the following 
recommendations for addressing hand washing.
Recommendation: The field supervisors (Community Health Facilitators) should conduct a 
participant-observer field study of a random selection of CHWs to determine if they wash their 
hands before preparing meal. Given that the barriers are a matter of habit, an important strategy 
for behavior change will be one of modeling; CHWs should set an example for the mothers. If 
the results of this study show that CHWs do not wash their hands before preparing a meal then 
the ACSP team needs to begin facilitating change among them first.
Recommendation: The ACSP team should conduct a Doer/Non-Doer analysis with mothers in 
order to identify the factors that are associated with the mothers who do wash their hands. These 
findings will be instrumental in crafting behavior change messages.
11
Recommendation: The ACSP team and CHWs could sponsor community events (festival style, 
i.e. make it fun!) that feature role plays, music and testimonials centered on critical health 
behaviors such as hand washing. Socialization is a helpful and creative way to help change 
habits, especially in situations where there are no resources or cultural taboos. This strategy also 
breaks up the monotony of repetitive verbal and printed messages. 
d. Acute Respiratory Infections
The health behaviors for this intervention in the DIP are the following:
· Continue to feed and offer more fluids, including breast milk, to children when they are 
sick; 
· Give children appropriate home treatment for infections; 
· Recognize when sick children need treatment outside the home and seek care from 
appropriate providers; 
· Follow the health workers’ advice about treatment, follow-up and referral.
The findings from the midterm KPC survey were that 10% of mothers recognized fast or difficult 
breathing as a danger sign, but that many reported “cough” as a danger sign. The evaluation 
team wanted to explore mothers’ perceptions and definitions of cough to better understand these 
results. The data from the midterm field interviews show signs of progress. As reported below, 
mothers readily distinguish between symptoms of respiratory illness and have a high level of 
knowledge of the appropriate actions to take.
The evaluation team asked mothers a series of questions about how they handled the most recent 
episode of respiratory illness in their child. A total of 121 mothers were interviewed, ranging
from 22 to 30 from each supervision area. The sequence of questions was as follows.
1. Has your child had a cough recently?
2. If yes, what type of cough was it?
3. Besides cough, did the child show any other symptoms? 
4. What did you do for the child?
Table 4 on the next page presents the results of the mothers’ answers.
These findings generate a number of important observations. First, there is a high incidence of 
cough in children under five. The reason for this is that it is the height of winter and most homes 
are heated by poorly ventilated wood or coal stoves. The combination of cold weather and 
smoke in homes stimulates a lot of respiratory illness.
Second, 64% of mothers knew how to distinguish between types of cough. This is something 
that the CHWs have been teaching the mothers and it is encouraging to see this extent of 
knowledge at the midterm.
Third, in the cases where mothers reported wheezing they could identify related symptoms. It is 
important to note that mothers were asked open-ended questions, thus they responded based on 
their knowledge, not because they were prompted. This level of knowledge is also congruent 
with what CHWs have been teaching.
12
Fourth, the crux of the issue is that mothers take their children to the clinic when wheezing type 
of cough occurs, and indeed mothers reported that they did so. While this finding is not from a 
random survey, the fact that such a high percentage of the 121 mothers (from all five supervision 
areas) reported this behavior is a strong indicator that it is widely practiced.
Table 4: Key findings regarding mothers’ knowledge and behavior in managing cough in 
children under five years old.
Question Findings
Child had cough recently Yes 75%
No 24%
No answer 1%
Types of cough Dry 49%
Wheezing 13%
Wet/moist 12%
Symptoms present with wheezing cough Fever 42%
Not eating well 25%
Diarrhea 17%
Tight chest/
difficult breathing 17%
What mother did when child had 
wheezing cough
Doctor/clinic 92%
Traditional medicine 17%*
__________________________
*Mothers could give more than one 
answer.
e. Immunizations
ACSP team members support the Department of Health by training CHWs in childhood 
immunizations, collaborating in promoting immunization and creating a system for CHWs to 
monitor mothers who default in children’s immunization.
Overall the ACSP team has made good progress in this intervention. The immunization rate 
from the midterm KPC is 75%. The rate from CHWs surveillance system is over 90%. It is 
important to note that within the geographic area of the program there are communities that do 
not have CHWs. Consequently they have not been reached consistently with child survival 
interventions. In South Africa the CHWs are hired by the Department of Health, with a salary. 
Consequently the ACSP team is not free to train CHWs in communities that do not have any.
This accounts for the discrepancy between the KPC data and the registries kept by the CHWs. 
This project has made two important contributions to childhood immunization. One is 
establishing a family registry that includes immunization. With this system the CHWs are now 
13
able to identify defaulters on a monthly basis. When the team’s Community Health Facilitators 
supervise the CHWs they visit the defaulters’ homes.
The second contribution is providing the CHWs with growth monitoring scales and training them 
in using the children’s Road to Health cards. At the time of growth monitoring the CHWs also 
review the immunization records and have the opportunity follow up with those who are behind 
schedule.
Data from the family registries demonstrate the effectiveness of these contributions. The 
following chart presents the immunization rate from the CHWs’ registries for the first quarter of 
2005. 
The monthly fluctuations reflect typical patterns 
of behavior and the changes in the age of children 
who are eligible for immunizations. It is 
encouraging to see the increase in March after a 
dip in the rate in February.
f. HIV/AIDS
HIV/AIDS interventions are currently directed 
towards four populations: seventh grade students;
pastors; vulnerable orphans and children (OVC)
and persons living with AIDS. 
Overall the program staff has made good progress 
with students and pastors, but less progress with OVC and home-based care. Following is a 
report and discussion of these interventions.
Schools
The strategy in schools is to target seventh graders for dialogue-based education about 
HIV/AIDS and on how to prevent its spread. The rationale for seventh graders is that this is the 
age just before the youth tend to become sexually active. The preventive behavior that is 
presented in this curriculum is abstinence. The intervention strategy uses recent high school 
graduates to facilitate learning by using a curriculum called Life Skills. These facilitators are 
called mentors. They are volunteers, but are given a travel and food stipend. They work in the 
classroom one hour a week for 6 months. Additionally the ACSP team recruits and trains 
students from the classroom as peer educators. They facilitate informal discussions and provide 
social support.
At the midterm, the program has been implemented in 13 schools in Mvunyane and 20 in 
Mondlo (all of the schools with grade seven in these supervision areas). At the time of the 
midterm school was out of session thus the evaluation team was not able to interview students. 
The team was able to interview some mentors, peer educators, principals, and teachers. Because 
school was out the team was not able to interview as many as they would have liked.
96
87 91
0
20
40
60
80
100
Percent
Jan-05 Feb-05 Mar-05
Chart 3: Immunization rates of 
children 11 to 24 months from family 
registries (N=970)
14
The evaluation team interviewed six principals, seven teachers, ten mentors and seven peer 
educators. Each group was asked questions regarding their perceptions of students’ behavior and
about their own experience as facilitators. Following are the responses of each group. 
Table 5: Principals’, teachers’, mentors’ and peer educators’ perceptions of change in students’ 
behavior. N=30
Interviewees Most Important Observed Behavior Changes
Principals 1. More open communication about sex and HIV
Teachers 1. Open communication about sex and HIV
2. More self control in classroom behavior between boys and girls (i.e. 
less passing notes, fewer suggestive comments, less flirting.)
Mentors 1. Open communication among students about sex and HIV.
Peer Educators 1. Boys show more respect to the girls
2. Open communication about sex and HIV
It is significant that each group independently observed more open communication among the 
students. It appears that the Life Skills program has established a good foundation for behavior 
change. The challenge that lies ahead is to monitor changes in sexual behavior. 
Recommendation: The ACSP team should implement a pre and post survey on seventh grade
students’ attitudes and practices regarding sexual behaviors. Ultimately the success of the 
program is in the extent to which students change their behavior. The ACSP health information 
system should track changes in order to determine if the program is making a difference.
Principals, teachers and mentors were asked to rate separately boys’ and girls’ attitudes about 
abstinence on a three-point scale. Not surprisingly, both mentors and teachers believe that girls 
give a higher value to abstinence. The chart on the following page portrays this finding.
The chart also shows a contrasting pattern in how these groups rate boys’ and girls’ attitudes. 
Girls are regarded as giving a higher value to abstinence than boys. In contrast they believe that 
boys are much more likely to give abstinence low importance. 
The Zulu staff on the evaluation team stated that their culture encourages men to demonstrate 
their manhood through sexual activity. Education about abstinence runs counter to this cultural 
moray. As a result there are a number of implications about abstinence education. These 
implications should be addressed by the ACSP team. Specific recommendations are found on 
the next page.
15
Chart 4: Principals’, teachers’ and mentors’ ratings of boys’ and girls’ attitudes towards 
abstinence. N=30
0
2
4
6
8
10
12
Number
Low
importance
Moderate
importance
High
importance
Low
importance
Moderate
importance
High
importance
BOYS GIRLS
Recommendations:
1. Using the same message about abstinence for boys and girls does not address difference 
in the culture about sexual activity. There should be different messages for boys and 
girls. 
2. Abstinence messages for boys need to address the cultural morays regarding male sexual 
prowess. 
3. Promoting the concept of abstinence emphasizes the “don’t” or the negative message. 
The focus is preventing disease. Teaching about sex and relationships should also include 
concepts such as self-respect, respect for others (especially boys/men for girls/women), 
commitment and role of sex in relationships. Young people should understand the place 
of sex in a relationship as well as self control for disease prevention.
 
Pastors
Another target population in the HIV/AIDS intervention is pastors. ACSP staff sponsored 
seminars for pastors on the epidemiology of HIV and on service to people with AIDS and their 
families. A total of XX pastors have been trained. The evaluation team interviewed 12 of these 
pastors. The team inquired about three areas: the pastors’ attitudes towards involvement in 
HIV/AIDS ministry, their actual involvement as a result of the seminars and their counseling 
advice. The results of the survey are presented in Table 6 on the next page.
16
Table 6: Pastors’ perceptions of what should be the church’s role regarding HIV/AIDS
Pastors’ feelings towards HIV/AIDS being part of the church Number of 
Responses
The church should not get involved 0
The church’s main task is to pray for people with HIV/AIDS 4/12
The church should send people with HIV/AIDS to receive 
community and government support
1/12
The church should get actively involved in helping people with HIV/AIDS 11/12
While there are no pre-seminar data, the data in the above table present a positive view of the 
role of the church. Given that many churches are still uncertain about their response, this project 
area now has a core group of supportive pastors.
In the follow-up questions about their actual involvement in HIV/AIDS ministry, nine of the 
twelve stated that their churches are involved. They provide encouragement through home 
visits, physical support (primarily food) and pray for them (either in congregational prayer or 
one-on-one prayer). 
Of the three who are not currently involved, two of the pastors say that they plan to get their 
churches involved. One pastor has no plans.
The survey of the pastors also included three scenarios and asked them how they would respond. 
The following table presents the scenarios. 
Analysis of the pastors’ responses indicates that 
they have a good understanding of how 
HIV/AIDS is transmitted. They were also 
technically correct in advising how to prevent 
the spread of HIV from one partner to another. 
They did not hesitate to advise married couples 
to use condoms.
It is also significant that no pastor included 
statements of condemnation or rejection of 
people in the scenarios.
The scenario that generated the widest range of 
responses was the third one that involved 
unmarried young people. One factor is that 
youth would not approach their pastors with 
this type of question. As noted by The Salvation Army Divisional Commander (area church 
superintendent) this fact is something that needs to be corrected. He would like to see pastors (or 
Table 7: Counseling scenarios for pastors
1. A woman from your church has been faithful 
to her husband, but he works in Durban and 
has not been faithful. What counsel would 
you give to the woman?
2. A man from your church tells you that he has 
just learned he is HIV positive and asks you 
for advice. What would you say to him?
3. A youth from your church approaches you 
saying that she is in love with her boyfriend. 
They have talked and have decided they are 
ready to have sex. What would you advise 
them?
17
youth leaders) have the kind of relationship with the youth that they would feel free to seek 
pastors’ advice regarding sex and relationships.
A conclusion that the evaluation team made regarding the responses to the third scenario is that 
pastors do not have a well-thought-out message to promote premarital sex. In analyzing the 
responses of the pastors it was not that they condoned premarital sex, but that they did not know 
what to say to the reality that premarital sex is widely practiced. It became evident in the data 
analysis work shop that the ACSP team needs to address this issue in the pastors’ curriculum.
Recommendation: The ACSP team should include scenarios such as the ones used in the mid￾term evaluation in their seminars for pastors. They need to engage the pastors in articulating 
what they believe and what they believe the Biblical message should be to Zulu youth in today’s 
culture.
As with the recommendations for those who work with the Life Skills program in schools, the 
messages about sex need go beyond saying “no to sex” to making positive statements about the 
role of sex in relationships, including respect for self, respect for partners (especially men 
towards women) and commitment.
OVC
Care for orphans and vulnerable children is provided by Lethempilo Youth Organization and 
churches. Additionally a few health committees are in the process of mobilizing their 
communities to provide care. The Department of Social Welfare provides grants for OVC and 
the primary intervention is to sign up eligible children. 
Data from the CHWs’ records indicate that in January 2004, 36% of orphans and vulnerable 
children receive child care grants. On the other hand, 81% of these children had birth 
certificates, which is a key prerequisite for obtaining a grant. The bureaucracy for obtaining 
grants is slow and complicated, especially when the caregiver does not have the child’s birth 
certificate or the parent’s death certificate.
The fact that only 36% of eligible children receive grants, however, is an indication that more 
effort needs to be placed on following up with these children. The section on capacity building 
(Section B.2.c.) makes recommendations on engaging local churches in this area. It is important 
that these children receive broad support from the community, not just from the ACSP and 
CHWs.
g. Revised Measurable Objectives
The evaluation team had difficulty in tracking changes in some of the measurable indicators from 
the baseline to the mid-term. The midterm team is not able to discern if data on questions such 
as giving more or less fluids refers to incidence of diarrhea, ARI or some other disease. 
Given this situation the midterm evaluation team recommends changes to the measurable 
indicators. The following table shows the baseline indicators from the DIP and the 
recommended changes.
18
Table 8: Revised measurable indicators.
DIP Indicators Indicators Modified at Mid-term
Percentage of caregivers who know at least 
two signs of childhood illness that indicate the 
need for treatment
Add to DIP indicator: 
Percent of caregivers who know at least two signs 
of dehydrati on in children ages 0 to 59 months.
Percent of caregivers who recognize fast or 
difficult breathing as a sign indicating the need for 
treatment. 
 Percentage of caregivers reporting 
 increased liquids and increased feeding 
 during child’s illness in the past two weeks
Add to DIP indicator: 
Percent of caregivers reporting increased liquids 
when a child has episodes of diarrhea.
Percent of caregivers reporting continued feeding 
when a child has episodes of diarrhea.
Percentage of mothers of children age 0-23 
months who wash their hands with soap/ash 
before food preparation, before feeding 
children, after defecation, and after attending 
to a child who has defecated.
Modify DIP indicator: 
Percent of caregivers of children 0 to 59 months 
who state that they wash their hands with soap or 
ash in at least two of the following circumstances: 
before food preparation, before feeding children, 
after defecation, after attending to a child who has 
defecated.
The youth abstinence indicators in the DIP should stay as they are. The evaluation team does not 
recommend any changes.
The indicators for HIV/AIDS in the DIP do not make reference to two major program areas: 
pastors and the Life Skills program for seventh graders. As noted in the discussion above of 
these areas the ACSP team needs to monitor these programs with measurable indicators. 
Recommendation: Regarding pastors, the team can collect pre and post data for pastors who 
attend their training program. Possible indicators can be:
1. Percent of pastors who believe that churches should have a ministry in either HIV prevention 
or to persons living with AIDS and their families.
2. Percent of pastors whose churches currently have a ministry in either HIV prevention or to 
persons living with AIDS and their families.
Recommendation: The ACSP team should collect pre and post data on students who participate 
in the Life Skills program. The same instrument that is used with ninth graders can be used with 
these students. This will give the ACSP three measurement points to assess change. One is the 
baseline prior to the Life Skills curriculum. The second is immediately after students experience 
the curriculum and the third is two years later in ninth grade, which is the indicator in the DIP. 
This will give the ACSP team the opportunity to assess the effect of the program over time. In 
19
order to identify students from the Life Skills program, the final KPC survey for ninth graders 
should include a question asking if the interviewee participated in this curriculum. 
2. Cross-cutting approaches
a. Community Mobilization
The foundation for community mobilization in this program is the Community Health Workers. 
They are selected by the village or township councils, receive a salary from the Department of 
Health and are expected to work full time in the community. Their work in the community 
revolves around home visits. Under the supervision of the ACSP Community Health Facilitators 
(supervisors/trainers), the CHWs visit every family with children under five. A review of CHW 
registries indicates that they faithfully do so.
Early on in the program the CHWs were equipped with scales so that they could conduct growth 
monitoring. The rationale was that this tangible and visible activity would serve as a gateway to 
the community. This has occurred on an individual basis. The data from the health information 
system demonstrate that the CHWs weigh approximately 90% of the children under five every 
month.
Little community-wide organizing however, has occurred. SAWSO intends to introduce and 
community mobilization program that they developed in Zambia called Community Counseling. 
It uses a participatory approach for analyzing problems, posing solutions and creating action 
plans. A SAWSO staff member from their headquarters in Washington has trained a Territorial 
Facilitation Team in South Africa who will work with program staff to implement the strategy, 
including training volunteers from TSA corps and other community leaders to facilitate the 
process.
b. Communication for Behavior Change
The behavior change strategy is based on one-to-one interaction between the CHWs and mothers 
during home visits. The program is using one pamphlet in Zulu that covers IMCI. Flip charts 
that describe the key interventions have been used in the past but are not used much at this time. 
A little bit has been done with drama and song, but for the most part the communication 
technique used in this project is oral during home visits. At the time of the mid-term there are no 
group learning events being used in this project.
One of the factors that have limited the communication strategy is how this component has been 
managed. The project team has relied on the IMCI Coordinator to direct and implement the 
communication plan. This cannot, however, be the responsibility of one person. The new 
Program Manager must get the Community Health Facilitators (field supervisors) more actively 
involved. They must have major responsibility for training and supporting CHWs in a 
comprehensive communication plan. (See below for more specific recommendations.)
The SAWSO Health Program Officer worked with the team in the fall of 2004 on conducting a 
Doer/Non-Doer analysis of exclusive breastfeeding practices. The purpose of the technique is to 
identify practices and beliefs that are associated with those who do breastfeed exclusively and 
20
those who do not. The results of the analysis are then to be used to refine behavior change 
activities. 
The previous Program Manager was slow in implementing this analysis and at the time of 
midterm the analysis had not been completed. At the midterm planning workshop the ACSP 
team made it a priority to finish revising their exclusive breastfeeding activities/messages based 
on the Doer/Non-Doer analysis. They also committed themselves to do the same analysis for 
hand washing.
As mentioned in the section on community mobilization, SAWSO will send a consultant to 
develop the community counseling program. While it will contribute to behavior change, it will 
not necessarily focus on the key child survival interventions of this project. In community 
counseling the community decides on the problem and poses the solutions. Given that ACSP is 
committed to facilitating specific behavior change it will need to use many other behavior 
change strategies. Following is a list of recommendations for this area.
Recommendations:
1. As stated above, the field supervisors must have major responsibility for spearheading the 
ACSP communication plan. This project cannot rely solely on the IMCI Coordinator.
2. The ACSP team must complete the design of their comprehensive behavior change plan. 
They have begun using the BEHAVE Framework and they should complete the analysis of 
enabling and conflicting factors and articulate a plan for messages and materials (e.g. flip 
charts, songs, pamphlets, etc.). 
3. The ACSP team could identify and train a group of CHWs to create and implement local 
behavior change plans. In South Africa child survival projects have an unusual resource in 
that the CHW are salaried. The ACSP team can do more to maximize this resource of full￾time community workers. There are CHWs who have musical, drama, drawing and 
leadership skills that can be used. The ACSP team could train and support them in creating 
area communication plans. They could become “communication specialists” and serve as a 
resource within the community health system. Thus the communication plan would become 
the responsibility of the CHWs, not the ACSP team.
c. Capacity Building Approach
Program Staff
At the mid-term all of the program staff has been trained in IMCI. The field supervisors received 
additional training in IMCI and supervision skills from the IMCI Coordinator. The project has a 
good system of cross-training so that team members fill in for each other and frequently work 
together. 
The Program Manager has made the commitment to revise the job descriptions for several staff 
members in order to allow for professional growth. For example, just before the mid-term 
evaluation the project hired a new bookkeeper. She demonstrated a wide range of skills in the 
community health area and the new project manager will modify her job description so that she 
can be involved in some of the community health work. Her competency in bookkeeping is such 
that it is not a full time task.
21
With the staff having good technical skills, the Program Manager needs to concentrate on 
developing leadership skills, especially with the field supervisors (Community Health 
Facilitators). The leadership skills that they need are in reference to enabling the CHWs to 
facilitate community health. A couple of examples are in the areas of health communication
(described above), supervision and program planning. They should be able to equip the CHWs 
and local clinics to set goals and monitor progress, instead of doing it themselves. There is no 
question that the ACSP team can implement these tasks; the larger challenge is developing their 
ability to work through others to improve the health of the community.
Department of Health Clinics
In the partnership between ACSP and the district Department of Health, the role of the 
Department is to develop the capacity of clinic staff, with the support and follow-up of ACSP. 
The focus of capacity building was to be IMCI. The reality is that the ACSP team has taken the 
lead in training clinic staff in IMCI, but its application has been minimal because of a lack of 
leadership from the Department of Health. 
At the mid-term the evaluation team interviewed staff at six clinics. In five of the six someone 
had been trained in IMCI, but training was the extent of their involvement. IMCI is not being 
implemented as a clinical practice in any of the clinics. None of the clinics have posted clinical 
care guidelines for IMCI (the IMCI algorithm for assessing a child). 
Another problem area is the relationship between the clinics and the CHWs. When asked about 
referral from CHWs, only one clinic stated they had more than seven a week; five stated they had 
one to six referrals a week; and one clinic had none. The staff in this last clinic stated that they 
did not even know who the CHWs were in their catchment area. There is no referral form for 
communication between clinics and CHWs. The ACSP team created one, but it is not being 
used.
Recommendations:
1. The Program Manager must reach an agreement with the district health authorities, in 
particular regarding the commitment to IMCI. IMCI is a national policy and the Program
Manager should facilitate discussions on how ACSP can help the Department of implement 
this strategy.
2. The ACSP team must facilitate planning meetings between CHWs and clinic staff regarding 
a stronger collaboration between the two. Each clinic and its CHWs should have measurable 
goals (related but not limited to the project’s goals), a regular planning and scheduling 
meeting (preferably monthly) and a patient referral system. As stated in the previous section 
the role of the ACSP team should not be to implement these tasks, but to facilitate the 
process of engaging the clinics and CHWs to complete these tasks.
Local NGOs
Functionally, the ACSP project area has three local NGOs: Lethempilo Youth Organization, The 
Salvation Army Mountain View Hospital and local churches (including TSA corps). The ACSP 
team has collaborated with Lethempilo in their home-based care program. Lethempilo however, 
relies on volunteer staff and only functions in Mondlo (one of the program’s five supervision 
areas). The greatest assistance that ASCP can give is to help Lethempilo apply for grant funding. 
22
One benefit of collaborating with ACSP in a funding application is that ACSP funding and 
program commitments can be used to leverage funding that would go directly to Lethempilo.
TSA’s Mountain View Hospital will have a leadership change that will enable it to participate in 
the project. The intention is that the hospital becomes responsible for the program in one of the 
project’s supervision areas. Their participation in the project will create the capacity to continue 
community health work after the USAID funding ends.
A third type of local NGO is churches. There is a wealth of churches scattered throughout the 
province. Through the pastors’ training program this project has the potential of creating a 
culture of ministry to the community (as opposed to just its membership). The participation of 
the Salvation Army Divisional Commander (or superintendent) in the mid-term evaluation 
generated a commitment on his part to do more to engage the Salvation Army churches in 
community health. The Salvation Army clearly understands the difference between ministering 
to people in need and proselytizing, thus its churches will be important collaborators in this 
program. 
At the end of the evaluation the team leader reported to the Salvation Army’s national leadership 
council. At this meeting the national leader (Territorial Commander) reiterated the 
organization’s intension of being involved in community health. He stated that the Territory will 
look to this program for leadership in engaging churches in other parts of the country.
e. Sustainability Strategy
Upon review of the sustainability indicators in the DIP the evaluation recommends that the
indicators be revised to reflect how the project has developed over the last year and a half. 
Following are the recommended indicators. By the end of the program:
1) 80% of the pastors/churches that attended the pastors’ training program will have in 
operation at least one of the following;
a) implementation of a home-based care program for persons living with AIDS;
b) implementation of a Life Skills program for either youth in their church or youth in their 
community;
c) implementation of a deliberate and structured program for OVC;
d) Sponsorship of a voluntary testing program for all community members.
2) 90% of the department of health clinics will have a functioning and documented;
a) CHW referral system,
b) A local plan with measurable child survival goals and an action plan for meeting the 
goals, in collaboration with CHWs.
3) 40% of the schools where the Life Skills curriculum has been implemented will have a 
Mentor who is from the community where the school is located. 
23
C. Program Management
1. Staffing
This project has experienced substantial staff turnover. Only one person has been on staff from 
the start of the project. In order to bring stability to the staff SAWSO has contracted with Faye 
Hannah MPH, to serve as Program Manager. Ms Hannah has over five years experience in 
working with SAWSO in child survival projects. Her professional degree, experience with child 
survival and knowledge of The Salvation Army equips her to give good leadership to the project.
As discussed in the previous section on capacity building, the current project staff has good 
technical knowledge. The challenge for staff development is that the staff to learn how to 
mobilize CHWs and their other partners to implement community health rather than doing it 
themselves. 
The SAWSO technical advisor, Claire Boswell, will need to continue to maintain monthly 
communication with the project manager on staffing and program management issues. They 
both need to be aware immediately of staffing problems and address them as soon as they arise. 
The project cannot afford any more staff changes. A good team is now in place and it needs to 
provide continuity for the remainder of the funding period.
2. Health Information System
One of the important accomplishments of this project has been the installation of a child survival 
data tracking system for the CHWs. Prior to the project the CHWs had no functioning 
information system. The ACSP team created a household card and worked intensively with the 
CHWs to implement the system. The project’s health information system now tracks child 
survival variables for all individual households that are served by CHWs. 
The major benefit of the household card has been in monitoring immunization defaulters. As 
discussed in the immunization section this system has resulted in an immunization rate of over 
90% in the communities that have CHWs. 
The health information system, however, does have areas for improvement. The most important 
limitation is that the data from the system are not used for program planning, except in the case 
of immunizations. Data are consolidated and aggregated monthly but are not used in the
monthly staff meetings. Additionally no indicator tracking reports are shared with the clinics 
and the Department of Health.
A second limitation is that CHWs do not have a way to easily review the status of families that 
they monitor. The household data are kept on separate sheets of paper so CHWs do not have an 
efficient way to visually scan the health indicators for all the families under their care.
A specific problem that is an outgrowth of the above is that there is no mechanism for following 
up on children with diarrhea and ARI. Because health data are on separate cards they cannot 
quickly identify which families reported an illness and what was the outcome. Reports of these 
illnesses only occur on the monthly consolidated report from the CHWs with no reference to the 
24
household or child. The number of cases of diarrhea in the past month, for example, is reported 
from memory. 
Recommendations:
1. The Program Manager must include a review of the health information system at each 
monthly planning meeting. The data should be used for setting quality improvement goals 
and then monitored at subsequent meetings.
2. The ACSP team must create a graphic reporting tool for communicating the status of the 
project with the provincial department of health and the individual clinics. The ACSP team 
was trained in using LQAS for program monitoring at the mid-term. They should repeat it at 
least semi-annually with a limited set of indicators and include these data in their reports. 
3. The ACSP team should consider substituting the individual household card with a registry 
book that lists all families and children under five followed by a grid that use symbols and 
check marks to indicate the status of health status variables for each child. These registry 
books are the most common registry that this evaluator has seen in child survival projects. It 
will cut down dramatically on the amount of paper that is used and makes monthly reporting 
more efficient because information on multiple families is on one page. And it is more likely 
that the CHWs will continue to use a registry book after the project is done rather than the 
individual household cards.
4. The ACSP team must set up a system where the CHWs set goals for their community and 
tract progress using their own data. CHWs should have graphs of immunization rates, ORS, 
etc, and report monthly progress to village headmen and township councils. These graphs 
should be placed where mothers can see them and visually monitor their progress. In other 
projects the graphs have been posted at clinics, schools, headmen’s houses, CHWs’ houses, 
and the village market center. CHWs and village leaders need to decide on the best place for 
them.
D. Additional Issues as identified by the evaluation team
No additional issues were identified by the evaluation team.
E. Conclusions and recommendations 
Major conclusions
Following is a summary of the conclusions contained in this report. Note that recommendations 
have been made throughout the report. They are linked to specific data and observations and 
thus are not repeated in this section.
1. Solid progress is being made in immunizations, control of diarrheal disease and HIV/AIDS 
prevention among young people. 
2. The data on ARI are mixed. The midterm KPC survey indicates negligible change but the 
midterm field survey indicated a high level of behavior change among mothers who are 
monitored by CHWs. The ACSP team will monitor this indicator more closely in its 
surveillance system.
3. Progress in exclusive breastfeeding and hand washing is limited.
25
4. The CHWs are diligent in keeping records and visit all targeted homes once a month. The 
project made a major contribution in equipping and training the CHWs in managing a 
household surveillance system.
5. The project has mobilized HIV/AIDS mentors and peer educators and pastors. This is an 
important contribution to expanding community involvement beyond the CHWs.
6. The collaboration with the district Department of Health is limited. The Program Manager 
will need to concentrate on strengthening this partnership.
26
F. Results Highlights
Of the 121 mothers who are interviewed during the midterm field evaluation, 69% said that they 
give ORT when their child has diarrhea.
During the field evaluation 92% of mothers who had a child with a “wheezing” type of cough
took the child to the health center. 
Over 90% of the children in the CHWs family registries are up-to-date on their vaccinations.
96
87 91
0
20
40
60
80
100
Percent
Jan-05 Feb-05 Mar-05
Chart 3: Immunization rates of children 11 to 24 
months from family registries (N=970)
The percent of ninth grade boys who state they are virgins increased from 48% at baseline to 
63% at the midterm.
The evaluation team interviewed six principals, seven teachers, ten mentors and seven peer 
educators. Each group was asked questions regarding their perceptions of students’ behavior and 
about their own experience as facilitators. As shown in the following table, it is significant that 
each group independently observed more open communication among the students. 
Interviewees Most Important Observed Behavior Changes
Principals 1. More open communication about sex and HIV
Teachers 3. Open communication about sex and HIV
4. More self control in classroom behavior between boys and girls (i.e. 
less passing notes, fewer suggestive comments, less flirting.)
Mentors 1. Open communication among students about sex and HIV.
Peer Educators 3. Boys show more respect to the girls
4. Open communication about sex and HIV