QUALITY 
ASSURANCE 
PROJECT
Center for Human Services • 7200 Wisconsin Avenue, Suite 600 • Bethesda, MD 20814-4811 • USA
TECHNICAL REPORT SUMMARY
The Zambia 
Quality Assurance Project
The Quality Assurance Project (QAP) is funded by the U.S. Agency for International Development
(USAID), under Contract Number HRN-C-00-96-90013. QAP serves countries eligible for USAID
assistance, USAID Missions and Bureaus, and other agencies and nongovernmental organizations
that cooperate with USAID. The QAP team consists of prime contractor Center for Human Services
(CHS); Joint Commission Resources, Inc. (JCR); and the following entities at the Johns Hopkins
University: the School of Hygiene and Public Health (JHSPH), Center for Communication Programs
(JHU/CCP), and the Program for International Education in Reproductive Health (JHPIEGO). QAP
provides comprehensive, leading-edge technical expertise in the design, management, and
implementation of quality assurance programs in developing countries. CHS, the nonprofit affiliate
of University Research Co., LLC (URC), provides technical assistance in the design, management,
improvement, and monitoring of healthcare systems in over 30 countries.
T E C H N I C A L R E P O R T S U M M A R Y
Table of Contents
I. Introduction ...................................................................................................... 1
A. Methodology ............................................................................................... 1
B. Quality Assurance in Zambia ..................................................................... 2
II. Achievements and Challenges ........................................................................ 3
A. Developing Standards of Quality ............................................................... 3
B. Communicating Standards ........................................................................ 5
C. Quality Performance of Health Facilities ................................................... 7
D. Evaluation of Problem-Solving Teams ....................................................... 9
E. Evaluation of the QA Support Systems.................................................... 13
F. Other Structures ....................................................................................... 17
III. Recommendations for Strengthening the QAP ............................................. 19
A. General Recommendations ..................................................................... 19
B. Standards of Care .................................................................................... 20
C. Quality Improvement Activities ................................................................ 20
D. QA Support Systems ................................................................................ 21
E. Prioritizing the Recommendations ........................................................... 22
IV. Conclusion ...................................................................................................... 23
Abstract
This report summarizes an earlier,
longer report titled, “The Zambia
Quality Assurance Program: Final
Evaluation.” Like the longer report,
it provides information on Zambia’s
Quality Assurance Program (QAP)
and the evaluation that took place five
years after the 1993 start of the QAP.
This report first describes the
purpose and methods of the evalua￾tion and then provides background
on the Zambian QAP. Amid severe
shortages, the QAP provided training
for problem-solving teams, created a
network of coaches and link facilita￾tors to support those teams, and
involved ancillary organizations in
improving the quality of healthcare in
Zambia. The summary is particularly
detailed in explaining the structures
and successes of the various aspects
of quality assurance that evolved with
the QAP’s assistance. It provides
interesting details in such areas as
the conditions necessary for problem￾solving teams to form and solve
problems, the difficulties teams had
in gathering data, and the innovative
ways healthcare centers supported
each other. Also detailed are
revelations on supervisors’ unwilling￾ness to observe the provision of
healthcare and the pitfalls encoun￾tered when clinical standards were
developed without the assistance of
the QAP or healthcare providers.
Lastly, the report provides extensive
recommendations for strengthening
Zambia’s QAP, particularly calling for
leadership from the central level in
developing policy, management
structures, and resources for the QAP.
While the lower levels of the system
have improved significantly in
providing quality healthcare, the
recommendations suggest numerous
changes to raise that quality to the
next level.
The Zambia
Quality Assurance Program
Acknowledgments
Appreciation is due to the members of the evaluation team who contributed
to the Evaluation Report that served as the basis for this Technical Report
Summary. The team included Bruno Bouchet, MD, MPH; Chimshimba
Lumbwe, MD; John Øvretveit, PhD; Eleanor Msidi; Maria Francisco; and
Maryann Lyby. Each was essential in making the evaluation and the report
a success.
The team members are indebted to Zambia’s Central Board of Health
(CBoH) staff for their assistance throughout the mission, both at a technical
level and for secretarial assistance, physical facilities, transportation, and
equipment made available. Without their efforts, the logistics undoubtedly
would have been more challenging and the output less fruitful. Senior
officials’ willingness to be available is recognized with considerable
gratitude. The atmosphere of collaboration that prevailed throughout the
mission characterizes the partnership well.
The evaluation team acknowledges the support provided by the United
States Agency for International Development (USAID) mission in Lusaka in
funding this evaluation and thanks the Agency for giving the Quality
Assurance Project the opportunity to contribute to health sector reform in
Zambia. The evaluation was supported by USAID under Contract No. HRN￾C-00-96-90013 to the Center for Human Services (CHS).
The team felt privileged to be associated with such an important undertak￾ing and would like to give its special thanks to Eddie Limbambala,
Mwendaweli Maboshe, Joyce Tembo, and Albert Mulungu for facilitating the
mission. We are also grateful to Augrey Mwenda, CBoH, and Pat Sayer,
University Research Co., LLC,-Center for Human Services (URC-CHS), for
their essential contributions.
Foreword
An evaluation of this nature could be threatening to the stakeholders of the
Quality Assurance Program (QAP) in Zambia, but the findings should
mitigate their concerns. The intention of the evaluation team was to draw
lessons that could benefit both Zambia and other countries. In doing so,
achievements are recognized and remaining challenges identified.
Despite its efforts to validate findings and their interpretation, the evaluation
team is aware of the risk of reporting inaccurate data and missing relevant
information. The team takes full responsibility for such omissions and
inaccuracies; the views expressed in this report reflect the opinion of the
evaluation team only.
Recommended citation
Bouchet, B. 2001. The Zambia Quality Assurance Program. Technical
Report Summary 1(3). Published for the United States Agency for Interna￾tional Development (USAID) by the Quality Assurance Project: Bethesda,
MD.
About this series
The Technical Report Summary series provides concise descriptions and
background information on the Quality Assurance Project’s technical
intervention results, evaluation results, and other project activities. This
summary is derived from a longer report that was presented to Zambian
and U.S. officials after the evaluation. For a copy of the longer report, write
to qapdissem@urc-chs.com or call 301-941-8532. For more information on
the evaluation of Zambia’s QAP, please contact bbouchet@urc-chs.com.
Abbreviations and Acronyms
BASICS Basic Support for
Institutionalizing Child
Survival
CBoH Central Board of Health
CHESSORE Centre for Health, Science
& Social Research
CHS Center for Human Services
CMAZ Churches Medical
Association of Zambia
DHB District Health Board
DHMT District Health
Management Team
DHO District Health Office
DySSSy Dynamic Standards
Setting System
GNC General Nursing Council
HIV/AIDS Human Immuno-Deficiency
Virus/Acquired Immuno￾Deficiency Syndrome
HMIS Health Management
Information System
HRIT Health Reform
Implementation Team
IMCI Integrated Management of
Childhood Illnesses
ITG “Integrated Technical
Guidelines for Frontline
Health Workers“
MCZ Medical Council of Zambia
MOH Ministry of Health
NGO Nongovernmental
Organization
NHC Neighborhood Health
Committee
OPD Outpatient Department
QA Quality Assurance
QAP Quality Assurance Program
STD Sexually Transmitted
Disease
TB Tuberculosis
UNICEF United Nations Children’s
Fund
URC-CHS University Research Co.,
LLC,-Center for Human
Services
USAID United States Agency for
International Development
UTH University Teaching
Hospital
WHO World Health Organization
ZHAC Zambia Health
Accreditation Council
The Zambia Quality Assurance Program (TRS) ■ 1
Bruno Bouchet
The Zambia
Quality Assurance Program
I. Introduction
This summary presents the results
of an evaluation of Zambia’s Quality
Assurance Program (QAP). A
developing country of nine million
people, Zambia is located in
southern Africa. Its infant mortality
rate is estimated at 109 deaths per
1,000 births, and maternal mortality
accounts for approximately 13
percent of all deaths of women
between the ages of 15 and 49.
Zambia has many serious health
challenges coupled with limited
resources, widespread poverty,
and rising unemployment.
The evaluation occurred from
September 14 to October 2, 1998.
Its main objective was to provide
information and guidance for the
Ministry of Health (MOH) to design
the next steps for improving the
quality of healthcare in Zambia.
Secondary objectives were to:
(a) review previous achievements,
(b) identify problems and chal￾lenges, and (c) make recommenda￾tions to overcome challenges. This
information would guide discussions
on this topic during a November
1998 meeting between the Zambian
MOH and its partners.
A. Methodology
The quality assurance (QA) evalua￾tion covered nine of Zambia’s 67
districts and included visits to 24
health facilities. Districts were
chosen to include health centers
with active QA teams and/or staff
who had received QA training.
Districts and facilities that were
easily accessible to the evaluation
team were given priority, though an
attempt was made to find represen￾tative sites, both rural and urban.
Because the choice of facilities was
not entirely random, the evaluation
team cautions that the sample may
not be representative of the entire
country.
The evaluation team worked in three
subteams: one each on standards,
problem solving, and support
systems. This composition corre￾sponds with the Quality Assurance
Project’s1 strategy to build QA
capacity at the district and health
center levels by training staff to:
(a) set standards for health services,
(b) monitor indicators of achieve￾ment, and (c) use team-based
problem solving. The evaluation
team collected information primarily
through semi-structured interviews
with relevant staff at the central,
regional, district, and health center
levels.2 The evaluation team
presented preliminary findings to the
Central Board of Health (CBoH: part
1 Headquartered in Bethesda, MD, the Quality Assurance Project provides technical
assistance for the design, management, improvement, and monitoring of healthcare
systems and service delivery in over 30 countries, including Zambia. While the Quality
Assurance Project is often identified by its acronym, “QAP,” that designation is reserved
here for Zambia’s Quality Assurance Program.
2 The data collection guides used by the subteams are available in “The Zambia Quality
Assurance Program: Evaluation Report.” Ordering information is provided on the
previous page under “About this series.”
The richness of the QA
experience in Zambia
provides lessons that will ovides lessons that will
benefit not only health
sector r sector reform in Zambia, eform in Zambia,
but also other QA pr but also other QA programs.
2 ■ The Zambia Quality Assurance Program (TRS)
of the MOH) and its partners at the
end of the mission and incorporated
their thoughts into the report.
The original scope of work included
estimating costs for various QA
support systems, but the local
expert identified for this component
was unable to join the team at the
last minute.
B. Quality Assurance in
Zambia
There are good clinical and financial
reasons for QA, especially where
resources are scarce. For example,
an immunization program may
achieve a high coverage rate, but
vaccines may be ineffective if staff
cannot ensure proper storage. QA
can prioritize and address these
problems close to the source.
In addition, patients in Zambia and
elsewhere who have begun making
co-payments for health services are
demanding that these services be
attractive and valuable. Health
providers need help in responding
to these new demands, which
include cultural and gender factors.
Quality methods enable providers to
work with the community to assess
its needs while providing high￾quality medical services.
There is no national, specific QA
program in Zambia, and the evalua￾tion team found no specific policy or
strategy to evaluate in terms of
objectives and outcomes. It was not
entirely clear to the team whether
one CBoH unit was in charge of QA
activities or if all CBoH units were
involved. Quality activities in Zambia
occur in two broad categories: (a)
the work of the national QA Unit, with
its national network of coaches and
link facilitators, and (b) various
quality activities that have occurred
since 1993.
The QA Unit
The Zambian QAP was the responsi￾bility of the QA Unit of the Health
Reform Implementation Team of the
MOH from 1993 to 1996. With the
creation of the CBoH in 1996, QA
activities became the responsibility
of the Directorate of Monitoring and
Evaluation. From the outset, the QA
Unit chose to concentrate on
developing standards and their
indicators, monitoring standards,
and problem-solving teams. The QA
Unit defined quality assurance as
“the measurement of the actual level
of the service provided plus the
efforts to modify, when necessary,
the provision of these services in the
light of the measurement.”
The QA Unit coordinated QA
training; supported the development
of QA activities at the local and
district level; helped create a quality
laboratory policy; issued a leaflet
titled, “Patient’s (Client’s) Entitle￾ments and Responsibilities”; and
worked to integrate QA into other
activities and sensitize personnel to
quality concepts and issues.
Training proceeded as follows:
■ The QA Unit’s national training
program first piloted QA training
and implementation in 1994–95 in
three districts. Sensitization
workshops (to introduce QA
concepts), training in DySSSy,3
and indicator development were
presented to each district office
and its health centers.
■ Next, in 1996, district training and
pilot testing of a problem-solving
and peer review system—where
one officer-in-charge would visit
another health center and assess
it using a UNICEF-based check￾list—occurred in six Livingstone
health centers.
■ Third, sensitization and DySSSy
training were given to three
provincial office personnel in all
nine provinces.
■ Fourth, DySSSy training was
provided to certain personnel
from all districts. About 300
people were trained, including 60
physicians, all district directors
and their deputies, and two other
District Health Management Team
(DHMT) members from each
district. This training lasted 10
days: one day on sensitization,
four on DySSSy, and five on
indicator development.
■ Last, starting in 1996, the QA
Unit coordinated training in team
problem solving, using a five-step
cycle advocated by the Quality
Assurance Project. This training
covered all districts except those
in North-Western Province.
Approximately 230 people were
trained, including 10 physicians.
Each trainee was expected to
become a district coach and to
train staff at their own health
center. This process would
extend understanding of QA
concepts and methods beyond
the original trainees. Provincial
(later termed “regional”) link
facilitators were identified to help
with the training. The training
lasted 14 days and included
three days on QA team building
3 The Dynamic Standard Setting System (DySSSy) is a step-by-step method for developing standards. It provides that those who will follow,
implement, and/or achieve certain standards should participate in their development.
The Zambia Quality Assurance Program (TRS) ■ 3
and nine on coaching and
problem solving. Coaches were
then validated through practice.
Three to six months later, each
regional link was to visit the
coach who would give a one-day
training session to staff at another
health center in order to be
validated.
National QA Structure of
Coaches and Links
The Zambia QAP’s structure and
process of quality supervision and
coaching includes supervisory staff
of two to three at the QA Unit, 30 or
more regional links, and about 200
district coaches (two to four per
district). The QA Unit established
and trained quality committees in
provinces, districts, and hospitals,
but did not create formal reporting
links between them (although there
were national meetings). It changed
its approach in 1996 to focus on
districts and health centers. The QA
Unit was dissolved in 1997 after the
creation of the CBoH.
Regular meetings, including national
quarterly meetings for all regional
links to report activities in districts,
began in 1996, but ceased in 1998
when workshops were banned.
Regional links visit and support
coaches and lead local training
with coaches for some staff from
all facilities in a district. In certain
cases, coaches are expected to
run the training themselves. These
district training events usually last
three to five days and often involve
the officer-in-charge of the health
center. Participants who come to
these events are expected to
become trainers; they can then train
colleagues at their own facility and
start problem-solving teams.
Coaches are expected to do
follow-up visits to trainers to provide
support to them and the problem￾solving teams. Some coaches train
staff only at their own or another
health center, rather than covering
the whole district.
Problem-solving teams: A team of
two to 15 people was expected to
form in each facility to identify and
work on problems (one at a time)
using the five-step, problem-solving
cycle. Approximately three to six
such teams have been formed in
most districts: a total of 150–300
teams. In addition, some hospitals
formed “quality committees,” which
may function as problem-solving
teams. Standard setting and
indicator development have
occurred in some health centers,
and some districts have developed
and disseminated protocols.
Local activities with a quality
component: Quarterly supervision
visits of health centers by district
personnel can involve assessment
and feedback about the facility,
clinical practice, and patient
relations, but often the last two are
not covered. Neighborhood, health
center, and area committees present
consumer views to health centers
and districts, and can work with staff
to resolve quality problems. Routine
systems for the collection and use
of information include some quality
indicators, including the HMIS4 self￾assessment forms, quarterly
progress reports, IMCI5 monitoring
surveys, and Safe Motherhood. In
addition, the performance audit by
regions was intended to include a
quality component.
Another significant QA activity: In
1998 the Zambia Health Accredita￾tion Council (ZHAC) worked in
collaboration with the Joint Commis￾sion International (now the Joint
Commission Resources, Inc.) to pilot
test the accreditation of hospitals.
II. Achievements and
Challenges
This section describes the achieve￾ments made by and challenges still
facing the QAP in: (a) the develop￾ment of standards of care, (b) the
communication of standards to
health providers, (c) mechanisms to
monitor compliance with standards,
(d) the productivity of the problem￾solving teams, and (e) the establish￾ment of support systems.
A. Developing Standards
of Quality
Standards are statements of
expected quality. They are essential
to quality because they facilitate the
objective assessment of perfor￾mance. By making an organization’s
expectation for quality explicit,
standards provide guidance for
actions and decisions in delivering
care. If quality is “doing the right
thing right, right away,” then stan￾dards indicate who should be doing
what, in what way, and when.
4 The Health Management Information System (HMIS) provides a mechanism for collecting data on health statistics and sets goals for
certain health indicators.
5 The Integrated Management of Childhood Illness (IMCI) algorithm is a set of clinical practice guidelines developed by the World Health
Organization (WHO). IMCI guides healthcare providers in the delivery of healthcare to children (and includes instruments to gather data
on childhood illness).
4 ■ The Zambia Quality Assurance Program (TRS)
Standards are usually adapted by
local experts from already-existing
material. In practice, locally recog￾nized experts are usually asked to
develop standards, using either
existing standards or standards from
other healthcare systems as a
starting point. When standards must
Case Study Number 1:
Kabwe District Health Office Develops Its Own Protocols
During supervisory visits, the link facilitator of the Kabwe DHMT noticed that the
referral of women in labor to the hospital was inappropriate, resulting in maternal
deaths before or just after arrival. Once women arrived, admissions procedures
were incomplete and the gravity of their condition was not correctly assessed.
The supervisors also noticed that the management of a patient at the outpatient
department of the hospital consisted mainly of a drug prescription for the relief of
the main symptom without any physical examination. In fact, the procedures for
clinical management of both situations varied considerably among health
providers.
The DHMT concluded that health providers needed a clear protocol for the
referral and admission of women in labor and for the screening of patients
consulting at the outpatient department. The link facilitator had been trained by
the Zambian QAP in a participatory method to set standards (DySSSy) and used
his skills to improve the work of the health providers. Two separate groups, one of
midwives and the other of clinical officers, met three times in one month for two to
three hours each time. The only costs were minimal transportation expenditures.
The midwives used some medical textbooks to adapt standards for referral and
admission of women in labor; the clinical officers brainstormed the screening
procedures at the outpatient department. Both groups quickly reached a consen￾sus, but encountered some difficulties at the beginning of implementation. Health
providers said that they do not like lengthy checklists because they are too time￾consuming and because they felt detailed procedures were a way to control their
practice. As a result, they take shortcuts. Their involvement in the development of
the protocols helped lift the resistance. (The main constraint was to be able to get
the people together because of time limitations.)
The Kabwe DHMT learned some lessons from this experience. First, the providers
that most resisted the application of the new protocols were the ones not involved
in their development. Second, the staff gained confidence by setting realistic
standards for their working environment. Third, the staff perceived that its work
was made easier because the protocols were posted on the walls as a reminder.
The district is following up on the compliance of health workers with the new
protocols through supervision visits. Motivated by its success, the Kabwe team is
setting standards for the packaging and prescription of antibiotics. The protocols
developed by the Kabwe District Health Officer are presented in Appendix B of
The Zambia Quality Assurance Program: Final Evaluation.
be scientifically sound, their validity
should be ensured by a review of
evidence-based medical literature.
When expert consensus is inappro￾priate for developing standards, the
DySSSy method guides a health
facility team in developing its own
standards (see Case Study 1).
There is no official policy for the
development and revision of national
standards of healthcare in Zambia,
although the QAP indicated that the
Systems Development Directorate is
in charge of this kind of work.
Numerous clinical care standards
have been developed, covering all
activities of the essential package of
services that first-level health
facilities provide. They mainly fall
into four categories:
Guidelines that focus either on one
health condition or on a set of
special issues: These efforts have
resulted in guidelines for malaria
case management, maternal health,
tuberculosis (TB), and HIV/AIDS.
Guidelines by international health
agencies: These guidelines promote
an integrated approach targeting a
special population. They include:
(a) the syndromic case management
of sexually transmitted diseases
(STDs), and (b) IMCI.
The “Integrated Technical Guidelines
for Frontline Health Workers” (ITG):
This manual of standards identifies
the six health thrusts that address
Zambia’s priority public health
problems—malaria, reproductive
health and family planning,
HIV/AIDS and STDs, child health
and nutrition, TB, and water and
sanitation—and prescribes their
management. The development of
the ITG is described below.
Standards for accreditation of
hospitals: The accreditation
standards will cover all hospitals in
Zambia. A document from the MOH
on certification of hospitals de￾scribes minimum standards for
hospital levels 1, 2, 3, and 4.
Adoption of the policy was pending
during the evaluation.
The Zambia Quality Assurance Program (TRS) ■ 5
The QAP approached the develop￾ment of standards by training
healthcare providers and their
managers to use the DySSSy. Staff
used this method to develop a
manual of standards for clinical and
nursing care. The manual served as
a training document for setting local
target standards aimed at initiating
action plans for the improvement of
health services by trainees. Trainees
were expected to involve their peers
in developing standards adapted to
their specific situation. DySSSy both
involves health providers in the
development of the standards they
will use and exposes staff to the
concept of a systems view and its
input, process, and outcome
components. Although the results of
this effort were uneven—with some
teams more advanced than others—
it helped build staff capacity to
develop action plans and monitor
their progress.
The evaluation team found that
despite the DySSSy training, the
development of clinical guidelines
remains centralized, with little
involvement of the QAP. The team
determined that numerous difficul￾ties prevented the trainees from
developing standards; no one
reason seemed to predominate.
One possible reason is that if only
one person per health facility is
trained, the absence of a sense of
ownership of the process might limit
their commitment to implement
standards. A sense of team authority
may also be necessary: trainees
who participated in successful
teams tended to be facility in￾charges, older than other team
members, and/or female. Having
sufficient staff with these character￾istics on the team may not be
achievable for some facilities. Also,
when a health facility’s staff is too
small (fewer than 10), it might be
more difficult to initiate and sustain
the team dynamic. Another contrib￾uting factor might be that trainees
are unable to transfer their knowl￾edge to their peers after being
exposed to a new concept they
have not entirely mastered.
At the central level, the Systems
Development Directorate noted a
lack of ownership and coordination
by the CBoH in the development of
clinical guidelines by different
programs. Because the various
existing guidelines may have been
causing confusion among health
staff, the CBoH developed the ITG
mentioned above. To create this
manual/reference book, a task force
on integration of vertical programs
assembled all existing guidelines,
and experts from the MOH, CBoH,
University Teaching Hospital (UTH),
and other institutions collaborated to
finalize the document.
The evaluation team did not identify
regional directorates or health center
staff who had been involved in the
development of any standards of
care, including the ITG. Most of the
DHMTs had received the ITG draft
and were asked to suggest
changes, but they largely felt that by
the time they received the draft, it
was too advanced for their input to
be useful.
The evaluation team was unable to
document some aspects of the
development of the ITG, which
involved the collaboration of many
public health and medical experts. It
would have been useful to evaluate
the use of evidence-based medical
literature, the cost of the ITG’s
development, and how it could be
produced in only five months.
The evaluation team also learned
that the role of the private sector in
the development of clinical
standards is limited to the irregular
participation of local experts in the
writing of medical textbooks or
guidelines for health providers.
B. Communicating
Standards
After clinical standards have been
set, the health providers expected to
comply with such standards must
know them, understand them, have
the skills to apply them, and accept
them. Success in achieving accep￾tance requires going beyond the use
of traditional channels and calls for
activities that will induce and sustain
a behavioral change in clinical
practice. This is complex since
practitioners accustomed to manag￾ing a health condition in a certain
way are unlikely to change their
practice solely on the basis of the
training. Substantial research in this
area has been done in industrialized
countries and has established
principles of behavior change—for
example, the fact that involving
users in developing the standards
facilitates change. However, there is
little research on this issue in
developing countries.
In-service training might be essential
in communicating standards, but it
should be supplemented with other,
cost-effective ways, such as:
■ The involvement of regulatory
bodies and professional
associations
■ Distance-learning programs
(computer, radio)
■ Distribution of user-friendly
leaflets
■ Inclusion of the new standards in
the preservice curriculums of
training institutions
6 ■ The Zambia Quality Assurance Program (TRS)
■ Observation of clinical practice to
ensure that health providers are
complying and, if not, to affect
correction
The plan to communicate standards
in Zambia relies entirely on class￾room training, which the CBoH is
using for combined ITG and HMIS
training. If only one staff member per
clinic is trained, in-facility dissemina￾Case Study Number 2:
Communicating Integrated Technical Guidelines
The CBoH published the ITG in May 1997. This 216-page document was intended
to help providers manage the six most important health thrusts by reminding them
of the protocols. It explicitly delineates standards for quality care and services
and was the first attempt to compile quality standards for the delivery of the
essential package of health services.
Eighteen months after its release, the use of the ITG by the target population
remained limited for two reasons: a delay in its dissemination and its format.
None of the regional directorates that the evaluation team met participated in the
distribution of the ITG to its target audience. In fact, one regional directorate was
not even aware that they had a copy in their library. All the DHMTs visited had
received it, but only one or a few copies and not enough for all health centers.
Most DHMTs did not know how to distribute it and were reluctant to make copies
because of the amount of paper it would consume. Only one DHMT reported
writing a summary and photocopying it for the health centers. All regional director￾ates and DHMTs reported that they found the ITG “bulky, not user-friendly, and too
complicated for some staff.” Consequently, the vast majority of the frontline health
workers that the evaluation team met did not have it. Some had heard about it,
and the few who had it did not know how to make the best use of it. Some district
officers, however, discussed the ITG content with health center staff during
technical meetings. Although the evaluation team did not further explore the issue
by testing users’ knowledge of the standards, the main lesson drawn is that any
attempt to develop standards must be accompanied by a clear and thoroughly
thought-out communication strategy. Just sending guidelines without explaining
how to use or communicate them is not enough to induce change.
The CBoH is currently providing ITG training combined with the training of the
districts in the HMIS. Classroom type in-service training seems to be a mandatory
step but might not be enough to ensure compliance with standards if user-friendly
job aids are not developed in a way that involves the health workers. More
operations research on this specific issue is required. The influence of guidelines
on clinical practice must draw on behavioral change theories. This also requires a
needs assessment for practical job aids that health workers will be willing to use.
tion might be unreliable and perhaps
ineffective.
Most of the regional directorates and
DHMTs met by the evaluation team
did not have a specific strategy for
communicating the ITG to staff.
Distributing it was a problem, since
photocopying resources were
limited, so few health centers had
received it. The communication
method has not been fully success￾ful: District Health Officers (DHOs)
were familiar with the ITG, but few
health practitioners were. It had not
been given to private practitioners
and other institutions (see Case
Study 2).
District health staff perceive the ITG
format not to be useful as part of
their clinical practices, although they
welcome it as a reference book. For
instance, different chapters have
different structures—some are
organized as flowcharts, others have
lists of instructions, and others are
narrative only.
Health providers had difficulty
articulating what kind of job aids
they need and would use during a
consultation to remind them of the
steps in the management of a
specific health condition. In addition,
district staff do not use the other
guidelines. In the districts trained
only in IMCI, a minority of health
workers mentioned using the IMCI
guidelines. Others said that follow￾ing the guidelines takes too much
time.
Lusaka District has developed some
protocols—and trained staff in their
use—for common causes of consul￾tation. The documents look user￾friendly, providing both an algorithm
and a narrative explanation of the
case management steps, but staff
did not appear to be using them.
The evaluation team concluded that
the main constraints to an effective
communication strategy are: (a) the
absence of an integrated policy for
the communication of standards that
relies on principles for behavior
change, and (b) the complexity of
the factors that influence a clinical
practice change in a specific society
or context. The most cost-effective
strategy to communicate standards
should be determined.
The Zambia Quality Assurance Program (TRS) ■ 7
C. Quality Performance of
Health Facilities
The evaluation investigated
mechanisms to assess quality
performance: the degree of compli￾ance with structural standards for
health facilities, with standard
operating procedures by managers,
and with clinical care standards by
health providers. The team looked
mainly at the technical quality of the
healthcare delivery process, trying
to answer the questions, Are health
providers doing what they should?
How can we know? And how can
service be improved?
Many health system stakeholders
are involved in monitoring perfor￾mance. Each level of the health
system monitors the performance of
the level directly below. Regional
directorates audit the district teams
who, in turn, supervise the health
centers.
All cost centers (every unit that
manages a budget) are involved in
some sort of self-assessment, since
they report on their performance
toward achieving their action plans
(quarterly progress reports) and
soon will report on the national
health outcome targets that are
part of the HMIS. The regulatory
bodies—the Medical Council of
Zambia (MCZ) and the General
Nursing Council (GNC)—monitor the
private clinics and hospitals.
The involvement of the users in
monitoring health services and
providers is not formalized through
well-defined mechanisms. Some
health facilities have suggestion
boxes; however, only literate patients
can use them and they usually do
not. The Neighborhood Health
Committees (NHCs) present a way
to involve the community in monitor￾ing quality while educating users on
what to expect. In at least one
instance, NHC members were
actively involved in a problem
solving.
The districts have four main mecha￾nisms to monitor performance:
Performance audits: Each regional
directorate is expected to perform a
quarterly performance audit,
consisting of an inspection of
DHMTs and facilities in their regions.
In practice, the regional team
inspects the district team, and
together the regional team and
district team inspect one or two
facilities per district. These audits
mainly collect information on facility
structural standards and manage￾ment. Progress according to action
plan is monitored through output
and outcome measures for all cost
centers, using record reviews,
observation, and interviews with
staff. Few indicators collect informa￾tion on process standards, and there
is no direct observation of care. This
means that the clinical performance
of health workers remains unknown.
Supervision visits: All DHMTs
indicated that they supervise all
health centers monthly or quarterly,
but this was not always confirmed
by the health centers or the supervi￾sion reports. Formal supervision
visits usually occur in teams of three
or four and last one to two hours.
The evaluation team identified four
different checklists that can be used
during supervisory visits; all have
serious shortcomings to assess the
quality of clinical care. An attempt
by the CBoH to develop an
integrated checklist involving direct
observation of care had been
started but not completed.
The evaluation team found that
DHMT staff avoid observing clinical
performance of health workers.
Consequently, supervision reports
emphasize the importance of
managerial functions (record
keeping) and structural criteria (a
painted wall or a well-kept garden),
signaling that compliance with
clinical standards is unimportant.
The Health Management Information
System: This system collects and
analyzes statistics. It includes
quarterly self-assessment forms that
are completed by health centers and
then by districts to monitor their
performance against national and
local targets. Information is collected
on 15 input and output indicators
related to utilization of services and
coverage. The indicators are
designed to help teams identify the
areas of low performance and then
take action. The only indicator that
measures performance is the “daily
staff load for curative and preventive
care.”
The quarterly progress reports: Each
district sends quarterly progress
reports to the CBoH. This self￾reporting of the achievements
toward district action plan targets is
used to disburse grant money. All
areas of administration (e.g., number
of meetings held), service activities
(e.g., coverage and utilization rates),
and purchase of supplies and
equipment are included. This
mechanism will be replaced by
HMIS procedures. Here again, the
focus is input and outcome data
rather than data that would measure
health providers’ compliance with
clinical care standards.
District-specific activities: The
evaluation team found examples of
innovative district-specific activities
to both monitor quality of care and
improve it. For instance, DHMTs
organize and perform unannounced
8 ■ The Zambia Quality Assurance Program (TRS)
spot-check visits of health centers
as needed. These activities and their
impact cannot be known without
additional documentation.
Documentation of Changes in
Quality Performance
While the results of special surveys
are limited and do not allow com￾parison over time, most information
on the technical quality of care
comes from such surveys or other
research. The evaluation team
identified the following:
A 1993 survey by the MOH Health
Systems Research Unit revealed
that “Standard WHO/Control of
Diarrheal Diseases guidelines for
case management were adhered to
by prescribers in 51.7 percent of
patients with acute diarrhea and
42.3 percent of patients with
dysentery.” As a result, in 77 percent
of cases, oral rehydration solution
and antibiotics were incorrectly
prescribed, and in 58 percent,
incorrectly dispensed.
A 1996 review by the Participatory
Assessment Group raised concern
that diseases are diagnosed simply
by talking with patients and that
even the use of a thermometer is
rare.
The 1997 Zambia Situation
Analysis of Reproductive and Child
Health Services found several low￾performance areas: 1 percent of
health providers test for pregnancy,
21 percent perform a clinical exam,
and 2 percent look for symptoms of
STD. It also revealed serious gaps in
the case management of sick
children under the age of five
according to IMCI standards.
A 1997 review of the HMIS
cascade training noted that “health
workers have difficulties in diagnos￾tic skills, which affects not only
quality of care, but consistency and
quality of data as well.” It also states
that the HMIS is hampered by a
tremendous load of insufficient
diagnostic and curative skills.
The 1997 review of the Extended
Program on Immunization in Zambia
revealed numerous performance
gaps that could prevent the high￾coverage levels needed to reduce
disease-specific morbidity and
mortality in children. Among those
gaps, 54 percent of refrigerators had
their temperature checked daily, 50
percent of sterile techniques met
standards, and 44 percent of the
mothers knew when to return.
A survey by the Centre for Health,
Science & Social Research
(CHESSORE) on quality of care
indicated that physical examination
was performed on only 39 percent of
patients with malaria at rural health
centers.
IMCI Documentation
The only results that the team found
on monitoring compliance with
process standards over time come
from repeated facility-based surveys
by the BASICS (Basic Support for
Institutionalizing Child Survival)
program for the implementation of
IMCI. The main results are presented
in Case Study Number 3. Results in
other IMCI provinces are consistent
with these findings.
A strong QAP creates an opportunity
to establish a quality performance
monitoring system. Although
monitoring compliance using input
and outcome standards is important,
any change in output or outcome
measures (such as coverage rates)
is very difficult to interpret without
knowing the specific intervention or
structural change. The paucity of
performance data was a serious
constraint to the evaluation.
How Performance Information
Is Used to Improve Quality
Performance information is used in
an ad hoc manner. DMHTs have no
formal process to review and
analyze performance information
collected during supervision, so
the compilation of results is not
systematic.
All supervisors remarked that they
give some feedback on their
findings to the health center staff.
Most write recommendations in the
health center visitor book and/or a
report kept at the district health
office. The evaluation team reviewed
some of these documents and found
little focus on clinical care delivery.
Health providers’ competency is
almost never tested. No district
asserted that they test the knowl￾edge and skills of health providers
to determine the causes of poor
performance. The supervisors
mentioned asking providers about
reasons for their findings, but they
asked no questions on the case
management of a specific health
condition.
The HMIS attempts to integrate
performance measurement with
quality improvement methods, but its
indicators do not capture information
on the process of care. The develop￾ment of a link between the results
from the HMIS and the problem￾solving teams through the “Triple A”6
approach is a positive development.
The way health output and outcome
data will be used to make decisions
6 “Triple A” stands for assessment, analysis, and action. It is an adaptation of the problem-solving methodology taught by the Quality
Assurance Project.
The Zambia Quality Assurance Program (TRS) ■ 9
Case Study Number 3:
Monitoring in Lusaka Improves Outcomes
After the introduction of IMCI, four health facility-based surveys documented
change in health worker performance over a two-year period in Lusaka District.
The compliance of health workers with IMCI was measured through direct
observation of provider/patient encounters three months before the IMCI training,
and then repeated at several month intervals. The results in Table 1 show that
performance was very poor before training, improved dramatically after, and then
started declining.
The DHMT was concerned about this trend and decided to include observation
of the sick child in the supervision visits. Supervisors were trained in IMCI and a
one-page checklist was designed to serve as a job aid and was added to the
supervision checklist. During their visits, supervisors identified tasks not
performed correctly by the health workers and provided on-the-job feedback
and training. This strategy produced improvements, as shown by the results in
Table 2.
This case illustrates the influence of regularly reminding health providers about
the standards of care after identifying shortcomings through direct observation of
their performance. When supervisors pay attention to the important aspects of
care, the impact of making a correct diagnosis, prescribing a correct treatment,
and the health outcome are better. While not unique, the Lusaka case represents
a best practice for Zambia that should be disseminated to other districts. Its
lessons should not be limited to the IMCI standards but expanded to encompass
the standards for the other health thrusts.
Table 1
Changes in Provider Performance with Training over Time
The Health Worker: Before Two Months Eight Months
(Percentage of Times) Training after Training after Training
Counted respiratory rate for cough 2 70 67
Checked dehydration by pinching skin 34 55 60
Prescribed antibiotics for a common cold 47 15 28
Explained the treatment 25 69 40
Table 2
Changes in Provider Performance with Clinical Supervision
The Health Worker: Before Clinical After Four Rounds
(Percentage of Times) Supervision of Clinical Supervision
Counted respiratory rate for cough 67 84
Checked dehydration by pinching skin 60 61
Prescribed antibiotics for a common cold 28 17
Explained the treatment 40 79
and solve problems should be
monitored.
The evaluation team is concerned
with supervisors’ apparent unwilling￾ness to observe the delivery of care.
The team hypothesizes that supervi￾sors: (a) do not feel competent to
assess the technical service they
would observe, (b) are uncomfort￾able observing a peer, (c) are aware
of the performance gap but do not
know how to address it, or (d) think
that staff is competent and poor
quality is caused only by a lack of
resources.
With regard to the costs of supervi￾sion, the team found enormous
differences within the region. The
main cost elements are transporta￾tion and personnel. Thus, the
differences correlate with the
distances to health centers. The
managers’ ability to answer cost
questions impressed the team, a
probable result of the emphasis on
financial accountability by the health
sector reform.
D. Evaluation of
Problem-Solving Teams
A problem-solving team is two or
more people meeting to identify and
solve a quality problem by working
through a series of steps (the
problem-solving cycle) and using
simple QA methods. The evaluation
team investigated 25 such teams at
health centers in eight districts.
There were reports that out of 127
health centers in the eight districts
visited, 34 teams (27 percent)
formed between 1996 and 1998.
Among the 34 teams, 8 (23 percent)
had stopped meeting before the
evaluation. Among the 26 active
teams, eight (35 percent) did not
10 ■ The Zambia Quality Assurance Program (TRS)
finish their first cycle, and three had
completed more than two cycles.
The team is not sure that its findings
can be generalized, but if these
figures represent all of Zambia, there
may have been 150 problem-solving
teams between 1996 and 1998.
The evaluation team tried to deter￾mine why teams sometimes do not
form. In the districts visited approxi￾mately 60 people had received
problem-solving training, ranging
from one to 14 days. Afterwards,
many trainees still lacked the
motivation, confidence, and skills to
form a team and perform the cycle.
The team found that centers with
at least 10 staff were the most
successful in forming and lasting.
Other important factors in team
formation and success include:
(a) having regular coaching,
(b) having an officer-in-charge
trained and actively supporting the
team’s work, (c) having more than
three people on the team, (d) having
at least five days’ training repre￾sented on the team, and (e) having
reasonable morale and a culture of
professionalism.
Several factors were found to be
necessary for teams—once
formed—to successfully complete a
cycle: (a) a 50 percent retention rate
of team members, (b) meeting at
least monthly, (c) the team choice of
a problem that was not broad or
complex, (d) the ability to define a
problem statement, (e) not spending
over four months on one step, and
(f) following the cycle steps.
To finish a second cycle, it appears
that all the above factors had to exist
and the team had to have achieved
perceptible or measurable results.
For a team to follow the cycle steps
and use the methods correctly, all
these conditions would have to exist.
In addition, the everyday workload
of team members could not have
increased considerably for over two
months.
With regard to working on meaning￾ful problems, the evaluation team
assessed the list of possible
problems and how they were
prioritized by examining documenta￾tion or by asking team members how
they chose problems. For districts
not visited, they assessed
storybooks. Some teams did not
have documentation, making it
difficult to determine how well
they had followed the steps. The
relevance and importance of the
chosen problems for patients,
community, and staff varied. About
50 percent chose physical and
facility problems, and about 40
percent chose clinical problems.
The QA training influenced the
choice of the problems, which
typically included long waiting times,
the careless disposal of sharps, an
increase in malaria cases, low
patient-fee collection, congestion at
tea time, staff reporting late, and low
immunization rates.
Reducing waiting times was a
problem chosen by many teams,
probably because it was used as an
example in training. The clinical
problems (e.g., malaria, late antena￾tal booking) were relevant and
important for the health of patients
and/or the community. They were
usually chosen because staff
noticed a rise in cases, rather than
because they studied statistical
trends for the center. Approximately
5 percent of the problems were
identified by soliciting the views of
the users or neighborhood.
Following the Steps of the
Cycle and Choosing and Using
Methods
Most teams correctly used the QA
tools and methods, such as problem
prioritizing and cause-and-effect
diagrams. The use of methods for
listing data sources and for gather￾ing and interpreting data could have
been better and more frequent. The
most serious divergences were not
following the steps of the cycle and
failing to use the conclusions of one
step as the basis for the next. Teams
were better at following the steps
when the officer-in-charge had taken
coach training or where the team
had been a coach’s training site.
The evaluation team scored each
team on how well they followed each
step and used the method within it,
generating average scores for each
step for 25 centers. The team used
criteria to give a score of between 0
(did not do it) and 5 (could not
imagine how to do it better).7
Step 1: Identifying the Problem—
Score 3/Range 3–5
1) Listing—Score: 3 Most teams
scored fairly well on this step,
having considered a reasonable
number of problems (four to
eight). However, only four teams
consciously included problems of
concern to the NHC, and only two
conducted user surveys.
2) Prioritizing matrix and using
criteria—Score: 4 Most teams
used the matrix to prioritize
problems and used the prioritiz￾ing criteria reasonably well.
However, about 30 percent chose
problems that were not the
highest priority on their list. It
might be that the team rightly
chose a problem perceived as
7 Although the teams were trained to follow the five-step cycle, six steps are described here. To simplify the model, step 5 of the original
model is divided into two steps, identified here as steps 5 and 6.
The Zambia Quality Assurance Program (TRS) ■ 11
Case Study Number 4:
The Nakoli Health Center (Kabwe District)
Problem Solving
Nakoli health center is a semi-urban clinic with a staff of 15,
all women. It seemed cheerful and well organized to the
evaluation team. In June 1996, the sister in charge took
part in a local five-day training run by the regional link. The
training was on the quality problem-solving cycle and
teamwork, with one day on standard setting. The sister used
notes from the course to teach all health center staff for two
hours for 16 weeks.
From the start she decided that all the staff would take the
training, and she decided that the training meetings would
become team problem-solving meetings. They started on the
first problem in late June 1996, and by September 1998 had
worked on five problems and were starting on a sixth. They
worked on the first problem—a rise in malaria cases—for 15
months. The QA notebook documentation was not available
to the evaluation team for any of the problems because they
had only one copy, but the problems and steps were
documented in exercise books. When they first met, they
identified five problems: a rise in the incidence of malaria
(which they scored 25), upper respiratory tract infection (24),
diarrhea (23), eye infections (21), and skin diseases (17).
They used the criteria for scoring each problem correctly
and decided to work on the problem with the highest score:
malaria. They stated the problem as, “Nakoli health center
recorded increases in the number of malaria cases, with the
lowest figure in 1995 being 163, to 242 in 1996. A rise in new
cases has been noticed since 1995 and has resulted in
increasing morbidity. The improvement should result in
morbidity rates reduced to at least 100 new cases.” The
statement used measures, had a target, and chose to focus
on treatment rather than prevention because the team had
more influence over treatment than prevention.
In common with many other teams, Nakoli did not change the
membership of the team in step 3, “Identifying who needs to
work on the problem.” The complex step 4, “Analyzing and
studying the problem to identify major causes,” consumed
time and was where the team encountered problems. They
used the bubble chart to show possible causes of the
problem. They decided to focus on why people appeared to
be coming back to the clinic with symptoms after being seen
only a few days before. They did a high-level flow chart
of the treatment process and then identified data they
needed to find the main causes of “malaria morbidity.” They
interviewed 43 patients who came back within five days with
the same symptoms, and the data showed that most had not
completed their full course of chloroquine. They did not
question the data further to understand why, but went
straight to step 5, “Developing and choosing solutions.” Like
many other teams, they used brainstorming to list solutions
and then choose among them, and chose to ask patients to
come back every day for a supervised drug treatment. They
set up a monitoring system and set standards for return
treatments.
The team had established a treatment supervision system
that monitored the number of people monitored and the
number with symptoms. The data show that 80 percent in
June and 90 percent in July completed treatment, indicating
that the intervention did have some effect, and that 13
percent in June and 7 percent in July returned within five
days with same complaint.
While looking at solutions to the treatment problem, they
decided to broaden the work to include prevention and went
straight to brainstorming ways to reduce the incidence of
malaria cases. They worked with the community to find
stagnant pools and increase the use of impregnated bed
nets. The statistical data showed that the six-month mean
cases for January to June 1996 were 496 and for January
to June 1997, after the preventative efforts, the case rate
was 370. However, at the time of the evaluation, they had
discontinued monitoring the malaria rates, and the evaluation
team calculated that the mean case rates for January to
June 1998 was 726, a significant rise.
This team was unusual in continuing to work on other
problems: scabies (four months), low fee collection
(one month to solve), congestion at tea time (two weeks
without using the problem-solving cycle), staff reporting late
(one week without using the problem-solving cycle), and low
immunization rates. They were also unusual in deciding
appropriately when to use the problem-solving cycle and
when not to. These problems were congestion at clinic at
11:00 am because all staff were going for tea at the same
time (solution: some staff going for a 10:30 break and some
for an 11:00 break), and reporting late for work (solution:
reporting-in book). The evaluation team asked why they
had not stopped the quality work at the time of “de-linkage,”
like many other teams. They admitted that some staff was
unmotivated, but they had decided to start another
problem cycle to “show what women can do.”
12 ■ The Zambia Quality Assurance Program (TRS)
solvable, rather than one with a
higher score that was perceived
as too difficult. In some cases the
team chose a lower-priority
problem because it was more
important to members.
Four teams inappropriately
chose to work on more than one
problem simultaneously. Two
teams became stuck in later
steps and started on their second
problem before finishing the first.
Step 2: Problem Statement—
Score 2/Range 0–4
3) Method for writing a precise
statement with measures and a
specific target: Most of the
problem statements followed the
guidelines reasonably well,
although about 50 percent did
not quantify objectives.
Step 3: Identifying Who Needs to
Work on the Problem—Score 2/
Range 0–3
4) Method for selecting the right
people for the team or for
involving them in other ways:
The evaluation team found only
two teams that had changed their
roster after choosing a problem.
Most had involved others outside
the team in data gathering or
solution identification and
implementation.
Step 4: Analyzing and Studying the
Problem to Identify Major Causes—
Score 1/Range 0–3
This step is the most complex and
was the least well followed. Most
teams did not use the methods
correctly. In general, the cause-and￾effect listing was done well, but most
teams either did not gather data or
did not correctly decide what data to
gather. When they did correctly
decide which data to gather, they
did not use it to find the real cause.
Step 5: Developing and Choosing
Solution—Score 2/Range 0–3
This step first requires listing and
assessing possible solutions, which
most teams did, although not
systematically. It then requires
identifying the link between solutions
and the use of data, but most teams
failed to gather data that they could
have used to determine the real
cause. Last, it requires choosing a
solution. Most teams used objective
criteria to choose possible solutions
and did follow the methods taught in
training.
Step 6: Implementation and Evalua￾tion of Quality Improvement Work—
Score 2/Range 0–3
Most teams did not fully plan the
implementation of a solution, and
the evaluation team saw only five
records of the Plan-Develop-Check￾Act cycle used for this purpose. Few
teams measured indicators before
and after interventions. Many teams
did not systematically evaluate their
work. Last, the evaluation team
found no problem-solving team that
performed follow-up; however, a few
set standards and quality indicators
and monitored them routinely.
Difficulties the Teams Met in
Following the Steps and
Using the Methods
Teams experienced the most
difficulty collecting and using data to
identify root causes of problems.
Teams of four or more, in centers
with over 10 staff with an able coach
on the team (or regularly available)
and with reference and training
materials, did not encounter serious
problems. This was especially true if
a link facilitator visited. Other teams
experienced the most difficulties in
accessing a coach, having quality
reference materials, and using
certain methods. Many teams had
problems implementing solutions,
usually due to lack of resources.
The evaluation team asked team
members and coaches about
difficulties they had in using the
steps and methods. Of the teams
that started using the problem￾solving cycle, about half followed
each step in the correct sequence.
Of those that did not follow the
steps, most missed data gathering,
often because of difficulties creating
the data matrix and then gathering
data.
Teams frequently failed to use the
list of possible causes to decide
what data to collect. The most
common and serious linking over￾sight was the failure to use the data
interpretation to create a list of
solutions. For the few teams that
reached this stage, most would
brainstorm solutions without consid￾ering their data. Many may not have
realized that the purpose of data
collection is to confirm the real root
causes. Nearly all of these difficul￾ties could be overcome by frequent
visits by capable and trained
coaches.
Team Performance in
Gathering and Using Data
About 50 percent of the teams that
collected data encountered prob￾lems, making it difficult for them to
select a solution on the basis of
findings from their data. Most did
not have reference material on data
gathering. Those teams that tried
collecting data had difficulties
completing the data matrix. There
were problems in using routine
statistics, often because there was
no full-time set, or because census
data were inaccurate.
Not collecting data made it difficult
for teams to identify the real root
The Zambia Quality Assurance Program (TRS) ■ 13
causes, so objectively evaluating
their interventions was difficult for
the evaluation team.
Results and Progress
of the Teams
Five teams achieved measurable
changes in quality. About eight
teams reported that they had
achieved significant quality
improvements but did not have data
to demonstrate it. Around 60 percent
of the problem-solving cycles did
not produce results perceivable
within six months or that could be
attributed solely to the team’s
intervention. In some cases the
problems chosen could be only
partially solved by the team.
The evaluation team could not
document many results, but this
does not mean that the teams were
ineffective. Ancillary benefits
included team building, improving
skills to set action plans and
budgets, and gaining practical
management skills. The evaluation
also found:
■ Measurable improvements to
community health (e.g., lower
malaria incidence), clinical care
processes (e.g., increased
immunization), and patient quality
(e.g., reduced waiting times)
■ Reported improvements to facility
and professional quality
■ Adapting the problem-solving
methods for setting standards
and measuring compliance
■ Dissemination of quality concepts
beyond a team’s immediate area
and into the everyday work of
others
■ Increases in the competence and
confidence of staff, enabling
them to systematically approach
a wide array of problems
■ Providing an outlet for frustrated
and unrecognized staff potential
The main constraints to teamwork
included: (a) the loss of team
members and coaches due to
promotion and transfers and no
system to ensure sustainability;
(b) an inability to follow the methods
and steps, usually because of
inadequate training and/or coach
support; (c) an inability to use data,
quantify indicators, and follow up
after the solution intervention to
check the results and hold the
gains; (d) taking too long to com￾plete a cycle; (e) not developing the
skills to decide whether a problem
needs team problem solving; and (f)
teams’ failure to form, finish a cycle,
and continue on to other problems.
E. Evaluation of the QA
Support Systems
In the context of the Zambian health
sector reforms, QA activities are
being implemented within an
integrated framework for service
delivery. Training, supervision,
monitoring, and other systems that
support the delivery of essential
health services have a QA compo￾nent. The QA coaches and link
facilitators support these systems by
providing training and technical
assistance to the districts and health
centers. This section reviews QA
training, the QA coaches/link
facilitators network, documentation
and reporting of QA activities, and
supervision of quality assurance
activities.
Training in Quality Assurance
Training was given to key provincial
level staff in all provinces8 and then
at the district and facility levels in
three pilot districts. Training in the
target districts focused on setting
and monitoring standards and
problem solving. Training varied by
district, but generally consisted of
one day on the concepts of quality
assurance, a week on DySSSy, five
days on the development of monitor￾ing indicators, and 14 days on QA
tools and techniques. Many staff
have had training, but they are
unevenly distributed.
QA Training Capacity and
Experience at Various Levels
of the System
At the central level, the Directorate
of Monitoring and Evaluation is
responsible for ensuring that staff
from all levels of the system are
sensitized to the concept of QA. This
unit provides training and technical
oversight in QA to the DHMTs and
their health centers.
At the regional level, no special
training has been given to staff since
the formation of the regional direc￾torates. The CBoH had intended to
provide training to the regional
office, but circumstances beyond its
control prevented this. Though
training appears to be a function of
the regional office, the position of
training specialist was unfilled.
Regional staff reportedly would plan
for training in 1999.
At the district level, staff received
varying degrees of training. Though
few directors, if any, were formally
trained, several DHMT staff became
QA coaches or facilitators or
otherwise participated in some form
of QA training.
8 QA concepts and methods were introduced in some form throughout all provinces before the evaluation; only North Western Province
had not received the problem-solving training.
14 ■ The Zambia Quality Assurance Program (TRS)
In one instance, administrative staff
were given specific training in
standards setting in order to improve
record keeping. Lusaka staff
reported receiving some introduction
to QA as part of general district￾level, capacity-building training.
Capacity-building sessions were
conducted to create effective
leadership, accountability (including
financial management and QA
systems), and community partner￾ship. When asked about the types of
QA training received, only the
Lusaka DHMT noted that QA was
introduced during the capacity￾building training.
There is no formal system for
identifying QA training needs. At
the district office, responsibility for
coordination of training activities
varied according to the composition
of the DHMT. Though each district is
supposed to form a staff develop￾ment committee to identify training
needs, reportedly few have been
formed. It is unclear how training
needs in general would be identi￾fied, though QA would presumably
be given some priority. Identifying
training needs is further complicated
by staff turnover that results in losing
someone who has had QA training
and replacing that person with
someone who has not.
Concern for quality at the district
offices is such that QA training
activities were budgeted in 1998
action plans, but some were
dropped due to funding problems
and/or the suspension of workshops.
Most interviewed district teams
hoped to include QA in their 1999
plans.
At the health center level, the extent
of QA training varied by district,
ranging from one to seven days and
from one staff person to several.
Training needs at this level are
reportedly determined through
supervision and on-the-job
performance, but it seems that no
uniform approach to training was
adopted at this level. Interviewed
health center staff recommended
refresher QA training.
Main Constraints to QA Training
Funding and limited link facilitators’
time are the main factors limiting
QA training. Other factors are:
(a) frequent and recurring shortages
of drugs and other resources that
take priority over training, (b)
inadequate transportation, (c) staff
turnover, and (d) the need for a QA
training approach appropriate for
health centers with too few staff and/
or at more remote locations.
Coaches and Facilitators
A typical QA training sequence for
coaches and link facilitators is
Phase I: five days of basic QA
concepts, indicator development,
and basic standards setting; Phase
II: five days of DySSSy; and Phase
III: 11 days of problem solving and
coaching, including practice.
To strengthen linkages between the
regions, districts, and centers, the
“best” training participants are
chosen to act as link facilitators and
coaches after Phase II training. After
Phase III problem-solving training,
they are expected to develop a roll￾out strategy for QA in their districts,
including the training and coaching
of problem-solving teams. After the
initial training, participants undergo
“validation” where facilitators train
staff in other districts or health
centers or coaches train two other
health centers. CBoH QA staff
evaluate and re-evaluate partici￾pants to gauge their willingness and
ability to serve at the next level.
Links and coaches reported that this
somewhat rigorous validation was
effective in ensuring adequate QA
technical skills.
Roles and Constraints
The coaches/links play an important
role in the success of the QA teams.
Coaches visit the health centers to
provide just-in-time training or
technical guidance to the teams.
Coaches should take this opportu￾nity to review the team’s work,
correct or rework it if needed, and
plan the next steps. However, most
coaching visits were not regularly
performed. Some teams reported
never having been visited by a
coach after initial training; others
reported being visited only once or
twice. Others reported quarterly or
more frequent contacts by coaches/
facilitators as part of the integrated
supervisory DHMT visit. It remained
unclear how often coaching visits
coincided with the team meetings
and how helpful the visits were.
Coaches and links have many
constraints in providing support
to teams, but most see QA as
important and necessary. However,
several could not provide adequate
support to the teams because of
shortages of time, funding, or
transportation.
Facilitators’ suggestions for improv￾ing the support to QA coaches
include: (a) regular meetings to
improve QA skills, (b) having
districts sponsor their coaches, (c)
integrating QA meetings with HMIS
quarterly meetings (if held) at the
district level, (d) integrating QA
meetings with managerial skills
workshops (if held), (e) performing
QA activities during supervisory
visits, (f) encouraging regions to
support QA links in supporting
The Zambia Quality Assurance Program (TRS) ■ 15
Case Study Number 5:
Supporting QA Teams:
The Work of a Link Facilitator
The link facilitator (“link”) has played an integral role in the
development and success of the Zambia QAP. To strengthen
communications and teamwork between the QA structures at
the central, provincial, district, and facility levels, QA
coaching participants who demonstrated the best aptitude
and enthusiasm were selected to be links. These health
staff were expected to develop a strategy for QA in their
respective districts that would include the sensitization of
district-level staff as well as the coaching of problem-solving
teams at the health centers. Despite the many difficulties
faced as a result of the health sector reforms (i.e., the
de-linkage of staff, recurrent drug shortages, and severe
funding constraints), several links were undeterred in their
efforts to implement QA activities. In Kitwe District, for
example, a link not only managed to create and support
several QA teams, but has also infused QA concepts into all
aspects of his work as well as that of his many colleagues.
He experienced the following training regimen:
Phase I: Five days’ sensitization, including basic
QA concepts, indicator development, and
standard setting
Phase II: Five days’ standards setting (DySSSy)
Phase III: 11 days’ problem solving and coaching,
including practice
As a “best” participant, he was chosen to be a link after the
Phase II training and invited back for Phase III problem￾solving training. After that training, he underwent a series of
validation checks where he was observed and evaluated
while training staff in other locations. He joined other link
facilitators at three different provincial sites (Luapula,
Copperbelt, and Eastern Provinces) to train selected
individuals from the DHMTs and health centers. In Kitwe, he
worked primarily as a Clinical Officer at a health center, but
also served as health centers’ representative on the Kitwe
DHMT. He was well regarded within many professional
circles and thus well placed to introduce QA into district as
well as facility organizational structures. Within Kitwe, he
provided QA training to nine clinics, including eight that were
reported to have active teams. Some examples of the
problems he coached them to address were:
Table 3
Sample Link Facilitator Results
Location Topic Actions Taken
Chemwemwe Long waiting time in OPD Flowchart, patient￾health center flow analysis
Itimpi maternity Low delivery rates at facility Redefine problem,
confirm causes
Kamfinsa clinic Low delivery rates at facility Compare antenatal
visits and deliveries
Besides his work with the teams, caring for patients at the
clinic, and activities with the DHMT, the link was also working
to create other supportive mechanisms to ensure that QA
activities (and quality of care, generally) could be effectively
monitored. He provided training to IMCI supervisors who
were required to monitor (and be monitored by) other IMCI
health centers. Staffs were given a three-day basic training
in QA and later instructed to check the teams’ storybooks
during supervisory visits. Elsewhere, he worked to develop
a supervision checklist that could be used by staff charged
with monitoring health centers as part of “weekend cover￾age” visits. Despite the constraints on his time and the
economic and other extreme difficulties being faced by the
district, the link has made an admirable start with regard to
QA. Although it remains to be seen what the ultimate out￾come of this work will be, at the very least, the enthusiasm
and support for QA are ensured under vigil of the Kitwe link
facilitator.
coaches, and (g) including link and
coaching activities in action plans.
Skills transfer and coaching support
seem to depend on: (a) the ability
and willingness of district staff to
plan for QA, (b) the resources
committed to training and coaching,
and (c) the motivation of coaches
and facilitators.
Supervision and Skill Maintenance
Supervision of the link facilitators by
the CBoH is reportedly done through
training activities, visits to the
district, feedback after observations,
16 ■ The Zambia Quality Assurance Program (TRS)
and quarterly meetings for link
facilitators. The manner and fre￾quency of supervising coaches
differ from place to place. Some
links are quite active in contacting
coaches. Coaches can also be
monitored as part of their meetings.
Coaching skills are maintained
through “learning by doing,”
validation courses in other districts,
quarterly meetings of link facilitators,
and random checks of coaches and
teams by central staff. In Kabwe, the
link facilitator gives coaches a test
before their quarterly meeting to
detect problems that should be
discussed.
Reporting/Feedback
Mechanisms
Information on QA activities is
unevenly shared among different
levels of the health system. At the
regional level, specific knowledge
regarding the work of the teams,
coaches, and facilitators is limited.
Reports of QA activities are not
submitted to the regional office,
and existing reporting forms do not
capture information on the QA
teams.
At the district level, a summary
report of health center activities is
completed by the link facilitator and
forwarded to the QA staff at the
CBoH. The reports list the active
teams and describe the status and
results of their efforts. Quarterly
meetings of all link facilitators are
held to disseminate results and
share experiences. The CBoH uses
the results of these meetings to
identify opportunities for improve￾ment and track team activity.
Health centers do not formally
exchange information about QA.
The storybook is recommended to
assure efficient and complete
documentation of the team activities,
but many are incomplete or irregu￾larly updated. Moreover, storybooks
were in short supply.
Communities learn of quality
improvements at their health centers
through their NHCs. Posters or
storyboards are sometimes dis￾played at health centers to commu￾nicate commitment to quality, call
attention to the need for similar
improvements in other departments,
and advocate for QA throughout the
health center and in the community.
Supervision of QA Activities
Zambia has called for a more
empowering approach to improving
performance than can be provided
through traditional, authoritarian
supervision, and much work has
been done to improve supervision.
Formal supervision occurs as a team
activity with the goal of assessing
performance and ensuring collective
responsibility for resolving problems.
Guidelines for supportive supervi￾sory visits have been integrated into
the HMIS, and staff know that
supervision can be strengthened,
not just by increasing the number of
supervisory contacts, but also by
improving the focus and quality of
the visits.
Supervision by the Regional and
District Offices
At the regional level, the quarterly
performance audit is the main
mechanism for supervising the
DHMT and certain health centers.
Discussions with regional staff
convinced the evaluation team that
the regions can influence quality of
care at the health centers by
including DHMT clinical staff in the
performance audit visit, and then
delegating responsibility for solving
problems to district-level staff.
Supervisory staff reported that the
focus of visits often goes beyond the
performance audit to technical
areas. For example, the clinical care
specialist might look more closely at
maternal health services and thus
help to identify opportunities for
problem solving. Since many quality
indicators (such as “patient satisfac￾tion”) are in the hospital subsection
of the audit form, the audit could
similarly contribute to QA activities.
Health centers that reported receiv￾ing formal visits from the regional
office generally felt it was done
infrequently and was largely non￾technical (though one performance
audit report listed a series of
“clinical problems”). In addition to
the performance audit, regional staff
reported unplanned or informal visits
at health centers, which could also
help in addressing quality problems.
Though districts reported having
performed supervision visits, health
centers often reported less frequent
visits. Formal, integrated team visits
of health centers to monitor and
otherwise support QA activities do
occur if one or more DHMT staff
have had QA training. In general,
health center staff appreciate the
need for frequent, supportive visits
from their DHMT; many want more,
especially to help with technical
issues.
Supervisory visits facilitate routine
monitoring of service quality. Six of
seven DHMTs interviewed said all
health centers were visited every
quarter. If the IMCI checklist was
used during the visit, Outpatient
Department (OPD) cards were
reportedly reviewed to check for
treatment, diagnosis, and direct
observation of patient care (the last
seemed not to be done routinely).
These visits contributed to the QA
teams’ work, helping identify
problems, such as clients’ bypass￾ing the health center to seek care at
The Zambia Quality Assurance Program (TRS) ■ 17
the hospital and problems with
user-fee collection.
Most feedback from the DHMT visit
was reportedly given immediately,
so individual clinical problems could
be corrected quickly. In Kitwe,
problems found at all health centers
were discussed at meetings with all
in-charges. Sometimes, feedback
would be given during a general
meeting at the end of the formal
DHMT visit and would include all
clinical staff and often NHC
members.
Other planned and unplanned visits
to health centers occurred between
formal quarterly visits; these interim
visits often addressed problems
identified during the quarterly visit.
District staff reported that they felt
such visits were beneficial. In Kitwe,
supervisors reportedly observed
patient flow and checked patient
records. In Kalulushi, supervisors
identified and discussed the need to
correct and monitor the use of a new
chloroquine regime.
Apart from the supervision given
primarily by the director through
daily interaction with other DHMT
staff, internal supervision at the
district office is done through the
HMIS self-assessment form. This
form was encountered in only one
district, though it was said that the
data generated had been used to
“address problems such as malaria
treatment.” The region was also said
to have reviewed and discussed the
form during the performance audit;
however, this was not confirmed.
Supervision within and between
Health Centers
Within the health centers, routine
supervision by the responsible staff
in various units was found to
contribute to team problem-solving
activities. In larger urban health
centers, in-charges routinely
supervise various departments. One
maternal and child health supervisor
identified the problem of late
antenatal visit, which was resolved
through a problem-solving cycle. At
another clinic, supervisors report￾edly checked OPD cards and
medical records routinely and
discovered low monthly postnatal
attendance, also resolved through
problem solving.
Supervision between health centers
occurs when IMCI-trained staff from
one health center monitor and are
monitored by another IMCI health
center. Weekend supervisory visits
by a team of health center staff also
occur, reportedly contributing to
problem-solving activities.
F. Other Structures
Neighborhood Health
Committees
A 1997 CHESSORE survey de￾scribes the participation of the
NHCs in improving health services.
Most such activities relate to the
rehabilitation of health centers,
information and mobilization of the
community, expression of community
dissatisfaction with health services,
and community surveys.
Since they represent the communi￾ties and express patients’ perspec￾tives on quality of care, the NHCs
should be involved in some QA
activities (e.g., problem solving).
However, it is not clear how much
they can contribute without formal
QA training. Some problem-solving
teams have included trained
members of NHCs in their work.
The evaluation team found anec￾dotal evidence of public involve￾ment. In Kabwe patients complained
about the shortage of drugs. The
NHC and clinic staff determined that
many patients were coming from
outside the clinic’s catchment area
and received NHC help in resolving
the problem. Problems discussed in
a meeting with the NHC, health
center staff, and DHMT included
long waiting times, availability of
drugs, nonsupportive attitudes of
supervisors and staff, and clinic
maintenance.
The District Health Boards
Some health center committees
were involved in the development of
the 1999 district action plan by a
district health board (DHB). The
team met with the Kabwe DHB to
discuss its role in improving quality
of care. Most board members are
not health professionals, but they
influence the quality of care through
decision making related to budgets,
staffing, and training. They partici￾pate in technical meetings with the
DHMT and facility inspections. The
DHB members noted that they
helped build an extension for
deliveries upon learning of an area
with a high rate of pregnancy-related
deaths resulting from home deliver￾ies. They also helped distribute the
patients’ rights booklet.
The Private Sector
The Churches Medical Association
of Zambia (CMAZ) coordinates
activities between different church￾owned health facilities (31 hospitals,
60 health centers), trains managers,
helps its members secure financing,
and liaises between these services
and the government. Most mission
health facilities have mission doctors
and nurses, though they are mainly
staffed by church-selected govern￾ment employees.
CMAZ confirmed the evaluation
team’s findings that many of its
members participated in the QA
18 ■ The Zambia Quality Assurance Program (TRS)
training and that districts usually
include mission health centers in
their quarterly supervisory visits.
The team was told that QA training
by the association is a three-hour
session on a regular management
course. CMAZ was interested in
taking a full part in national and
district training and quality activities,
but emphasized its limited re￾sources. The association will shortly
serve as an agent for USAID to
channel funds for specific health
programs to districts and evaluate
the results, which will include a
quality evaluation component.
The Faculty of Private Practitioners
has 300 registered members,
medical officers who see about 1 to
2 percent of outpatients in Zambia.
Although the faculty has no official
QA policy, its members are required
to attend clinical meetings
(continuing medical education). The
association does not monitor or
evaluate the quality of clinical care.
The Regulatory Bodies
The General Nursing Council
sets, monitors, and evaluates
performance standards for nursing/
midwifery education, clinical
practice, management, and
research. It: (a) inspects health
institutions and schools of nursing
and midwifery; (b) develops curricu￾lums for nursing and midwifery
training; (c) sets entry standards for
nursing and midwifery basic
training; (d) certifies qualified nurses
and midwives; (e) approves training
institutions for nursing and mid￾wifery; and (f) disciplines students,
nurses, and midwives. The GNC
inspection guidelines focus on the
structure of the facilities, standard
operating procedures, and policies.
Most of the judgement criteria are
not explicit.
Monitoring the quality of care is a
problem because of financial and
material constraints. The GNC can:
(a) meet with management; (b)
check for compliance with medical,
nursing, and administrative stan￾dards; (c) observe clinical care;
(d) interview clients and staff;
(e) inspect nursing and medical
records; (f) note whether resources
(equipment and time) are available
for providing nursing care; (g)
ensure valid licenses; and (h)
correct malpractice on the spot.
The Medical Council of Zambia
inspects private hospitals and
clinics, using an inspection certifi￾cate that focuses exclusively on the
structural aspects and not on the
assessment of the care. It also
approves the curriculums for certain
paramedical training institutions.
The Patients
Several surveys informed the
evaluation team about the patients’
perspectives. Patients believe the
most important features are the
availability of drugs, the politeness
of staff, a short waiting time, and a
physical examination. One survey
revealed that only 38.7 percent of
patients believed they were properly
examined, while 67 percent thought
that staff had poor clinical skills.
Another looked at urban/rural
differences and found that in rural
areas the basic minimum level of
resources was lacking, especially
staff and drugs, and that in urban
areas, mismanagement and drug
shortages were the main problems.
The patients’ rights booklet, describ￾ing the level of quality that people
should expect, was distributed to all
hospitals, districts, health centers,
and consumer associations. How
well patients know their rights is
undocumented.
The Training Institutions
The University Teaching Hospital
(UTH) graduates about 60 registered
nurses a year. Tutors reported that
the nursing curriculum had not been
revised or reviewed since 1990.
Overall, new knowledge is imparted
to students informally and irregularly.
Tutors participate in programs such
as Safe Motherhood and IMCI, or go
to periodic training, and then
disseminate technical knowledge
from their notes and material in
these courses.
Another way new knowledge is
shared is through field practicums.
To acquire skills in delivery of
primary healthcare, nursing students
may be sent to the field and have
the costs paid by the nursing
school. (The district gains
temporary—but free and highly
trained—labor.)
Action plans for 1996 included
activities like internships, but they
were never implemented due to
funding problems. The GNC report￾edly monitors nursing standards
through their periodic evaluation of
hospitals and nursing school
accreditation activities.
Tutors were interested in including
QA in their management curriculum.
Since MOH or other specialists are
invited to speak as part of a two￾week management course given
separately from basic nurses
training, this might be a way to
introduce students to QA. The
expressed need, however, was to
integrate QA as part of preservice
and post-basic training curriculums.
The evaluation did not determine
whether the School of Medicine
teaches clinical quality standards to
medical and other post-qualification
students. QA did not seem to be
taught, but quality was beginning to
The Zambia Quality Assurance Program (TRS) ■ 19
be addressed by faculty in aca￾demic journals. Formal links do exist
between the School of Medicine and
the CBoH: the Dean is on the CBOH
Board. Increasing and improving the
links between the health system and
the school would help ensure that
students become familiar with the
standards of clinical care. Also, the
school could develop its own input
for formulating and revising care
standards and protocols to ensure
that they conform to the latest
research. The school expressed
interest in developing a Masters￾level program in Health Service
Quality and introducing QA
programs and faculty; this could
complement the CBoH QA training
and contribute to long-term QA
capability.
III. Recommendations for
Strengthening the QAP
This chapter presents recommenda￾tions, detailing the most important
ones and listing some options for
their implementation when appropri￾ate. The team notes that its long list
of recommendations results from its
methodology: no suggestion of
weaknesses in the QAP is intended,
but activities are suggested that
would take the Zambian QAP to the
next level of institutionalization.
A. General
Recommendations
The evaluation team feels that a
program as important as the
Zambian QAP justifies more frequent
reviews, perhaps every two years.
Also, regular monitoring of QA
activities and their effects would
help in collecting the documentation
necessary to continuously improve
the program. QA is more than tools
and methods. It is a spirit that comes
only from a mentality change, where
clients make demands and express
their expectations, where health
providers question their own
performance, and where managers
make client-oriented, data-based
decisions to constantly improve
healthcare systems. It cannot
happen overnight. It took five years
to initiate change and the emer￾gence of a quality culture in Zambia.
A national QA policy describing the
vision, strategy, and objectives of
Zambia’s QAP should be created.
This would facilitate an action plan
for implementing, monitoring, and
evaluating QA activities. The design
of such a policy should cover
actions by both CBoH and the
nongovernmental institutions that
have a role in QA. Such a policy
would help define the QA roles and
responsibilities of various stakehold￾ers in the health system.
The vertical integration of QA into all
levels of the health system should
be strengthened by creating or
reinforcing the links between the
QAP and:
■ The regions (or provinces), by
building QA capacity in the
regional team and adapting the
performance audit instruments;
this requires sensitizing and
training the regional teams
■ The districts, by building QA
capacity in the DHMT and
adapting the performance
monitoring instruments for
supervision; this means that
DHMTs should be at least
sensitized in QA and some
members trained
■ The hospitals, by bringing QA
into their management board and
services
The horizontal integration of QA into
other directorates of the CBoH
should be strengthened by creating
or reinforcing the links between the
QAP and:
■ The Directorate of Health
Services Commissioning,
which should provide technical
assistance to the hospital boards
and measure the impact of
accreditation on the quality of
inpatient care
■ The Directorate of Systems
Development, which should
coordinate the development of
clinical guidelines and job aids
■ The Information and Health
Systems Research Unit
(Directorate of Monitoring and
Evaluation), which should
develop operations research on
QA and quality of care, as well as
monitor the impact of the HMIS on
the work of the QA teams
The integration of QA within the
private health sector and parastatal
institutions should be strengthened
by creating or reinforcing the links
between the QAP and:
■ The regulatory bodies, which
should develop and communicate
standards and the quality
performance monitoring system.
These entities already inspect
training institutions and would
benefit from the QAP approach to
monitoring clinical performance.
■ The medical, nursing, and
paramedical training institutions,
which should help develop
standards and ensure their
inclusion in preservice and
post-basic curriculums
■ The private associations, which
should help develop and commu￾nicate standards and assess
quality in the private sector
20 ■ The Zambia Quality Assurance Program (TRS)
B.Standards of Care
Development of Standards
The development, adaptation, and
revision of clinical care standards
should be defined in an official
policy. This requires that one
structure coordinate the work of
technical experts, providers, and
users. The involvement of health
providers from all levels of the
system ensures that standards are
realistic and accepted by users,
while experts’ involvement guaran￾tees scientific validity. Evidence￾based medical literature is key in
developing standards.
These standards should be based
on research into, and an assessment
of, health providers’ needs for
guidance through job aids, which
have enormous potential to improve
quality. Research should determine
the factors that influence the use of
job aids.
The development/adaptation of
standards should be consistent with
the development of other materials,
including training materials and
instruments to measure compliance.
In other words, the development of
new standards should trigger
changes in the other mechanisms
used to communicate the standards,
such as preservice and in-service
training. Similarly, the criteria on
which performance is assessed and
the instruments used to assess
performance (supervision checklists,
etc.) should change.
Communication of Standards
A national strategy to communicate
standards that does not rely solely
on classroom training should be
developed. It should draw on
principles for behavior change that
would improve clinical practice and
should include a combination of
interventions based on behavioral
science. (This is a topic for research
since little is known about the factors
that cause health providers to adopt
standards.)
The QAP did not participate in
communicating the ITG, but could
have contributed significantly to this
effort, especially with regard to
developing job aids, writing
standards, and developing tools to
measure compliance. Medical
reference books are needed, and
the initiative to develop one consis￾tent with the essential package of
health services is relevant. However,
the ITG is only a starting point for
various activities to strengthen the
communications strategy for
standards.
Clinical guidelines should be
designed to reflect the type of job
aids health providers need and
would willingly use during their
practice.
The continuous reinforcement of
standards is key: when regularly
reminded of the standards, health
providers do better. Districts and
officers-in-charge should be
encouraged to use all opportunities
(supervisory visits and technical
meetings) to reinforce standards of
care and encourage their use.
Preservice training is often over￾looked in strategies to introduce new
standards, leading to a paradoxical
situation where young graduates
need retraining as soon as they start
work. Medical textbooks should be
used as reference material for
preservice curriculums, and nursing
and medical curriculums should
include up-to-date standards. These
practices would both ensure better
knowledge and reduce in-service
training costs.
Performance Monitoring
The assessment of quality
performance should include measur￾ing health workers’ compliance with
process standards through the
direct observation of the delivery of
care. Performance audits and
supervision visits provide opportuni￾ties for this, but the entire strategy of
these processes will have to be
reworked, from identifying the skills
needed and understanding the
constraints to redesigning the forms.
A specific strategy is needed to
explore poor performance and
determine whether it is caused by
lack of competency or something
else. In-service training is often
identified—sometimes erroneously—
as the solution to poor performance.
Problem-solving teams should
determine whether training or
problem solving is the better
approach for improving competency.
A formal recognition and reward
system based on quality perfor￾mance would create incentives to
improve quality. Rewards should be
based on their effectiveness and
acceptability to DHBs. The staff
appraisal system should be
redesigned to assure that quality
performance is linked with career
promotion and that the measurement
is objective and fair.
C. Quality Improvement
Activities
The evaluation team recommends
an investigation into the methods for
improving quality in health centers
with five staff or fewer. If the CBoH
wants problem-solving teams in
these health centers, it will be
necessary to identify the conditions
that would allow these teams to
succeed.
The Zambia Quality Assurance Program (TRS) ■ 21
Zambian healthcare providers
should continue using the problem￾solving model, but with simplifica￾tions and changes to training. The
team problem-solving method can
be effective in health centers with
more than 10 staff under certain
conditions. However, the last step of
the model should be divided into two
parts: “implementation” and “evalua￾tion.” Step four could be broken into
two steps: “initial problem analysis”
and “data gathering and root cause
location.” This would emphasize the
importance of data collection.
Each trainee should have a copy of
simple guidelines for team problem
solving and use a storybook. The
training materials and sessions
should recommend choosing a
simple problem for the first cycle,
involving users or NHCs, following
steps in succession and linking
them, and gathering and interpreting
data.
A simpler, modular five-day training
package for district training should
be developed (incorporating the
above simplifications) for those who
will not be coaches. Its developers
should draw on the experience of
links and coaches who have
designed five-day or 20-hour
courses.
Teams with little training should use
their first problem-solving cycle as a
training exercise. A coach should
attend each meeting to help the
team and to show good practices in
documentation, how to use the
methods, and how to link the steps.
Identification of the priority problems
could be done during the QA
training.
The evaluation system created by
the evaluating team should be used
to help teams learn how to be more
effective, assessing teams and
giving them feedback. Teams should
be taught how to use this system for
self-assessment and to guide a peer
review of the work of other teams.
Links and coaches should develop
skills to decide whether to use a
team problem-solving cycle. They
should then train and advise teams
on how to make this decision.
NHCs and healthcare consumers
should be involved in QA problem
solving, and patients should drive
the quality program. Pilot efforts
should be evaluated to generate
recommendations for increasing
NHC and client involvement.
All teams should document their
work in storybooks, and the docu￾mentation should be available, with
at least one copy at the health
facility. Coaches should use the
storybook to monitor team progress
when they cannot attend meetings.
Health centers should formally
exchange information about the
work of their teams at meetings of in￾charges and among health centers
that share a coach.
Other methods to improve quality for
different circumstances should be
investigated. While the approach
taken by CBoH has achieved a great
deal, different QA methods are
needed for different settings, such
as small facilities. The QA Unit
should become more familiar with
other methods and assess which
ones might be appropriate in
different facilities and for different
types of services. An assessment
should be made of the value of
training and problem-solving teams
compared to other quality improve￾ment methods (clinical training;
patient education and counseling
skills; on-the-job training; supervi￾sion of clinical skills; protocol and
guidelines development; standard
setting, indicator development, and
monitoring; and clinical case review
and clinical audit).
D. QA Support Systems
QA Training
CBoH should develop enough QA
trainers and experts at the national
level to provide adequate technical
support to all levels of the system.
Two trainers per regional or provin￾cial level team could suffice. The
objective is to have the capacity to
ensure responsive, consistent
training support.
A training needs assessment should
be done as part of the yearly action
plans. In reviewing these plans, the
CBoH or regional office should
ensure that QA training activities
have been adequately planned and
budgeted, and district plans should
include coaching or link facilitator
visits to health centers.
The DHMT should identify candi￾dates for QA training, in particular
staff at the health center level who
could become coaches or facilita￾tors. At least two people per health
center should be trained (including
the in-charge). Each district should
have trained coaches to cover as
many facilities as possible. All
professional staff at the regional and
district offices should take some QA
training. QA could also be incorpo￾rated into other management
training. Relevant professional
regulatory boards should take QA
training and incorporate QA
concepts into preservice and
post-basic training curriculums.
22 ■ The Zambia Quality Assurance Program (TRS)
Coaches and Link Facilitators
At least one staff on the DHMT
(i.e., a district-level coach or link
facilitator) should have responsibility
for QA activities in the district. This
person should participate in supervi￾sion visits to the health centers and
include QA in the integrated supervi￾sory approach.
Better coach support would improve
teams’ ability to use the QA methods
and data. Competence levels should
be improved in following and linking
the steps of the cycle. This involves
fully understanding why the steps
should be followed and knowing
when to diverge from them. Learning
by doing is the best way to reach
this kind of understanding.
QA training should further
strengthen capacity in QA planning,
effective communication and
teaching, and supportive supervi￾sion. Furthermore, support visits by
the central level to coaches and link
facilitators should be intensified
(at least early on) to further ensure
that training efforts actually do result
in QA activities.
District offices and coaches should
take advantage of existing opportu￾nities (e.g., meetings and work￾shops) to update their QA training
and share information on QA
activities and issues.
QA Documentation
and Feedback
Existing reporting systems should
include reports of QA activities;
timely feedback should be given to
the appropriate levels; and informa￾tion should be shared between and
within levels. QA staff at the CBoH
should determine what information
on QA activities would be useful and
how it should be reported.
QA staff should ensure complete
documentation (e.g., in storybooks)
of QA activities and effective use of
the documentation in order to
monitor the work of teams. Sufficient
storybooks should be available for
this.
Supervision of QA Activities
Regional offices should ensure that
DHMTs are providing adequate and
frequent supervision to the health
centers. Frequent review of achieve￾ments could evaluate supervision
problems (e.g., quarterly visits not
occurring, incomplete supervisory
reports).
Each district should plan and
budget for an adequate number of
supervisory visits and ensure that
they occur. Staff suggested that the
minimum number of visits would
include quarterly visits plus four
others per health center per year.
Visits should initiate and support QA
problem-solving activities, solicit
feedback, and advocate local
resolution of problems.
Those responsible for QA activities
at the DHMT should participate in
quarterly supervisory visits to health
centers. Many QA facilitators/
coaches were found to be DHMT
staff, which appeared to facilitate the
integration of QA into their technical
supervisory duties. Quarterly visits
could be followed by visits in
support of teams and their problem￾solving processes.
Regional staff want to improve their
interpersonal skills for interacting
with district and health center staff.
Some also requested observation
checklists for relevant technical
areas to help structure supervisory
visits.
E. Prioritizing the
Recommendations
The evaluation found that the lack of
QA expertise in most of the DHMTs
hampered the work of the facility
teams. When no DHMT member has
had QA training, no one at the
district level takes responsibility for
quality improvement in the district,
and no one coaches the teams. The
teams have a hard time implement￾ing QA methods, and they may not
have time to teach and coach
others. At least two DHMT members
per district (one being the District
Medical Officer) should be trained in
QA and take responsibility for QA
activities.
Despite its tremendous efforts and
remarkable success in covering the
whole country with a network of
coaches and trainers, the CBoH is
unable to sustain and expand the
QA program, because only two
people at the central level are in
charge of QA. While “quality is
everybody’s business,” if there are
insufficient people to do the job, it
cannot be done fully. The dedication
of the two QA staff is remarkable, but
the central QA team must be
strengthened.
A national QA policy would boost the
interest in QA and health sector
reform and enable progress on the
larger problems that hamper the
health system. Milestones covering
the next five years with a clear
strategy to institutionalize QA would
make it easier for the CBoH partners
to design their roles.
Currently, the focus of supervision is
not quality of care, and changing
supervisory processes could have a
tremendous impact on quality.
Supervision should focus on quality
monitoring and improvement through
The Zambia Quality Assurance Program (TRS) ■ 23
the definition of standards for
supervision. Supervisory visits
should facilitate the direct observa￾tion of the delivery of care, feedback
the results, and help teams solve
quality problems or improve
healthcare processes.
A capacity for QA training must be
built through a network of certified
QA trainers and the introduction of
QA into preservice curriculums. This
would build self-reliance and ensure
QA’s sustainability.
The evaluation team is puzzled by
the reluctance of supervisors to
observe the delivery of care and of
health workers to use job aids.
Research into these areas should be
undertaken as soon as possible.
Other research topics are: (a) ways
to increase the cost-effectiveness of
QA training; (b) standards develop￾ment and the best communication
strategy for improving clinical
practices; (c) workers’ perceptions
of the scientific validity, feasibility,
reliability, and clarity of the ITG
standards; and (d) factors that
influence the productivity of the
problem-solving teams.
IV. Conclusion
There is growing evidence of the
impact of QA methods on the quality
of care in resource-constrained
environments, where simple solu￾tions such as re-training of staff or
the supply of additional resources
have failed. The richness of the QA
experience in Zambia provides
lessons that will benefit not only
health sector reform there, but also
other QA programs. While much
remains to be done, the enormous
number of discrete changes result￾ing from the work of the QA Unit and
the Directorate of Monitoring and
Evaluation has had a tremendous
impact on the way patients interact
with their health system and demand
quality. Health providers’ behavior is
also changing under the positive
influence of the attention districts
are paying to their work and the
empowerment mechanisms that
have been established through
decentralization.
The key lessons learned from
Zambia can be summarized as
follows:
■ Support systems, such as
coaching and QA training
capacity, must be well estab￾lished at the district level in order
to assist the teams. In a decen￾tralized system, the QA program
should first target the districts so
that team ownership for QA
activities will develop.
■ Numerous factors influence the
productivity of the problem￾solving teams. The tools devel￾oped for this evaluation proved
useful, and coaches could use
similar materials to assist teams.
■ The health workers’ resistance to
job aids that would improve their
compliance with standards and
supervisors’ reluctance to directly
observe the delivery of care are
barriers to better healthcare
quality. More research would
determine the causes of these
behaviors and assist in their
improvement.
■ A detailed documentation system
of the QA program would help in
monitoring the QAP’s impact and
making adaptations.
■ Such a program should be
regularly evaluated (every two
years).