QUALITY 
ASSURANCE 
PROJECT
Center for Human Services • 7200 Wisconsin Avenue, Suite 600 • Bethesda, MD 20814-4811 • USA
EVALUATION REPORT
The Zambia Quality Assurance
Program Final Evaluation
Bruno Bouchet, MD, MPH
Chimshimba Lumbwe, MD
John Øvretveit, PhD
Eleanor Msidi
Maria Francisco
Maryann Lyby
Elizabeth Goodrich
Quality Assurance Project
7200 Wisconsin Avenue, Suite 600
Bethesda, MD 20814-4811
301-654-8338, FAX 301-941-8427
www.qaproject.org
The Zambia
Quality Assurance Program
Final Evaluation
© Center for Human Services (CHS), 2000
Acknowledgments
The authors are indebted to the Central Board of
Health (CBoH) staff for their assistance throughout the
mission, both at a technical level and for secretarial
assistance, physical facilities, transportation, and equip￾ment made available throughout the mission. Without
the efforts of the CBoH, the logistics would have un￾doubtedly been more challenging, and the output less
likely in the time available. Senior officials’ willingness
to make themselves available at all times is recognized
with considerable gratitude. The atmosphere of collabo￾ration that prevailed throughout the mission illustrates
well the partnership spirit in which it was designed.
The evaluation team acknowledges the support
provided by the USAID mission in Lusaka in funding
this evaluation and thanks the Agency for providing the
Quality Assurance Project with the opportunity to
contribute to the success of the Health Sector Reform in
Zambia. The evaluation team’s work was supported by
USAID under Contract No. HRN-C-00-96-90013 to the
Center for Human Services.
The team felt privileged to be associated with such an
important undertaking and would like to give its special
thanks to Dr. Eddie Limbambala, Dr. Mwendaweli
Maboshe, Ms. Joyce Tembo, and Mr. Albert Mulungu
for facilitating the mission. We are also grateful for
Mr. Augrey Mwenda, CBoH, and Ms. Pay Sayer,
URC-CHS, for their essential contributions.
Foreword
An evaluation of this nature could be threatening to
all stakeholders of the Quality Assurance Program in
Zambia, although the findings should relieve part of the
stress. The intentions of the evaluation team were to
draw lessons that could benefit both Zambia and other
countries, as objectively as possible. In doing so,
achievements are recognized, and remaining challenges
are identified. Successes are praised, whereas failures
and shortcomings are addressed through constructive
recommendations. This evaluation report is intended to
motivate all those who continue to work for a higher
level of quality of care and health services in Zambia.
Despite its efforts to validate findings and their
interpretation, the evaluation team is aware of the risk
of reporting inaccurate data and missing other parts of
relevant information. The evaluation team takes full
responsibility for such omissions and inaccuracies; the
views expressed in this report reflect the opinion of the
evaluation team only and no other institution.
Recommended citation
Bouchet, B., C. Lumbwe, J. Øvretveit, E. Msidi, M.
Francisco, M. Lyby, and E. Goodrich. 2000. The
Zambia Quality Assurance Project: Final evaluation.
Evaluation Report 1(2). Published for the United States
Agency for International Development (USAID) by the
Quality Assurance Project (QAP): Bethesda, Maryland.
About this series
The Evaluation Report series presents the findings,
recommendations, and lessons learned of completed
quality assurance (QA) evaluations in countries with
long-term QA programs. An electronic copy of this
publication may be found at qapdissem@urc-chs.com.
Table of Contents
List of Tables and Figures .............................................................................................................................................. iv
List of Appendices.......................................................................................................................................................... v
Abbreviations and Acronyms......................................................................................................................................... vi
Abstract ....................................................................................................................................................................... vii
Executive Summary ....................................................................................................................................................... ix
A. Introduction ..................................................................................................................................................... ix
B. Findings and Recommendations: General ........................................................................................................ ix
C. Findings and Recommendations: Development of Standards ........................................................................... ix
D. Findings and Recommendations: Communication of Standards ........................................................................ x
E. Findings and Recommendations: Assessment of Quality Performance .............................................................. x
F. Findings and Recommendations: Problem-Solving Teams ............................................................................... xi
G. Findings and Recommendations: QAP Support Systems................................................................................. xii
I. Introduction ............................................................................................................................................................. 1
A. Purpose of the Evaluation ................................................................................................................................. 1
B. Methodology .................................................................................................................................................... 1
C. Structure of the Report ..................................................................................................................................... 2
II. Background: Quality Assurance in Zambia .............................................................................................................. 2
A. The Relevance of QA for Zambia ..................................................................................................................... 2
B. Zambia’s QA Activities .................................................................................................................................... 3
III. Achievements and Challenges ................................................................................................................................. 4
A. Standards of Quality for Zambia’s Healthcare System....................................................................................... 4
B. Evaluation of Problem-Solving Teams............................................................................................................ 17
C. Evaluation of the QA Support Systems ........................................................................................................... 25
D. The Role of Other Structures and Sectors in QA Activities ............................................................................. 32
IV. Detailed Recommendations for Strengthening QAP ............................................................................................... 34
A. General Recommendations ............................................................................................................................. 34
B. Recommendations on Standards of Care ......................................................................................................... 35
C. Recommendations on Performance Monitoring .............................................................................................. 36
D. Recommendations on QA Activities ............................................................................................................... 36
E. Recommendations Regarding the QA Support Systems .................................................................................. 37
F. Prioritizing the Recommendations .................................................................................................................. 38
V. Conclusion ............................................................................................................................................................ 39
Work Cited ................................................................................................................................................................... 40
iv
List of Tables and Figures
Table 1–1: Summary of the Main Recommendations of the Evaluation Team............................................................ xv
Figure 1–1: CBoH Organization Chart ......................................................................................................................... 3
Table 4–1: Changes in Provider Performance with Training and with Time ............................................................... 10
Table 4–2: Changes in Provider Performance with Clinical Supervision .................................................................... 10
Table 4–3: Features of Supervision Checklists .......................................................................................................... 12
Table 4–4: Types of Data Collected on Health Center Performance ........................................................................... 13
Table 4–5: Changes in Health Services Performance (Percentage) ............................................................................ 15
Table 4–6: Costs of Performance Monitoring for Regions and Districts..................................................................... 16
Table 4–7: Number of Teams Formed in Districts Visited and Number of Problem-Solving Cycles Completed ......... 17
Table 4–8: Examples of DHMT Training in Quality Assurance ................................................................................. 26
Table 4–9: Sample Link Facilitator Results ............................................................................................................... 27
Table 4–10: Types and Frequency of Technical Supervisory Visits.............................................................................. 31
v
List of Appendices
Appendix A: Selected Sections of the Scope of Work and Terms of Reference .......................................................... A-1
Appendix B: Protocols Developed by the Kabwe District Health Office .................................................................... A-4
Appendix C: Examples of Supervision Checklists Used by Districts ......................................................................... A-5
Appendix D: Guides for Interviews Conducted by the Evaluation Team .................................................................... A-9
Appendix E: Achievements of the Problem-Solving Teams..................................................................................... A-35
Appendix F: People Met during the Mission ........................................................................................................... A-47
Appendix G: Systems View of a Quality Assurance Program .................................................................................. A-53
vi
Abbreviations and Acronyms
Acronym Abbreviation
BASICS Basic Support for Institutionalizing Child Survival
CBoH Central Board of Health
CHESSORE Centre for Health, Science & Social Research
CHS Center for Human Services
CMAZ Churches Medical Association of Zambia
DHB District Health Board
DHMT District Health Management Team
DHO District Health Office
DySSSy Dynamic Standards Setting System
EPI Extended Program on Immunization
FAMS Financial Accountability Management System
FPP Faculty of Private Practitioners
GNC General Nursing Council
HC Health Center
HIV Human Immunodeficiency Virus
HMIS Health Management Information System
HRIT Health Reform Implementation Team
IMCI Integrated Management of Childhood Illnesses
ITG Integrated Technical Guidelines for Frontline Health Workers
MCH Maternal and Child Health
MOH Ministry of Health
vii
Acronym Abbreviation
NGO Nongovernmental Organization
NHC Neighborhood Health Committee
OPD Out-Patient Department
PDCA Plan, Do, Check, Act
PHN Population Health and Nutrition
PHP Public Health Practitioners
QA Quality Assurance
QAP Quality Assurance Program
QI Quality Improvement
RHC Rural Health Center
RN Registered Nurse
RO Regional Office
STD Sexually Transmitted Disease
TBA Traditional Birth Attendant
TDRC Tropical Diseases Research Centre
URC University Research Co., LLC
USAID United States Agency for International Development
UTH University Teaching Hospital
WHO World Health Organization
ZHAC Zambia Health Accreditation Council
ZMA Zambia Medical Association
viii
Abstract
This document reports on an evaluation of the Quality
Assurance Program (QAP) in Zambia. The evaluation
took place in the fall of 1998, during the QAP’s fifth
year.
The QAP’s strategy was to build quality assurance
(QA) capacity at the district and health center levels by
training health providers and their managers in setting
standards for health services, monitoring indicators of
achievement, and team-based problem solving.
Problem-solving teams are expected to identify quality
of care problems from both users’ and providers’
perspectives, document the root causes, design and
implement solutions, and use indicators to monitor
progress in solving problems. In addition, a network of
coaches and link facilitators was created to support and
guide the teamwork and monitor performance and
achievements.
This report presents the findings, lessons learned, and
recommendations developed by the evaluation team.
Key recommendations relate to the importance of
communicating standards in a way that will be accepted
by users, ensuring that standards are consistent in all
communications (training, job aids, etc.), improving
performance by having supervisors observe staff while
they are providing healthcare, and addressing the failure
of teams to form or remain active in small health
centers. Important successes detailed in the report
relate to significant improvements to healthcare quality
where teams formed successfully and the development
of teamwork. Several recommendations are made for
further research that would help improve the quality of
healthcare in Zambia and beyond.
Among the evaluation team’s most important findings
are:
■ Support systems, such as coaching and QA training
capacity, must be well established at the district level
to ensure the continuity of the teams
■ In a decentralized system, the QA program should
first target the districts so that team ownership for
QA activities will develop
■ Numerous factors influence the productivity of the
problem-solving teams. The tools developed for this
evaluation (difficulty index and team failure index)
proved useful and could be used by coaches to assist
teams
■ A detailed documentation system of the QA program
would help in monitoring the QAP’s impact and
making adaptations
ix
Executive Summary
A. Introduction
Zambia began an important reform of its healthcare
sector in 1993, including a Quality Assurance Program
(QAP) to improve the quality of healthcare. The QAP
strategy was to build quality assurance (QA) capacity at
the district and health center levels by training staff in:
(a) setting standards for health services, (b) monitoring
indicators of achievement, and (c) team-based problem
solving. A network of coaches (to assist QA teams at
health centers) and link facilitators (QA trainers who
provide support to coaches and are in charge of several
districts) now covers almost the entire country. QA
teams are expected to identify quality of care problems
from both users’ and providers’ perspectives, document
the root causes, design and implement solutions, and
use indicators to monitor progress in solving problems.
Prior to 1996, the Zambian QAP was the responsibility
of the QA Unit of the Health Reform Implementation
Team (HRIT) of the Ministry of Health (MOH). Since
the creation of the Central Board of Health (CBoH)
in late 1996, QA activities are coordinated by the
Directorate of Monitoring and Evaluation, through the
Service Quality and Performance Audit unit, and this
constitutes the QAP.
Zambia’s QAP was evaluated to identify lessons and
challenges and to make recommendations that would
shape the vision and strategy of its next phase. A team
of three international and three local experts was
assembled and worked in three subteams of two to
evaluate the QAP in the following areas: (a) develop￾ment and communication of standards, and quality
performance monitoring, (b) the work of the QA teams,
and (c) QAP support systems.
This chapter summarizes the main findings and related
recommendations.
B. Findings and Recommendations:
General
1. General Findings
The amount and quality of the work of the QAP is
impressive. In only five years, a small team of senior
staff built a QA structure and capacity throughout the
country, generated enthusiasm for QA, and initiated
teamwork on quality of care issues by healthcare
providers.
The evaluation team found no official document
describing the vision, strategy, and objectives of
Zambia’s QAP, so it was unable to evaluate the QAP
against government policy and objectives.
Although some attempts have been made to link with
other health institutions, units, and stakeholders, the
QAP remains isolated in its efforts to improve quality
of care through a QA methodology.
2. General Recommendations
Regarding policy issues, a national QA policy that
includes the private sector should be developed, pro￾viding the vision, strategy, and objectives of the QAP.
Regarding strategic issues, the evaluation team
believes: (a) The vertical integration of QA into all
levels of the health system must be strengthened by
creating or reinforcing the links between the QAP
and the regions, districts, and healthcare facilities,
(b) The horizontal integration of QA at the CBoH level
must be strengthened by creating or reinforcing the
links between the QAP and the various directorates
and units of the CBoH, and (c) The integration of QA
within the private sector and parastatal institutions
must be strengthened by creating or reinforcing the
links between the QAP and the regulatory bodies, the
training institutions, and the private associations.
C. Findings and Recommendations:
Development of Standards
1. Findings on the Development of Standards
Developing the “Integrated Technical Guidelines for
Frontline Health Workers” (ITG) was an excellent
initiative but did not result in anticipated gains. Neither
the QAP nor healthcare providers were formally in￾volved in its development, which partly explains why
providers who had a copy said it had a limited impact
on their practice.
The QAP approach to setting standards consisted of
training about 300 staff to use the Dynamic Standard
Setting System (DySSSy), a step-by-step method to
develop standards that are situation based, realistic, and
owned by the users themselves. Those trained indicated
that it helped them in developing their district or health
center (HC) action plans, but the impact of this training
on quality remains undocumented.
x
2. Recommendations for the Development of
Standards
The development, adaptation, and revision of clinical
care standards should be defined in an official policy.
Health providers’ involvement at all levels of the health
system would ensure that standards are realistic and
accepted by those who should use them, while experts’
involvement would guarantee their scientific validity.
The development/adaptation of standards should be
consistent with the development of other materials,
such as pre- and in-service training curriculum material,
job aids, and any instrument used to measure
compliance.
The development of job aids should be based on an
assessment of the needs of health providers. More
research is needed into the kind of job aids that
providers would use and how the aids should be
developed and communicated.
D. Findings and Recommendations:
Communication of Standards
1. Findings on the Communication of Standards
The communication strategy of the ITG suffered major
delays, and very few health workers know of its exist￾ence. The district staff usually consider its format to be
non-user-friendly and its content too complicated for
some health workers who need a more practical instru￾ment to guide them in delivering care.
In general, health workers are unlikely to use existing
guidelines during a consultation. The extent to which
they would use any kind of job aids remains unclear.
2. Recommendations for the Communication of
Standards
A national strategy for communicating standards that
would not rely on only classroom training should be
developed. Its development should draw on principles
for behavior change that would improve clinical
practice.
Regulatory bodies and stakeholders should include the
new standards in the curriculums of Zambian nursing
and medical training institutions. This would reduce the
need for in-service training and might save some costs.
The continuous reinforcement of standards is key to an
effective communication strategy. Mechanisms should
be developed to help districts and officers in charge to
use all opportunities to do so (e.g., supervisory visits
and technical meetings).
E. Findings and Recommendations:
Assessment of Quality
Performance
1. Findings on the Assessment of Quality
Performance
The focus of the many mechanisms to monitor perfor￾mance is on inputs and outcome data and neglects the
process of care. No formal instrument is used to directly
observe healthcare worker performance.
There is no formal strategy to differentiate causes of
poor performance, so supervisors cannot distinguish
whether problems are caused by competency or some￾thing else. In-service training may be seen as a solution
when a lack of knowledge and skills is not the actual
cause of poor performance.
Good quality is not formally recognized and rewarded,
but healthcare staff would be receptive to a formal
system for doing so.
2. Recommendations for Assessing Quality
Performance
The assessment of quality performance should include
the measurement of health workers’ compliance with
process standards through direct observation of the
delivery of care, and not be focused only on inputs and
outcome standards.
A specific strategy should be defined to explore the
causes of poor performance in order to distinguish a
competency problem from other causes. Clinical
competency should be tested regularly to identify
training needs.
A formal quality performance-based recognition and
reward system should be established to create incen￾tives to improve quality. Those planning and imple￾menting this system should identify the kinds of rewards
that would be both effective and acceptable to district
health boards.
xi
F. Findings and Recommendations:
Problem-Solving Teams
1. Findings on the Number and Success of Teams
The evaluation confirmed that 26 out of 127 health
centers (20 percent) had active QA (or “problem￾solving”) teams.
The main reasons why teams do not form seem to be
the small number of staff at a health center, the lack
of regular visits by the coach, and the absence of a
QA-trained officer in charge. If this is true throughout
Zambia and while resources are scarce, then it is ques￾tionable whether teams should be formed absent these
conditions. The risk of failure appears high.
The evaluation team identified reasons why some teams
do not successfully complete a first problem-solving
cycle. The main ones are the complexity of the problem
and not following the steps of the cycle.
2. Recommendations for Quality Improvement
and the Problem-Solving Teams
Investigate methods for improving quality in HCs with
five or fewer staff. The findings suggest that the smaller
the staff in a center, the more difficult it is to form and
sustain a team that uses the methods effectively, possi￾bly because the staff time and skills are not available at
such centers. If the CBoH wants teams to form in small
HCs, further investigation should be done of the condi￾tions required for team success and the feasibility of
these conditions. Absent these conditions, the CBoH
should assess whether the teams can receive adequate
additional support from the district, and if not, which
quality assurance and improvement methods would
work at these centers.
Zambia should continue using the problem-solving
model, but with simplifications and modifications to
training. The problem-solving method can be effective
in HCs with over 10 staff under certain conditions. It
should continue at these HCs but be simplified for
people with minimal training.
3. Findings on the Problems Chosen, Methods
Used, and Difficulties Experienced
The relevance and importance of the chosen problems
for patients, community, and staff at the centers varied.
Waiting times was often chosen (n=7), in part because it
was an example in training. About half the teams chose
physical and facilities problems, and about 40 percent
chose clinical problems. The clinical problems were
relevant and important for the health of patients or
community (e.g., malaria, late antenatal booking).
About 5 percent of problems were identified by
soliciting user views or by neighborhood health
committee (NHC) input.
In general, specific features of the problem-solving
process, such as problem prioritizing and cause-and￾effect diagrams, were used correctly. The use of
methods for listing data sources (data matrix) and for
gathering and interpreting data could have been better,
and about 70 percent of teams could not show or
describe any data collected.
Teams experienced various difficulties, many relating
to deciding which data to gather and then gathering
reliable and valid data. Many teams encountered
problems in implementing solutions, usually because of
resource restraints (e.g., drugs, physical improvements,
transportation).
4. Recommendations for Training to Improve the
Work of the Teams
Zambia’s healthcare system should develop a simpler
five-day training package in team problem solving for
district training. It should draw on the experience of
links and coaches who have designed five-day or 20-
hour courses.
For the first cycle, use “learning by doing” for teams
with little training. In these cases a coach should attend
each meeting to train the team through learning by
doing and to show good practices in documentation,
how to use the methods, and how to link the steps.
5. Findings on the Results and Benefits of the
Teams’ Work
The results of the teams’ work were assessed through
measurable changes documented or reported. Five of
the 26 active teams achieved measurable changes in
quality, and eight others reported that they had achieved
significant quality improvements but did not have data
to show it. (The evaluation confirmed that some teams
achieved measurable improvements to community
health [e.g., lower malaria incidence], clinical care
process [e.g., increased rate of immunization], and
patient quality [e.g., shorter waiting times].)
6. Other Recommendations to Improve the Work
of the Teams
The “team evaluation” system used in this study should
be piloted as a method that coaches could use to assess
teams and to give feedback on how teams can be more
effective.
Zambia’s healthcare managers and providers should
develop skills in deciding when to use the full problem￾solving cycle and when not to. Links and coaches
should have more training in how to judge and advise
on which problems require the full problem-solving
cycle. They should train and advise teams on how to
decide and when to use an alternative, more cost￾effective method for solving problems.
Teams should experiment with user and neighborhood
involvement in QA problem solving. Users should be
added to selected teams, and neighborhood involvement
in problem identification and solution should be
developed. Pilot efforts to make recommendations for
increasing community involvement should then be
evaluated.
G. Findings and Recommendations:
QAP Support Systems
In the context of the Zambian health sector reform, QA
activities are being implemented within an integrated
framework for service delivery. As a result, training,
supervision, monitoring, and other systems that support
the delivery of essential health services have included
a QA component. Consequently, the main support
systems for QA evaluated in this report include QA
training, the QA coaches/link facilitators network,
documentation and reporting of QA activities, and
supervision of QA activities.
1. Training in Quality Assurance
Findings on Training in QA
At the central level, the Service Quality and Perfor￾mance Audit unit of the Directorate of Monitoring and
Evaluation is responsible for ensuring that all levels of
healthcare are sensitized to the concept of quality assur￾ance. This unit provides training and technical oversight
in QA to District Health Management Teams (DHMTs)
and health centers. Staff reported that the main con￾straints to delivering QA training are the funding limita￾tions, lack of transport, and coaches’ locations.
At the regional level, no special training has been given
to staff since the formation of the regional directorates.
The intent of the CBoH had been to provide training to
the regional office to sensitize these staff to QA. How￾ever, the ban on workshops, delays in appointing re￾gional office staff, and the simultaneous introduction of
hospital accreditation activities may all have contrib￾uted to the lack of QA training at this level.
At the district level, staff have received varying degrees
of training. Though few directors, if any, were known to
have received any formal QA training, several DHMT
staff had been trained specifically as QA coaches or
link facilitators or had otherwise participated in some
level of QA training.
At the HC level, the extent of QA training varied by
district. It seems that no uniform approach to training
was adopted at this level, but rather facilitators and
coaches devised training plans to suit local circum￾stances. Training needs, including the selection of staff
for QA training, are reportedly determined through
supervision and on-the-job performance.
Recommendations on Training in QA
To ensure that priority is given to QA training, a train￾ing needs assessment should be done as part of the
development of the yearly action plan. In reviewing
these plans, the CBoH should ensure that QA training is
adequately planned and budgeted.
An adequate pool of QA trainers/experts at the national
level should be developed. While QA training is cur￾rently a function of the CBoH, its capacity to provide
responsive, consistent training support relies on having
a sufficient number of staff to perform this function at
various levels of the system. At present, the expertise to
provide adequate QA training is too limited.
QA training should be given to the relevant regulatory
boards to incorporate QA concepts into preservice as
well as post-basic training curriculums.
All professional staff at regional and district offices
should be given some training in QA, within resource
constraints. As suggested by some districts, QA could
be incorporated into other management training or
capacity-building sessions.
xii
2. Coaches and Link Facilitators
Findings on Coaches and Link Facilitators
Health center staff reported that most coaching visits to
the HCs were not regularly planned and performed
unless they occurred as part of the DHMT supervisory
visit. How often coaching visits coincided with the team
meetings and how timely the visits were in addressing
difficulties encountered by the teams remained
uncertain.
Some link facilitators reported that the motivation and
ability of the QA teams to work through the problem￾solving process usually appeared to be externally
driven, i.e., a visit by the coach or facilitator would
prompt a team to complete one or more steps in the
process.
Several coaches and facilitators recognized some
limitations in their ability to provide adequate support
to the teams. For example, some felt that they could not
devote enough time to QA activities, particularly if they
were managers or held other positions on the DHMT
(as many did).
Local factors, such as staff de-linkage, drug shortages,
and funding problems, were commonly seen as ob￾stacles to effective implementation of QA activities.
However, the evaluation team believes that the degree
of skills transfer and coaching support usually depends
on the ability and willingness of district staff to effec￾tively plan for QA, the amount of resources committed
to train HC staff and support coaching, and the motiva￾tion of individual coaches and facilitators.
Recommendations on Coaches and Link Facilitators
At least one staff on the DHMT, i.e., a district-level
coach or link facilitator, should have responsibility for
QA activities in the district. Ideally, this person should
participate in the supervision visits to the HCs and
include QA in the integrated supervisory approach.
QA training should further strengthen capacity in QA
planning, effective communication and teaching, and
supportive supervision. Furthermore, support visits to
coaches and link facilitators should be intensified (at
least early on) to further ensure that training efforts do,
in fact, generate QA activities.
District offices and coaches should take advantage of
existing opportunities (e.g., district-level meetings and
other training workshops) to update their QA training
and share information on QA activities and quality of
care issues.
3. Documentation and Feedback Mechanisms for
QA Activities
Findings on Documentation and Feedback Mecha￾nisms for QA Activities
At the regional level, specific knowledge regarding HC
quality assurance activities and the work of the coaches
and link facilitators is limited. Reports of QA activities
are not submitted to the regional office, and existing
reporting forms do not capture information about QA.
At the district level, a summary report of HC activities
is completed by the link facilitator and forwarded to
QA staff at the CBoH. Quarterly meetings of all link
facilitators are held to disseminate results and share
experiences regarding the work of their teams. The
CBoH uses the results of these meetings to identify
opportunities for improvement and track the number
and locations of active QA teams.
At the HC level, the recommended storybook format is
intended to facilitate efficient and complete documenta￾tion of the team problem-solving activities. Many were
incomplete or had not been regularly updated by the
teams, and the supply of storybooks was often
inadequate.
Recommendations on Reporting and Feedback
Mechanisms for QA Activities
Reporting of QA activities should be included in exist￾ing reporting systems at regional, district, and facility
levels; timely feedback should be given to the appropri￾ate levels; and information should be shared between as
well as within levels.
QA staff at the CBoH should help determine what
information on QA activities would be useful at each
level of the health system and how it should be
reported.
QA staff should ensure complete documentation (i.e.,
in storybooks) of QA activities and effective use of
these documents to monitor the work of teams.
Storybooks should be available in sufficient quantity.
xiii
4. Supervision of QA Activities
 Findings on Supervision of QA Activities
Discussions with regional staff indicated that the region
can influence the quality of care at the HC level by
including DHMT clinical staff on the performance audit
visit, then delegating responsibility for solving identi￾fied problems to those responsible at the district level.
At the district level, supportive supervision, in the
form of a formal integrated team visit to the HC, was
reportedly often used to monitor QA activities and
identify problems to be addressed by the problem￾solving teams.
In general, HC staff appreciated the need for frequent
and “supportive” visits from their DHMT, but many
expressed a desire for more frequent formal visits from
district staff, especially to address technical issues.
Though technical supervision was reported to be
routinely planned and performed, HCs often cited less
frequent visiting than their district had reported.
Other planned and unplanned visits to the HCs were
also reportedly done, quite often to address problems
identified during the formal visit. In general,
unannounced visits by the director or other district staff
were thought to encourage continued attention to
quality.
Recommendations for Strengthening Supervision of
QA Activities
Regional offices should ensure that adequate and
frequent supervision is provided to the HCs by the
DHMT. Since supervision is planned and budgeted as
part of the yearly action plan, some interim review of
achievements could reveal where supervision problems
exist.
The district should plan and budget for an adequate
number of technical supervisory visits and ensure that
they are done as planned. These visits should be used to
initiate and support problem-solving activities at the
HCs and, in the spirit of supportive supervision, allow
for adequate feedback and local resolution of problems.
The person(s) responsible for QA activities at the
DHMT should ideally be involved in quarterly inte￾grated supervisory visits to health centers. Many QA
coaches and link facilitators were found to be DHMT
staff, which appeared to facilitate the integration of QA
into their technical supervisory duties.
xiv
Table 11
Summary of the Main Recommendations of the Evaluation Team
Category Topic Recommendations
General QA Policy ■ Develop a national QA policy that includes the private sector
Strategic Planning ■ Strengthen the vertical integration of QA into all levels of the health system
■ Strengthen the horizontal integration of QA into other directorates of the CBoH
■ Strengthen the integration of QA within the private health sector and parastatal institutions
Standards of Care Development of Standards ■ Design a policy for the development, adaptation, and revision of clinical care standards
■ Develop and adapt training and other reference materials that are consistent with the
standards of care
■ Develop job aids based on assessment of the needs of health providers
Communication of Standards ■ Develop a national strategy for communication of standards that relies on behavior
change principles
■ Include new standards in the curriculums of Zambian nursing and medical training institutions
■ Continuously reinforce the standards of care
Performance Measuring Compliance ■ Measure health workers’ compliance with process standards
Monitoring System with Standards ■ Explore the causes of poor performance
■ Establish a formal quality performance-based recognition and reward system
Quality Productivity of the ■ Investigate methods for improving quality in HCs with five or fewer staff
Improvement Problem-Solving Teams ■ Continue using the problem-solving model, but with simplifications and with modifications Activities to training
■ Develop a simpler, modular 5-day QA training package for district training
■ For the first cycle, use learning by doing for teams with little training
■ Pilot-test the team evaluation system as a method for helping teams to learn how to be
 more effective
■ Develop skills to decide when to use the full cycle and when not to
■ Involve users and NHCs in problem solving
QA Support QA Training ■ Assess QA training needs for the yearly action plan
Systems ■ Develop a pool of QA trainers/experts at the national level
■ Deliver QA training to the regulatory boards
■ Deliver QA training to all professional staff at regional and district offices
■ The DHMT should identify staff within the HCs for QA training
■ Train at least 2 people per HC in QA
QA Coaches and Links ■ At least one DHMT staff should have responsibility for QA activities in the district
■ QA training should further strengthen capacity in QA planning, effective communication and
teaching, and supportive supervision
■ District offices and coaches should update QA training and share information on QA activities
Documentation and ■ Include QA activities in existing reporting systems at regional, district, and facility levels
Feedback on QA Activities ■ QAP staff should identify what information on QA activities is useful at each level of the
health system
■ Ensure complete documentation of QA activities and their effective use to monitor the work
of teams
Supervision of QA Activities ■ Regional offices should ensure that DHMTs carry out supervision according to standards
■ Each district should plan and budget for appropriate supervision visits and ensure that they are
carried out as planned
■ The person(s) responsible for QA activities at the DHMT should be involved in quarterly
integrated supervisory visits to HCs
xv

1
The Zambia
Quality Assurance Program
Final Evaluation
I. Introduction
A. Purpose of the Evaluation
Since the 1993 start of the Zambian QAP, there has
been no evaluation of its impact or documentation of its
successes and challenges. After completion of the initial
plan, which consisted of building a QA capacity
in-country, the CBoH requested an evaluation to help
design the next steps to further improve quality of
healthcare. With USAID financial support, CHS was
selected to lead an international evaluation team.
This report reflects three weeks of fieldwork, from
September 14 to October 2, 1998.
The main objective was to identify the main lessons
learned from five years of QAP in Zambia in order to
design the next steps for improving the quality of its
healthcare. This information would guide discussions
on this topic during a joint CBoH-donors meeting in
November 1998.
Related objectives were to: (a) review the previous
years’ performance through achievements, (b) identify
problems and challenges, and (c) suggest recommenda￾tions to overcome the challenges.
More information on the mission’s purpose is presented
in Appendix A.
B. Methodology
The QA evaluation took place in nine districts in four
regions. Districts were chosen to include HCs with
active QA teams and/or staff who had received QA
training. Consideration was also given to selecting
districts and facilities that were easily accessible to the
evaluation team, though some attempt was made to find
representative sites, including both rural and urban
health centers. Altogether, 24 health facilities were
visited. Because the choice of facilities was not entirely
random, the evaluation team cautions that the sample
may not represent the entire country.
The evaluation team divided into three subteams: one
each on standards, problem solving, and support
systems. The topics for review were identified from a
systems view of a QA program after discussion with the
CBoH. Information was collected primarily through
semi-structured interviews with relevant staff at the
central, regional, district, and HC levels. Questionnaires
were developed by each subteam and refined during
use. To validate the results, the evaluation team
presented preliminary findings to the CBoH and its
partners at the end of the mission and incorporated their
thoughts into the report. This meeting allowed the
CBoH and its partners to comment on the feasibility of
the recommendations. A draft of the report was given to
the CBoH and USAID before the team departed from
Zambia, and all parties had an opportunity to make
comments on the findings and recommendations.
A summary of data collection methods by subteam
follows:
The Standards Subteam interviewed regional office
staff, DHMT staff, facility staff, and regulatory boards
and private and training institutions in the health sector;
it also reviewed supervisory reports, checklists, and
other reports used to monitor performance.
The Problem-Solving Subteam conducted focus group
discussions with the problem-solving teams or, when
not enough team members were present, interviews with
the in-charge or other relevant facility staff; reviewed
storybooks and other reports of QA activities at health
centers (e.g., link facilitator reports); and interviewed
QA coaches/link facilitators.
The Support Systems Subteam interviewed regional
office staff, DHMT staff, the in-charge or other relevant
facility staff, and coaches/link facilitators; it also
reviewed supervisory reports, checklists, and other
2
reports used to monitor QA activities (e.g., link
facilitator reports, storybooks).
Though the original scope of work had included the
costing of various QA support systems such as training
and supervision, not having a local cost analyst
precluded any meaningful analysis of costs and mea￾sures of cost-effectiveness. (The local expert initially
identified for the costing work could not join the team
at the last minute.)
C. Structure of the Report
The Executive Summary presents the salient findings
and recommendations of the full evaluation report,
while Chapters I and II provide an introduction to the
evaluation and background on the QAP, respectively.
Thereafter, information is organized according to the
framework used for developing the scope of work and
the design of the Zambian QAP. The QAP carried out
activities to build capacity in: (a) setting standards, (b)
developing monitoring indicators, and (c) problem
solving. It followed a logical QA rationale:
■ First, standards of care must be developed and com￾municated to the personnel in charge of meeting them
(Chapter III, Section A)
■ Then, compliance of the health workers with
standards of care must be measured (Chapter III,
Section A)
■ Finally, if quality performance is not satisfactory, a
team problem-solving approach can be used to make
improvements (Chapter III, Section B)
There is no sustainable program without support
systems; Chapter III, Section C, covers: (a) the training
of health staff in QA, (b) the coaching of the problem￾solving teams and the role of the link facilitators, (c) the
supervision of QA activities, and (d) the documentation
of and feedback on QA activities. Chapter III, Section
D describes the role of other organizations that
influence healthcare quality, such as neighborhood
committees and regulatory bodies.
Chapter IV presents recommendations from the identifi￾cation of achievements and challenges. The evaluation
team decided not only to state recommendations, but
also to present a range of available, feasible options, as
appropriate, to provide decision makers with choices.
This decision was made on the basis of team members’
concern that recommendations are sometimes too gen￾eral and might not be effective because implementation
issues have not been addressed. Nevertheless, this
report is not intended to be a detailed implementation
plan, but rather a guide to stakeholders in making strate￾gic choices for the next steps of the QAP in Zambia.
The methodology chosen for this evaluation allows
documenting both success stories and remaining chal￾lenges. This report includes five case studies developed
to illustrate both best practices and shortcomings.
II. Background: Quality
Assurance in Zambia
The CBoH QA Unit chose from the outset to concen￾trate on standard setting, indicator development, and
problem-solving teams. In 1993, the QA Unit defined
quality as “performance of intervention according to
agreed standards” and quality assurance as “the
measurement of the actual level of the service provided
plus the efforts to modify, when necessary, the provision
of these services in the light of the measurement.”
(Ministry of Health 1993)
A. The Relevance of QA for Zambia
Apart from the moral arguments for healthcare quality
in any country, there are good clinical and financial
reasons for QA, especially when resources are scarce.
An immunization program may achieve a high coverage
rate, but vaccines may be ineffective because of poor
cold storage or because staff do not know how to ensure
that the cold chain is maintained. Drugs may run out
because the process for ordering or supplying the right
drugs to the right place at the right time is inadequate.
Patient attendance or drug compliance may be low due
to poor quality interactions between patient and health
provider or poor quality patient education by providers.
In addition, patients are customers now that they are
making co-payments. Services must be attractive and
valued, or patients will forgo them or complain. Health
providers should pay more attention to what patients
want, including cultural and gender issues, in order to
attract and be available to all sectors of the population.
Training in quality methods makes it easier for health
providers to work with the community to assess needs.
This helps providers change their attitudes toward
community involvement and gives them ways to discern
individual and community needs. Using quality
methods, providers’ skills improve and their status in
the community increases.
3
B. Zambia’s QA Activities
There is no national, integrated QA program in Zambia
and therefore no specific policy or strategy to evaluate
in terms of objectives and outcomes. The evaluation
team found no document with details of a vision, policy,
and plan for quality assurance or of the role of the initial
QA Unit (and later the CBoH). Within the new struc￾ture, it is not entirely clear whether there is one unit
specifically in charge of QA activities or if all CBoH
units are involved to various degrees.
Quality activities in Zambia occur in two broad catego￾ries: (a) the work of the national QA Unit, with its
national network structure of local links and coaches
and (b) a variety of quality activities and initiatives that
have occurred since 1993. Some were instigated by the
QA Unit (e.g., quality problem-solving teams in HCs)
or involved the QA Unit (e.g., the design and imple￾mentation of the Health Management Information
System [HMIS]), and some are independent of the QA
Unit (e.g., quality components of the Integrated
Management of Childhood Illness [IMCI] program).
The following describes activities within these two
categories.
The national QA Unit (1993–97) initiated quality
efforts in Zambia in 1994: It established quality com￾mittees at the provincial, district, and hospital levels. In
1995 it collaborated with laboratory services to help
create a quality laboratory policy. The following year
it issued the “Patient’s (Client’s) Entitlements and
Responsibilities” leaflet to all districts with the intent
that they would copy and distribute it to the HCs. It also
worked to integrate QA into other activities and sensi￾tize different personnel to quality concepts and issues
(HMIS, IMCI, and mental health standards using
DySSSy).
The QA Unit’s national training program first piloted
QA training and implementation in 1994–95 in three
districts (Mansa, Lusaka, and Monze). Sensitization
workshops were presented to each district office and its
HCs, as was training in DySSSy and indicator develop￾ment (training was based on the model described in the
1993 MOH Health Center Quality Assurance Manual).
Next, in 1996, district training and pilot testing of a
Figure 11
CBoH Organization Chart
The Central Board of Health
The Director-General
Internal Audit Unit
Financial Unit
Board Secretary/
Administrative Unit
Directorate of South-West Region
Directorate of South-East Region
Directorate of North-Central Region
Directorate of North-West Region
Directorate of Monitoring
and Evaluation
Directorate of Health
Services Commissioning
Directorate of Systems
Development
Secretary
Information & Health Systems Research Unit
■ Manager - Information & Health Systems
Research
■ Information Technology Specialist
■ Documentation & Statistics Specialist
■ Epidemiological Monitoring Specialist
■ Health Systems Research Specialist
Service Quality & Performance Audit Unit
■ Manager - Service Quality & Performance
Audit
■ Service Quality Specialist
■ Environment Health Specialist
4
problem-solving and peer review system—where one
officer-in-charge would visit another HC and assess it
using a UNICEF-based checklist—occurred in six
Livingstone HCs. Third, training was given to three
provincial office personnel of all nine provinces. This
included sensitization and standard setting. Fourth,
training for certain personnel from all districts was
provided in standard setting and developing and moni￾toring locally specific indicators (DySSSy). About 300
people were trained, including 60 physicians and all
district directors, their deputies, and two other DHMT
members from each district. (Because of staff transfer,
QA capacity is not in all DHMTs.) The training was 10
days. In phase 1 some staff from all provinces were
trained in sensitization (one day) and DySSSy (four
days). Phase 2 provided indicator development training
(five days).
Finally, team problem-solving (five steps) training was
given from 1996–98. This covered all districts except
those in North-Western Province. About 230 people
were trained, including 10 physicians. Each was
expected to act as a district coach and to train HC staff
as part of the “extension” of the program. Provincial
link facilitators were identified to help with the training
(they became “regional links” when provinces were
disbanded in 1996). The training was 14 days: three
days of QA team building and nine on coaching and
problem solving followed by practice to validate
coaches. Then, three to six months later, the regional
link was to visit the coach who would give a one-day
training to staff at another HC in order to be validated.
National QA structure of coaches and links: The
structure and process of quality supervision and coach￾ing includes supervisory staff of two to three at the QA
Unit, 30 or more regional links, and about 200 district
coaches (two to four per district). The QA Unit
established and trained quality committees in
provinces, districts, and hospitals, but did not create
formal reporting links between them, although there
were national meetings. It changed its approach in 1996
to focus on districts and HCs. The QA Unit was
dissolved in 1997 after the creation of the CBoH.
Regular meetings and networks, including national
quarterly meetings for all regional links to report activi￾ties in districts, began in 1996. These meetings ceased
in 1998 when workshops were banned. Regional links
visit and support coaches and run local training with
coaches for some staff from all facilities in a district, or
they expect coaches to run the training themselves.
These district training events are usually three to five
days and often involve the officer in charge of the HC.
The coaches expect people coming to these events to
become trainers and to train their colleagues in the
facility and start problem-solving teams. Coaches are
expected to do follow-up visits to trainers to provide
support to them and the problem-solving teams. Some
coaches just train staff at their own or another HC,
rather than covering the whole district.
Problem-solving teams: A team of two to 15 people
was expected to form in each facility and identify and
work on a problem using the five-step cycle. Approxi￾mately three to six such teams have been formed in
most districts, holding from 150–300 team meetings
between 1996 and 1998. Some hospitals have formed a
“quality committee,” which may function as a
problem-solving team. Standard setting and indicator
development have occurred in some HCs, and some
districts have developed and disseminated protocols.
Local activities with a quality component: Quarterly
supervision visits of HCs by district personnel can
involve assessment and feedback about the facility,
clinical practice, and patient relations, but often the last
two are not covered. Neighborhood, HC, and area
committees present consumer views to HCs and
districts, and can work with staff to resolve quality
problems. Routine systems for the collection and use
of information include some quality indicators, includ￾ing HMIS self-assessment forms, quarterly progress
reports, IMCI monitoring surveys, and Safe Mother￾hood. Lastly, the performance audit by regions was
intended to include a quality component.
Another significant QA activity: In 1998 the Zambia
Health Accreditation Council (ZHAC) pilot tested the
accreditation of hospitals with the Joint Commission
International.
III. Achievements and
Challenges
This chapter describes the achievements and challenges
in: (a) the development of standards of care, (b) their
communication to health providers, (c) the mechanisms
to monitor compliance with standards, (d) the produc￾tivity of the problem-solving teams, and (e) the estab￾lishment of support systems for the QAP in Zambia.
A. Standards of Quality for Zambia’s
Healthcare System
Standards are a key element of a QAP. Without explic￾itly stated standards, it is difficult to objectively assess
5
performance and measure the quality gap. Simply
stated, a standard is a statement of expected quality.
Standards make explicit an organization’s expectation
for quality and provide guidance for actions and deci￾sions in relation to the provision of healthcare. If quality
is “doing the right thing right, right away,” then the
standard makes explicit who should be doing what, in
which way, and when.
The evaluation team asked health staff at various levels
of the health system what their understanding of a
standard was. Their answers included: “A statement of
performance that is measurable,” “A statement of an
achievable and acceptable target,” “The level at which
something can be attained,” “A set of procedures and
rules to follow,” “The way you care for a patient,”
“Agreed and acceptable ways of doing things, which
are measurable and feasible,” and “Unit of measure
against which you measure performance.” These
answers show a generally good understanding of
standards, but most respondents had some difficulty in
identifying available standards.
1. The Development and Adaptation of Standards
Standards are rarely developed from scratch, but
usually adapted, by local experts, from already existing
material, to a specific situation or environment. In
practice, locally recognized experts are usually asked to
work as a group to reach a consensus on the standards
of care. It is of utmost importance that standards be
scientifically sound and that their validity be checked by
a review of evidence-based medical literature.
Expert consensus is not necessarily appropriate for the
development of action-oriented standards, such as target
standards developed through DySSSy. This method
guides a health facility team in developing its own
standards to improve its specific situation. Such stan￾dards look more like targets (see Case Study #1).
The Main Documents on Standards in Zambia
Numerous clinical care standards have been developed
in Zambia, covering all activities of the essential pack￾age of services that first level health facilities provide.
The main ones fall into four categories:
Case Study Number 1:
Kabwe District Health Office Develops
Its Own Protocols
During supervisory visits, the link facilitator of the
Kabwe DHMT noticed that the referral of women in
labor to the hospital was inappropriate, resulting in
maternal deaths before or just after arrival. Once
women arrived, admissions procedures were incom￾plete and the gravity of their condition was not
correctly assessed. The supervisors also noticed that
the management of a patient at the outpatient
department of the hospital consisted mainly of a drug
prescription for the relief of the main symptom with￾out any physical examination. In fact, the procedures
for clinical management of both situations varied
considerably among health providers.
The DHMT concluded that health providers needed a
clear protocol for the referral and admission of women
in labor and for the screening of patients consulting
at the outpatient department. The link facilitator had
been trained by the Zambian QAP in a participatory
method to set standards (DySSSy) and used his skills
to improve the work of the health providers. Two sepa￾rate groups, one of midwives and the other of clinical
officers, met three times in one month for two to three
hours each time. The only costs were minimal trans￾portation expenditures. The midwives used some
medical textbooks to adapt standards for referral and
admission of women in labor; the clinical officers
brainstormed the screening procedures at the out￾patient department. Both groups quickly reached a
consensus, but encountered some difficulties at the
beginning of implementation. Health providers said
that they do not like lengthy checklists because they
are too time-consuming and because they felt detailed
procedures were a way to control their practice. As a
result, they take shortcuts. Their involvement in the
development of the protocols helped lift the resis￾tance. (The main constraint was to be able to get the
people together because of time limitations.)
The Kabwe DHMT learned some lessons from this
experience. First, the providers that most resisted the
application of the new protocols were the ones not
involved in their development. Second, the staff
gained confidence by setting realistic standards for
their working environment. Third, the staff perceived
that its work was made easier because the protocols
were posted on the walls as a reminder. The district
is following up on the compliance of health workers
with the new protocols through supervision visits.
Motivated by its success, the Kabwe team is setting
standards for the packaging and prescription of
antibiotics. The protocols developed by the Kabwe
District Health Officer are presented in Appendix B.
6
Special programs have developed clinical guidelines:
These guidelines focus either on one health condition or
on a set of special issues. Among them are: (a) guide￾lines for malaria case management from the Tropical
Disease Research Centre (TDRC), (b) maternal health
guidelines from the Safe Motherhood program, (c)
tuberculosis guidelines from the tuberculosis control
program, and (d) HIV/AIDS guidelines from the HIV/
AIDS unit.
Some guidelines developed by international health
agencies have been adapted to Zambia: These guide￾lines promote an integrated approach targeting a special
population. These are: (a) the syndromic case manage￾ment of sexually transmitted diseases (STDs) and (b)
IMCI.
The CBoH developed of the “Integrated Technical
Guidelines for Frontline Health Workers” (ITG): The
ITG identifies the six health thrusts that are Zambia’s
priority public health problems and prescribes their
management: (a) malaria, (b) reproductive health and
family planning, (c) HIV/AIDS and STDs, (d) child
health and nutrition, (e) tuberculosis, and (f) water and
sanitation.
Standards for accreditation of hospitals: These were
developed jointly by the CBoH and the Joint Commis￾sion International and were being tested during the
evaluation in 20 hospitals. They will eventually cover
all hospitals in Zambia. A document from the MOH on
certification of hospitals describes minimum standards
for hospital levels 1, 2, 3, and 4 (MOH Undated), but it
has not been officially adopted.
The QAP took a more action-oriented, problem-solving
approach to the development of standards. In 1995, the
QA Unit of the Health Reform Implementation Team
(HRIT) used the DySSSy method to develop a manual
of standards for clinical and nursing care (MOH
1993b). It was used as a training manual for setting
local target standards aimed at initiating the develop￾ment of action plans for the improvement of health
services by trainees. After returning to work, trainees
were supposed to involve their peers in developing
standards adapted to their specific situation. The
advantage of this method is that the health providers are
involved in the development of the standards they will
use, and it provides a simple approach for problems that
are not too complex. It also exposes health staff to the
concept of a systems view and its input, process, and
outcome components. Follow-up on this training
occurred during the second round of training in moni￾toring indicators. Although the results were uneven with
some teams more advanced than others, the capacity it
built helped the staff develop their action plans and
monitor their progress. However, there is little docu￾mentation on the full impact of the training.
The Development and Adaptation of Clinical
Standards: The Roles of Stakeholders
The development of clinical guidelines remains central￾ized, with little involvement of the QAP.
The Systems Development Directorate noted a lack of
ownership and coordination by the CBoH in the devel￾opment of clinical guidelines by different programs. But
because the various guidelines could cause confusion
among health staff, the CBoH decided to develop a
manual of standards (a reference book) for the six
health thrusts. A task force on integration of vertical
programs assembled all existing guidelines, and groups
of experts from the MOH, CBoH, University Teaching
Hospital (UTH), and other institutions together devel￾oped the ITG. The guidelines were not tested but were
well received by the health workers who received
copies.
The regional directorates visited by the evaluation team
had not been involved in the development of any stan￾dards of care, including the ITG.
Most of the DHMTs the team met had received the ITG
draft and were asked to make comments and suggest
changes. Some took this more seriously than others and
organized technical meetings to address it, but in gen￾eral this activity was not undertaken enthusiastically.
Staff largely reported that they felt that the work was
too advanced, with little room for change. Their
involvement might have been greater at an earlier stage.
This is important for future revisions since the sense of
ownership of standards of care influences the effective￾ness of the communication and monitoring strategies,
which the districts manage.
The evaluation team met no health center staff who had
been involved in the development of the ITG.
The private sector is not sufficiently organized to pro￾duce its own guidelines. Its role, through the Faculty of
Private Practitioners, in the development of clinical
standards is limited to the irregular participation of local
experts in the writing of medical textbooks or guide￾lines for health providers, initiated by special programs
or the CBoH.
7
Official Policy for the Development and Revision of
Clinical Standards
There is no official policy for the development and
revision of national standards of healthcare. The QAP
indicated that the Systems Development Directorate is
in charge of this kind of work.
Problems in and Factors Affecting the Success of the
Development of Standards
The evaluation team was unable to document some
aspects of the development of the ITG, which involved
the collaboration of many public health and medical
experts with support for coordination from an interna￾tional agency. It would have been useful to learn the
extent to which evidence-based medical literature was
used and the cost. The ITG was produced in a remark￾ably short time: five months.
The staff trained in DySSSy met peer inertia in their
facilities. The evaluation team found that not much
happened after the training because of numerous diffi￾culties, especially the setting of unrealistic standards
(requiring unavailable resources) and the inertia of their
working environment. Staff mentioned how difficult it
was to create a team spirit and initiate quality standards.
Although no one reason appears to predominate, the
evaluation team developed several hypotheses:
■ When only one person per health facility is trained,
the absence of a sense of ownership of the process in
the majority of staff might limit their commitment to
implement standards
■ If trainees are not the health facility in-charges, their
authority to take a lead role might encounter some
resistance
■ There might also be an age factor; young trainees
may not have enough authority to lead a team
■ There might be some gender issues; women’s teams
seemed more dynamic
■ When health facility staff is too small (fewer than
10), it might be more difficult to initiate and sustain
the team dynamic
■ The trainees’ capacity to transfer their knowledge to
their peers might be limited after being exposed to a
new concept not entirely mastered. This may prevent
the health facility staff from grasping the concept of
standards and thus be unable to operationalize it
(much less train others)
■ New trainees need close follow-up and access to
technical support, which they might not request, such
as a coaching mechanism
■ When a standard involves a change in individual
clinical practice, the influence of the team might be
less than for structural standards (cleanliness of the
facility) or organizational issues (personnel shifts).
This relates to the fact that a patient consultation is
usually not a team activity
■ The transfer of QA-trained staff between health
facilities hampered the continuation of the QA
activities. After the departure of the trainee, the team
would cease to function
■ A minimum level of resources is necessary for devel￾oping and meeting ambitious standards, and rural
health centers, usually being the poorest, suffer from
great inertia
Costs of Developing Clinical Care Standards
The cost of developing standards could not be deter￾mined without a cost analyst on the evaluation team.
The global cost of the development of the ITG could
not be documented because it was funded by a donor
agency and the person in charge had since left the
country. The retrieval of this information would have
required considerable time.
2. The Communication of Standards
After clinical standards have been set, the health
providers who are expected to comply with them must
know them, understand them, have the skills to apply
them, and accept them. This “communication” strategy
goes beyond the simple use of classic information and
dissemination channels and encompasses activities to
induce and sustain a behavioral change in clinical prac￾tice. This is complex since practitioners accustomed to
managing a specific health condition in a certain way
are unlikely to change their practice after training only.
Substantial research in this area has been done in indus￾trialized countries, and general principles to change
behavior are well known, such as the involvement of the
users in developing the standards and the leadership of
recognized experts in their dissemination. However,
there is little research on this issue in developing
countries.
One way to communicate the standards to users is
through in-service training. Although this might be an
essential step, it is not the only one. In fact, decision
makers should be creative in seeking the most
8
cost-effective combination of ways to communicate
standards, such as:
■ The involvement of regulatory bodies and profes￾sional associations
■ Distance-learning programs (computer, radio)
■ Distribution of user-friendly leaflets
■ Inclusion of the new standards in the preservice cur￾riculums of training institutions
Communicating Standards to Healthcare Providers
The plan to communicate standards in Zambia relies
entirely on classroom training, which the CBoH is using
for combined ITG and HMIS training. This training
was expected to be complete in all districts by 1999.
This strategy should be monitored to study the real
impact of only in-service training on provider behavior.
If only one staff per clinic is trained, in-facility dissemi￾nation will be unreliable and perhaps ineffective.
(Training in ITG is also part of the curriculum for
Public Health Practitioners, a new cadre of professional
HC staff to be developed.)
Most of the regional directorates and DHMTs that the
evaluation team met did not have a specific strategy for
communicating the ITG to staff. The distribution itself
was a problem; photocopying resources are limited, so
few HCs had received it. Because the communication
method has not been fully successful, District Health
Officers (DHOs) were familiar with the ITG, but few
health practitioners were. (Several districts mentioned
presenting and discussing the guidelines during their
technical meetings with health center staff.) The ITG
booklet has not been given to private practitioners and
other institutions. The guideline most frequently cited
by public facilities was the TDRC malaria booklet
(TRDC 1997).
Healthcare Providers’ Use of the Clinical Guidelines
District health staff perceive the format of the ITG as
not user-friendly. They welcome it as a reference book,
but it resembles a medical textbook, raising the question
of its expected use to improve performance. As part of
the communication strategy, health providers should be
trained to make the best use of the ITG, both during and
outside patient consultations. Health providers had
Case Study Number 2:
Communicating Integrated Technical Guidelines
The CBoH published the ITG in May 1997. This
216-page document was intended to help providers
manage the six most important health thrusts by
reminding them of the protocols. It explicitly delineates
standards for quality care and services and was the
first attempt to compile quality standards for the
delivery of the essential package of health services.
Eighteen months after its release, the use of the ITG
by the target population remained limited for two rea￾sons: a delay in its dissemination and its format. None
of the regional directorates that the evaluation team
met participated in the distribution of the ITG to its
target audience. In fact, one regional directorate was
not even aware that they had a copy in their library. All
the DHMTs visited had received it, but only one or a
few copies and not enough for all health centers. Most
DHMTs did not know how to distribute it and were
reluctant to make copies because of the amount of
paper it would consume. Only one DHMT reported
writing a summary and photocopying it for the health
centers. All regional directorates and DHMTs reported
that they found the ITG bulky, not user-friendly, and
too complicated for some staff. Consequently, the
vast majority of the frontline health workers that the
evaluation team met did not have it. Some had heard
about it, and the few who had it did not know how to
make the best use of it. Some district offices, how￾ever, discussed the ITG content with health center
staff during technical meetings. Although the evalu￾ation team did not further explore the issue by
testing users’ knowledge of the standards, the main
lesson drawn is that any attempt to develop stan￾dards must be accompanied by a clear and thoroughly
thought-out communication strategy. Just sending
guidelines without explaining how to use or commu￾nicate them is not enough to induce change.
The CBoH is currently providing ITG training combined
with the training of the districts in the HMIS. Class￾room type in-service training seems to be a manda￾tory step but might not be enough to ensure compli￾ance with standards if user-friendly job aids are not
developed in a way that involves the health workers.
More operations research on this specific issue is
required. The influence of guidelines on clinical prac￾tice must draw on behavioral change theories. This
also requires a needs assessment for practical job
aids that health workers will be willing to use.
9
difficulty articulating what kind of job aids they need
and would use during a consultation to remind them
of the steps in the management of a specific health
condition.
The limited number of HC staff who have the ITG view
it as a reference document and use it during spare time
to update their knowledge on treatments, which is a
positive finding. However, the ultimate impact on per￾formance and prescription habits must be documented.
District staff also do not use the other guidelines. In the
districts trained only in IMCI, a minority of health
workers mentioned using the IMCI guidelines. The
main reasons given for not using them were that they
take too long and that it is not practical to do so with a
patient. One nurse said, “We know these diseases. . . .
They are easy to treat.” She said that she had memo￾rized the IMCI guidelines. Others mentioned the fear of
looking ignorant by reading a document in front of a
patient. Various surveys also indicate that health work￾ers are reluctant to use guidelines during their clinic
(Centre for Health, Science and Social Research
[CHESSORE] 1997).
Lusaka District has developed some protocols for com￾mon causes of consultation and trained staff in their use.
The documents look user-friendly, providing both an
algorithm and a narrative explanation of the case man￾agement steps. However, the evaluation team found no
information on their use; personnel seemed not to be
using them and even had difficulty finding them.
In general, the design of clinical guidelines should be
driven by the type of job aids health providers need to
guide them in their work. The ITG’s different chapters
have different structures. Some are organized as flow￾charts, others have lists of instructions, and others are
narrative only.
The Role of QAP in Communicating Standards
The QAP is not involved in communicating the ITG,
which is being done with the training of district staff in
the HMIS. The impact of this strategy on health work￾ers’ performance should be monitored.
The communication of the ITG would benefit from the
QAP expertise. Medical reference books are needed,
and the initiative to develop one consistent with the
essential package of health services is relevant and
would be appreciated by staff. However, such a docu￾ment is only a starting point for various activities that
would strengthen the standards communication strategy,
such as:
The development of job aids: Developers of clinical
guidelines should think of the material and format that
health workers need to help them deliver quality care.
Medical textbooks represent the reference material for
the development of job aids.
The statement of standards: Medical textbooks can be
used to make standards of care explicit and develop
instructions and protocols.
The development of tools to measure compliance with
standards: The standards that have been communi￾cated, provided they meet the criteria of good standards,
should be the ones used to measure performance of
health workers.
The development of preservice training materials:
Medical textbooks are good references for the content
of preservice curriculums. If this were the practice, then
the cost of in-service training would be reduced.
The QAP could be involved in the first three activities,
which require specific QA expertise.
The Main Constraints in Communicating Standards
The main constraints to an effective communication
strategy are: (a) the absence of an integrated policy for
the communication of standards that relies on principles
for behavior change and (b) the complexity of the
factors that influence a clinical practice change in a
specific society or context. The most cost-effective
strategy to communicate standards should be
determined.
The Cost of the Communication Strategy
The evaluation team was unable to derive any meaning￾ful cost without a cost expert.
3. The Quality Performance of Health Facilities
Performance can mean different things, ranging from
the physical improvement of the facility to the achieve￾ment of specific service coverage. The dimension of
performance investigated by the evaluation can be
called quality performance and can be defined as com￾pliance with structural standards for health facilities,
with standard operating procedures by managers, and
with clinical care standards by health providers. In this
evaluation, the team looked mainly at the technical
quality of the healthcare delivery process, trying to
answer the questions, Are health providers doing what
they are supposed to do? How can we know? and How
can service be improved?
10
Case Study Number 3:
Monitoring in Lusaka Improves Outcomes
After the introduction of IMCI, four health facility￾based surveys documented change in health worker
performance over a two-year period in Lusaka
District. The compliance of health workers with IMCI
was measured through direct observation of
provider/patient encounters three months before
the IMCI training, and then repeated at several
month intervals. The results in Table 41 show that
performance was very poor before training, improved
dramatically after, and then started declining.
The DHMT was concerned about this trend and
decided to include observation of the sick child in
the supervision visits. Supervisors were trained in
IMCI and a one-page checklist was designed to
serve as a job aid and was added to the supervi￾sion checklist. During their visits, supervisors iden￾tified tasks not performed correctly by the health
workers and provided on-the-job feedback and train￾ing. This strategy produced improvements, as shown
by the results in Table 42.
This case illustrates the influence of regularly
reminding health providers about the standards of
care after identifying shortcomings through direct
observation of their performance. When supervisors
pay attention to the important aspects of care, the
impact of making a correct diagnosis, prescribing a
correct treatment, and the health outcome are bet￾ter. While not unique, the Lusaka case represents a
best practice for Zambia that should be dissemi￾nated to other districts. Its lessons should not be
limited to the IMCI standards but expanded to en￾compass the standards for the other health thrusts.
Table 41
Changes in Provider Performance
with Training and with Time
The Health Worker: Before Two Months Eight Months
(Percentage of Times) Training after Training after Training
Counted respiratory rate 2 70 67
for cough
Checked dehydration by 34 55 60
pinching skin
Prescribed antibiotics for 47 15 28
a common cold
Explained the treatment 25 69 40
Table 42
Changes in Provider Performance
with Clinical Supervision
The Health Worker: Before After Four Rounds
(Percentage of Times) Clinical of Clinical
Supervision Supervision
Counted respiratory rate 67 84
for cough
Checked dehydration by 60 61
pinching skin
Prescribed antibiotics for 28 17
a common cold
Explained the treatment 4 79
Assessing Quality Performance of the Health System
and Services
Many health system stakeholders are involved in
monitoring performance. The various levels of the
health system monitor the performance of the level
directly below. Regional directorates audit the district
teams who, in turn, supervise the HCs.
All cost centers (every unit that manages a budget) are
involved in some sort of self-assessment, since they
report on their performance toward achieving their
action plans (quarterly progress report) and soon will
report on national health outcome targets (HMIS).
The regulatory bodies—Medical Council of Zambia
(MCZ) and the schools of nursing (General Nursing
Council [GNC])—are involved in monitoring the pri￾vate clinics and hospitals.
The Zambian Health Accreditation Council (ZHAC) is
starting an ambitious, nationwide accreditation program
for all hospitals.
The involvement of the users in monitoring the perfor￾mance of health services and providers is informal.
Individual patients express their dissatisfaction with
quality of care irregularly, most often when they have to
pay and do not get medicines. (The notion of paying for
11
services only, and not goods, has not yet reached the
poor.) Some health facilities have suggestion boxes, but
only literate patients can use them and usually do not.
The NHCs represent an opportunity for involving the
community in a more structured way to monitor quality
while educating users on what they should expect.
District supervision of health centers is one among
many mechanisms to assess performance. There are
four main mechanisms to monitor performance of
health services: performance audits, supervision visits,
the HMIS, and quarterly progress reports.
Performance audits: Each of the four regional director￾ates is expected to perform a quarterly performance
audit, consisting of an inspection of DHMTs, hospitals,
and health centers. In practice, the regional team
inspects the district team and they together inspect one
or two facilities per district. Four different forms are
used: (a) the regional directorate quarterly performance
audit form, (b) the integrated performance audit check￾list for financial assessment, (c) the aged imprest analy￾sis, and (d) the health services standards. Performance
audits mainly collect information on financial
management, accounting procedures, facility structural
standards, and managerial and planning processes.
Progress according to action plan is monitored through
output and outcome measures for all cost centers, using
record reviews, observation, and interviews with staff.
Few indicators collect information on process stan￾dards, such as the pattern of antibiotic prescription, the
investigation of maternal deaths, and the proportion of
patients diagnosed and treated according to standards.1
Supervision visits: DHMTs irregularly perform super￾vision visits of HCs. All DHMTs indicated that they
supervise all HCs monthly or quarterly, but this was not
always confirmed by the HCs themselves or by the
supervision reports. A 1997 situation analysis of repro￾ductive and child health services found that in six
months, 32 percent of HCs were visited three times, 23
percent twice, 23 percent once, and 22 percent had not
been visited (Central Statistical Office 1998). Formal
supervision visits usually occur in teams of three or
four, which see two to three HCs per day. The average
visit lasts one to two hours.
Checklists are used during supervision. The evaluation
team identified four different ones (see Table 4-3 for
their main features; see Appendix C for examples). All
checklists have serious shortcomings to assess the
quality of clinical care. An attempt by the CBoH to
develop an integrated checklist involving direct obser￾vation of care has been started but not completed. Only
IMCI tasks are being recorded and other activities of
the six health thrusts are just mentioned without any
details. Although most districts that the team visited had
received the integrated checklist developed by the
central level, only one is currently using it. The main
reasons for not using it are that it is too time-consuming
to observe the health worker deliver care and that super￾visors prefer checklists that focus on inspection of
facilities and record reviews.
The evaluation team found a consistent pattern among
DHMT staff in avoiding the assessment of clinical
performance of the health workers through direct obser￾vation of a consultation. One district even re-designed
the checklist they had been using and deleted this
previously carried-out task. Another one uses the
“health center supervision checklist” and fills in every
section except the one on observation of care for
children. As a result, recommendations found in super￾vision reports emphasize the importance of managerial
functions (record keeping) and structural criteria
(painted wall, well-kept garden), signaling that compli￾ance with clinical standards is unimportant. There are
some exceptions to this trend. For instance, the Monze
DHMT reported that a clinician directly observes health
workers (this was confirmed by HC staff), but they
don’t use any checklist, and the supervision reports do
not mention anything specific in this regard.
The Health Management Information System
(HMIS): This is a redesign of the former Health Infor￾mation System wherein health facilities reported service
statistics to the central level with no local follow-up.
The system has been pilot tested in 15 districts and is
being rolled out nationwide, combining training on the
Financial Accountability Management System and the
ITG over a two-week period.
The parts of the HMIS most relevant to this mission are
the quarterly self-assessment forms that are completed
by HCs and then by districts to monitor their own
performance against predetermined national and local
targets. The forms are designed to help teams identify
the areas of low performance and take action. Health
center forms (HIQ.1 and HIQ.2) collect information on
15 input and output indicators related to utilization of
services and coverage statistics. The only indicator that
1
 The latter is measured through review of a sample of 10 records for out-patients and in-patients. Since medical records usually indicate only the symptoms
and the treatment, these reviews check the appropriateness of the treatment but not the accuracy of the diagnosis. There is no direct observation of the
delivery of care, so the clinical performance of health workers remains unknown. Review of performance audit reports by the evaluation team confirmed
this.
12
Table 43
Features of Supervision Checklists
Name Health Checklist for A Supervisory Other
(Description) Center Assessing Basic Checklist Locally
of the Supervision Quality of Care at the for the Six Adapted
Checklist Checklist Health Center Level Health Thrusts Instruments
Used by Lusaka, Lufwanyama Kalulushi, Kitwe Kalulushi, quarterly Kabwe
Methods used
Direct observation of the provider Yes, for case management of No Yes, only for case No
sick child (according to IMCI management of sick
standards), family planning child (according to
services, and postnatal visits IMCI standards)
Interview with patients No No Yes, health center No
exit interviews
Record review Yes Yes Yes Yes
Inspection of the facility Yes Yes Yes Yes
Information collected
Physical aspect of the facility Yes Yes Yes Yes
State and amount of equipment Yes Yes Yes Yes
and supplies
Drug stock Yes Yes Yes No
Coverage and utilization rates Yes Yes Yes
Compliance of health workers Yes, for IMCI, family planning, No Yes, IMCI only No
with process standards and postnatal visits
Appropriate use of drugs Yes, for diarrhea and ARI only No Yes, for diarrhea and ARI only No
Finances Yes No Yes Yes
Service statistics Yes No Yes Yes
Health services checked
Sick child care Yes Yes Yes No
Antenatal care Yes Yes Yes No
Postnatal care Yes Yes No Yes
Outpatient care Yes Yes No No
Maternity care Yes Yes No No
Family planning Yes Yes Yes Yes
Tuberculosis Yes Yes Yes Yes
QAP activities
Setting standards No No No No
Developing indicators No No No No
Problem-solving activities No No No Yes
13
measures performance is the “daily staff load for cura￾tive and preventive care.” Once HCs identify indicators
that do not meet the threshold, they are supposed to
implement the “triple A approach” (assessment, analy￾sis, and action: an adaptation of the problem-solving
methodology taught by the QAP). The links between:
(a) performance and (b) output targets and quality
improvement activities are explicit. The self-assessment
of service performance might help QA teams focus on
problems directly related to the delivery of health
services. However, the HMIS does not collect informa￾tion on clinical performance and thus does not identify
process and competency issues in the delivery of care.
The quarterly progress reports: Each district sends
quarterly progress reports to two CBoH directorates:
Monitoring and Evaluation, and Health Services
Commissioning. The reports consist of a self-reporting
of the achievements of each district in meeting its action
plan targets. The information is used to disburse grant
money to the districts. All areas of administration
(number of meetings held), service activities (coverage
and utilization rates), and purchase of supplies and
equipment are included. This mechanism is supposed to
be replaced by the HMIS self-assessment forms once
districts are trained in the new system.
The focus of performance monitoring is on input and
outcome data. None of the monitoring mechanisms
captures the information that would measure health
providers’ compliance with clinical care standards.
Table 4–4 analyzes the focus of the data collected
through performance audits, supervision visits, and the
HMIS self-assessment forms, as a percentage of the
total number of items checked at the HC level.
Obviously, the focus is not on the process of care.
Supervision checklists that had included the monitoring
of care delivery have been discarded by the DHMTs.
In Lusaka, the IMCI part of the checklist was never
filled out, although supervisors say they are directly
observing the treatment of sick children. The DHMT
in Kabwe redesigned its own supervision checklist and
deleted the direct observation of sick children. It is not
clear why district teams are uncomfortable observing
health workers.
District-specific activities: Apart from the main
mechanisms described above, the evaluation team
found examples of innovative district-specific activities
to both monitor quality of care and improve it:
■ Spot-check visits: organized without warning by
DHMT to HCs on an as-needed basis, targeting the
centers that do not do well (Monze)
■ Peer review of IMCI: performance monitoring done
by one health center staff who goes to supervise
another HC (Kitwe)
■ Weekend coverage visits: a team of HC staff taking
turns visiting all the other HCs in the district (Kitwe)
■ Technical supervision visits: focus on specific issues
or programs, such as malaria control (Monze) or
screening of pregnant women for syphilis (Kitwe)
■ Maternal deaths reviews: discussing the case among
technical staff to identify what went wrong
(Chimwemwe, Kitwe)
■ Patients’ satisfaction surveys: organized biannually
through exit interviews of patients by coaches
(Kabwe)
While numerous and varied, these activities and their
impact cannot be known without documentation.
Table 44
Types of Data Collected on Health Center
Performance
Form Input Process Output and
Data Data Outcome Data
Performance audit 21 7 71
Health center supervision 76 20 4
checklist
Checklist for assessing 65 5 30
basic quality of care at
the health center level
A supervisory checklist for 66 29 5
the six health thrusts
Health center self- 33 0 67
assessment form (HIQ.1)
Numbers indicate the percentage of items checked; Totals may not equal 100%
due to rounding
Input Data: availability and state of the facility, equipment, and consumables
Process Data: direct observation of the care delivered or proxy indicator
Output and outcome data: coverage and utilization rates, health statistics
14
Documentation of Changes in Quality Performance
While the results of special surveys are limited and do
not allow comparison over time, most information
available on the technical quality of care comes from
them or research. The evaluation team identified the
following:
A 1993 survey by the MOH Health Systems Research
Unit: This revealed that “Standard WHO/Control of
Diarrheal Diseases guidelines for case management
were adhered to by prescribers in 51.7 percent of
patients with acute diarrhea and 42.3 percent of patients
with dysentery.” (MOH 1993b ) As a result, in 77
percent of cases, oral rehydration solution and antibiot￾ics were incorrectly prescribed, and in 58 percent,
incorrectly dispensed. Among the factors responsible
for inappropriate use of drugs were the providers’ lack
of correct knowledge (44 percent) and the limited use
of standard treatment guidelines (14 percent).
A 1996 review by the Participatory Assessment
Group: This revealed that “technical competence and
qualifications of clinic and hospital staff was not
questioned by patients. However, great concern was
expressed over the fact that diseases are diagnosed by
simply talking with patients and even the use of a
thermometer is rare.” (MOH 1996a)
The 1997 Zambia Situation Analysis of Reproductive
and Child Health Services: This assessed quality of
family planning services using six criteria (interpersonal
relations, choice of methods, information given to
clients, technical competence, mechanisms to encour￾age continuity, and constellation of services). (Central
Statistical Office 1998)
The findings point out low performance areas: 1 percent
of health providers test for pregnancy, 21 percent per￾form a clinical exam, and 2 percent look for symptoms
of STD. It also revealed serious gaps in the case man￾agement of sick children under five according to IMCI
standards. Among them: 20 percent of the children are
weighed; 2 percent of health providers searched for all
four danger signs; 10 percent of all children with a
cough had their respiratory rate counted; and 21 percent
of pneumonia cases received antibiotics. The study
shows the gap between satisfactory knowledge of case
management and real performance.
A 1997 review of the HMIS cascade training: This
noted that “health workers have difficulties in diagnos￾tic skills, which affects not only quality of care, but
consistency and quality of data as well.” (Heydelberg,
Mubonda, and Tembo 1997) It also states that the
HMIS is hampered by a tremendous load of insufficient
diagnostic and curative skills, which influences the
reliability of HMIS data.
The 1997 review of the Extended Program on
Immunization in Zambia: This revealed numerous
performance gaps that could potentially prevent the
high coverage levels to reduce disease-specific morbid￾ity and mortality in children (Republic of Zambia
1997). Among those gaps, 54 percent of refrigerators
had their temperature checked daily, 50 percent of
sterile techniques met standards, and 44 percent of the
mothers knew when to return.
A survey on quality of care: This indicated that
physical examination was performed on 96 percent of
patients with malaria at general hospitals but only on 39
percent at rural health centers (CHESSORE 1997).
The CHESSORE performance study: This monitored
the performance of health services over three years in
four provinces, using malaria as a tracer condition.
Three main types of indicators were collected: case
fatality rates, health worker knowledge, and preventive
activities. The main results are presented in Table 4-5.
Although the results have not yet been analyzed and
the use of a tracer is questionable, the health workers’
lack of knowledge on the treatment of the most com￾mon cause of morbidity in Zambia is striking, as is the
worsening of this knowledge over three years. Again,
these surveys did not observe the case management of
malaria by the health workers, so actual performance is
unknown. (CHESSORE 1998b and c).
IMCI Documentation
The only results that the team was able to find on the
monitoring of compliance with process standards over
time come from repeated facility-based surveys by the
BASICS (Basic Support for Institutionalizing Child
Survival) program for the implementation of IMCI. The
main results are presented in Case Study Number 3.
Results in other IMCI provinces (Kitwe) are consistent
with the Lusaka findings.
A strong QAP creates an opportunity to establish a
quality performance monitoring system. Although
monitoring compliance using input and outcome stan￾dards is important, any change in output or outcome
measures (such as coverage rates) is very difficult to
interpret without knowing the specific intervention or
structural change. In other words, the process must be
documented if one wants to understand a change in
outcome or interpret its value. This information is not
available in the surveys that we found. The paucity of
performance data, as well as the absence of repeated
15
measurements over time accompanied by a thorough
documentation of a QA intervention, is a serious
constraint in the evaluation of a QA program. In
addition, in the context of the health sector in Zambia,
it would be difficult to relate a change to a specific QA
intervention, given all the other influences, unless a
quasi-experimental research design had been created at
the start.
How Performance Information Is Used to Improve
Quality of Care
Performance information is used in an ad hoc manner.
DMHTs have not been able to describe a formal pro￾cess to review and analyze the information on the per￾formance of health facilities that is collected during
supervision. Notwithstanding the fact that information is
not focused on the process of care, the compilation of
results was not done systematically.
One DHMT computed an overall performance score
but, unfortunately, used a poor checklist, thereby
deriving meaningless figures. The trends in the overall
score were not analyzed by the district, so it could not
objectively interpret any change.
All supervisors said they give some feedback on their
findings to the HC staff. Most write recommendations
in the HC visitor book and/or a report kept at the district
health office. The evaluation team reviewed a number
of these documents; they focus mainly on structural
improvements and exceptionally report on clinical care
delivery.
Health providers’ competency is almost never tested.
No district said that they test the knowledge and skills
of health providers to determine the causes of poor
performance. The supervisors mentioned asking
providers about reasons for their findings, but they
asked no questions on the case management of a
specific health condition. The evaluators felt that
district teams were not comfortable testing the health
providers’ knowledge and skills.
95/96 97/98 95/96 97/98 95/96 97/98 95/96 97/98
MALARIA CFR
General Hospitals
Under 5s 4.5 3.7 5.2 4.7 6.3 5.9 9 3.2
Over 5s 3.5 2.5 1.7 3.5 5.1 4.7 8.6 4
Other Hospitals
Under 5s 2.6 4.3 3.8 4.5 6.5 11.6 5.5 4
Over 5s 1.4 5.5 1.9 2.5 4.5 43.4 5.8 5.8
HEALTH WORKER (HW) KNOWLEDGE
No update of HW knowledge through training session 75 63 72 69 64 63 59 57
HW knows the dose of Quinine
Child 10001293
Adult 10101111
HW knows the dose of Chloroquine
Child 5 0 4 1 11 10 11 3
Adult 30314214
PREVENTIVE ACTIVITIES
Use of bednets by the population, as a result of the 6 7 9 6 5 5 17 6
HW’s promotion activities
(CHESSORE 1998b and c)
Table 45
Changes in Health Services Performance (Percentage)
Copperbelt Eastern Northern Western
16
Districts want to reward the best performers, so they
need to identify them. Districts were very open and
receptive to creating a performance-based incentive
system. Several districts mentioned that some staff had
been given a Labor Day award; some awarded money
to the best HC to buy cleaning products, while others
are thinking of giving trophies. Limited resources will
influence the type and level of award, but the willing￾ness to identify best practices is a major asset.
Some districts see another reason to identify the best
performers: to use them to train others. The effective￾ness of this peer training approach is largely undocu￾mented, but it might help disseminate best practices.
As part of the de-linkage of the health staff from civil
service to district health boards, a formal competency
appraisal is planned.
The HMIS represents an attempt to integrate perfor￾mance measurement with quality improvement meth￾ods, but its indicators do not capture information on the
process of care. The development of a link between the
results from the HMIS and the problem-solving teams
through the triple A approach is a positive development
initiated by the QAP. The way health output and out￾come data will be used to make decisions and solve
problems should be monitored.
Main Constraints to Performance Monitoring
Supervisors expressed some constraints to performance
monitoring, including:
Lack of time: The observation is seen as too time
consuming, because most supervisors do not spend
more than two hours at a site. A user-friendly, rapid
assessment method may be necessary.
There is no case: When supervisors arrive, there is no
case present that corresponds to the checklist. This
surprised the evaluation team: The only situation
described in the checklists is the care for the sick child,
which is about 50 percent of consultations.
They don’t need to observe directly: Supervisors
believe they can identify poor performance without
observation. As one said, “It does not take too much to
recognize poor performance.”
The evaluation team believes that the real reasons may
be different and that supervisors were not willing to
observe the delivery of care. The team hypothesizes that
supervisors: (a) do not feel competent in the technical
service they would observe, because they are not clini￾cians, (b) are uncomfortable observing a peer, (c) are
aware of the performance gap but do not know how to
address it, or (d) think that staff is competent and poor
quality is caused only by a lack of resources that they
cannot provide. More qualitative research would
answer some of these questions and is called for in the
recommendations.
The Costs of Performance Monitoring
Regional variations of supervision costs are enormous.
All regional directorates and DHOs could provide
estimates of the costs of carrying out performance
audits as well as supervision tours (see Table 4-6). The
managers’ ability to answer immediately all questions
regarding the costs was impressive, probably a result of
the emphasis put on financial accountability and bud￾geting by the health sector reform.
Variations in the costs of supervision visits relate to the
distances to health centers. Unit costs per HC range
from 14,210 K to 182,000 K. The main cost elements
of both performance audits and supervision visits are
the transportation (fuel) and staff allowances, which
depend on the number of days and therefore time spent
at each HC.
Table 46
Costs of Performance Monitoring for Regions
and Districts
Performance Supervision: Cost
Audits: Cost of One Visit of All
per Audit Health Centers
North Western 14,000,000 K
Regional Directorate (US$ 7,000)
Kalulushi DHO 700,000 K
(7 health centers) (US$ 350)
Lufwanyama DHO 850,000K
(13 health centers) (US$ 425)
Kitwe DHO 270,000 K
(19 health centers) (US$ 135)
North Central Regional 12,000,000 K
Directorate (US$ 6,000)
Kabwe DHO 400,000 K
(13 health centers) (US$ 200)
Monze DHO 750,000 K
(14 health centers) (US$375)
Lusaka DHO 4,000,000 K
(22 health centers) (US$2,000)
17
Main Recommendations Regarding the Assessment
of Quality Performance
The assessment of quality performance should include
the measurement of health workers’ compliance with
process standards through direct observation of the
delivery of care and not be focused only on input and
outcome standards. Performance audits and supervision
visits represent good opportunities to observe clinical
performance and provide on-the-job training.
A specific strategy should be defined to explore
whether poor performance is caused by a competency
problem or something else. Clinical competency should
be tested regularly to identify training needs. Other
performance deficiencies might better be addressed
through problem solving.
A formal recognition and reward system based on qual￾ity performance should be established to create incen￾tives to improve quality. The kind of reward that would
be both effective and acceptable to district health boards
should be identified.
B. Evaluation of Problem-Solving
Teams
A problem-solving team is two or more people meeting
to identify and solve a quality problem by working
through a series of steps (the problem-solving cycle)
and using simple QA methods. The evaluation team
investigated 25 such teams in HCs in eight districts
(See Table 4-7). The data-gathering methods were:
Table 47
Number of Teams Formed in Districts Visited and Number of Problem-Solving Cycles Completed
Site/Region/ District Health Center/ Finished Started Finished
[Number of Health Centers] Number of Team First Second More Than
(Population) Staff Started Cycle Cycle 3 Cycles
Site 1/ Kitwe Chimwemwe/50 Yes Yes No
[19] Ndeke/42 Yes Yes No
(Urban: 460,000) City Square Yes Yes No
Lufwanyama Chati/5 Team formed
[13] for training
(Rural: 61,000) but stopped
Kalulushi Government clinic/26 Yes Yes
[14: 7 public; 7 private] Chambishi Yes No
Chimbuluma Yes No
Mphongwe St. Theresa hospital/ 46 Yes No
(Rural) (not considered in our analysis)
Site 2/Kabwe Nakoli/15 Yes Yes Yes Yes
[13] Mahatma Gandhi Yes Yes Yes
(Urban: 214,000) Makululu/11 Yes No No
Bwacha Yes Yes No
Kapiri Center 1 Yes Yes
[18] Center 2 Yes Yes
(Urban: 228,000) Center 3 Yes Yes
Chibombo Chisamba/9 Yes Yes No
[23: 21 public; 2 private] Chibombo/9 Yes Yes No
(Rural: 211,000) Chipembi/14 Yes Yes No
Kayosha/5 Yes Yes No
Site 3/Lusaka Kalingalinga/45 Yes No Yes No
[22 public] Chawama/97 Yes No Yes
(Urban: 1 million) Kamwala Yes Yes Yes Yes
George/77 Yes Yes Yes
Civic Center/21 Yes No Yes
Site 4/Monze Rusangu/5 Yes No
[14 public; 1 mission] Manungu/9 Yes No
(Rural: 206,000)
TOTAL 127 public health centers in N=25 N=16 N=6 N=3
 the districts visited (20% of 127) (13% of 127) (5% of 127) (2% of 127)
18
■ A semistructured questionnaire to guide interviews
with team members, coaches, and others (Appendix
D), including methods for evaluating how a team
used quality methods and for scoring their difficulties
and calculating each team’s “difficulty index”
■ Predictive testing of conditions for successful teams,
summarized in a “risk of team failure index” with a
scoring method
■ Analyses of team storybooks and other documents,
such as the CBoH “link reports”
Number of Quality Problem-Solving Teams
Of the HCs visited, 27 percent have a functional quality
improvement team. In addition, there were reports that
out of 127 HCs in the eight districts visited, 34 teams
(27 percent) were formed between 1996 and 1998.
Among the 34 teams, 8 (23 percent) had stopped
meeting before the evaluation.
Among the 26 active teams, eight (35 percent) did not
finish their first problem-solving cycle; most were
coach training sites and stopped after the coach’s train￾ing. Three teams had completed more than two cycles.
The team is not sure that its findings can be generalized,
but if these figures represent all of Zambia, there may
have been 150 problem-solving teams between 1996
and 1998. A February 1998 quarterly link meeting
report (Central Board of Health 1998c) shows a total
of 225 teams, but comparing the eight districts visited
with findings from these districts indicates that about
40 percent of teams in the report were actually using
team problem solving. Using both the link reports and
the team’s evidence, the total was probably between 70
and 100 in late 1997.
Why Teams Do Not Form
The team identified five predictive factors for team
formation. The main reason for not forming a team after
training appears to be lack of access to coaching sup￾port. The evaluation team believes that, in the districts
visited, about 60 people had received training in team
problem solving, ranging from one- or two-day training
by a coach at a health center to the full 14-day coach
training. Many still lack the motivation and confidence
to form a team or the skills to carry out the steps
properly.
The team found that centers with at least 10 staff were
most successful in forming and continuing. They found
only one team in a health center with five or fewer staff,
and this team was not sustained or sustainable. It had
been a coach training site.
Of the 127 health centers visited, those with a team had:
(a) 10 or more staff (unless it was a training site), (b) a
coach on the team or visiting at least once a month and
for every meeting of the first problem cycle, (c) an
officer in charge trained and on the team or actively
supporting its work, (d) more than three people on the
team, (e) at least five days’ training represented on the
team (e.g., one person with five days’ training or five
with one day’s training), and (f) reasonable morale and
a culture of professionalism. The team urges that
research determine whether these factors consistently
predict team formation.
Successfully Completing a Problem-Solving Cycle
The team identified additional factors necessary for
teams to successfully complete a problem cycle,
including: (a) retention at the health center of at least 50
percent of team members, (b) meetings at least monthly
and no gaps over two months, and the team had: (c)
chosen a problem that was not broad or complex, (d)
precisely defined a problem statement, and (e) spent
over four months on one step and followed the steps of
the cycle.
To finish a second cycle, it appears that all the above
factors had to exist and the team had to have achieved
perceptible or measurable results.
For a team to follow the cycle steps and use the meth￾ods correctly, all these conditions would have to exist
and the everyday workload of team members could not
have increased considerably for over two months.
The predictive value of these factors for success in
completing the problem-solving cycle remains to be
tested through research. If these findings are valid for
other areas and while resources are scarce, it is ques￾tionable whether teams should be formed where these
conditions are absent, as the risk of failure appears to be
high. As part of its process the team constructed a “risk
of failure index” (see Appendix D). The team believes
these factors interrelate but was unable to investigate
Much of the work of quality problem￾solving teams stagnated after people
started going for interviews as part
of de-linkage. People were not sure
where they would be employed and
did not see the point of gathering
data and working on the problems.
(Coach)
19
how. (For example, Would more training compensate
for less coach support?) The team was also unable to
study whether teams are successful where there is
already relatively higher quality and staff competence.
Types and Relevance of the Problems Identified by
the Teams
Most quality improvement teams worked on meaningful
problems. The evaluation team assessed the list of
possible problems and how they were prioritized by
using teams’ documentation or asking coaches and team
members how they chose problems. For districts not
visited, they assessed 49 reports in the storybooks.
There was also some documentation of the teams’ work
in trip reports by CHS staff. Where teams had docu￾mentation (e.g., the standard QA notebook or written
notes from a team), it was easier to assess the relevance
of the problem. About 10 of the 25 teams did not have
documentation, making it difficult to determine how
well teams followed the steps.
The relevance and importance of the chosen problems
for patients, community, and staff at the centers varied
among the teams. About 50 percent chose physical and
facility problems, and about 40 percent chose clinical
problems.
The QA training heavily influenced the choice of the
problems, which included:
■ Long waiting times (Chimwemwe, Ndeke,
Government Clinic, Chambishi, and Chawama)
■ Careless disposal of sharps (City Square)
■ Rise in number of malaria cases (problem cycle 1),
Scabies (cycle 2), low patient-fee collection (cycle 3),
congestion at tea time (cycle 4), staff reporting late
(cycle 5), low immunization rates (cycle 6) (Nakoli)
■ Long waiting times due to staff arriving late for work
(Mahatma Gandhi)
■ Long waiting times and congestion at the outpatient
and maternal and child health departments
(Makululu)
■ Low postnatal attendance (Bwacha)
■ Rise in number of malaria cases (Centre 1)
■ Rise in number of malnutrition cases (Centre 2)
■ Reducing pneumonia rates (Centre 3)
■ Need for a shelter for mothers (Chisamba)
■ Need to extend the rural health center (Chibombo)
■ Low attendance of family planning clients
(Chipembi)
■ No regular fresh water (Kayosha)
■ Late antenatal booking (Kalingalinga)
■ Nurses not competent in treating TB cases (cycle 1),
increase in malaria and diarrhea (cycle 2), shortage
of casual workers (cycle 3), lack of discipline and
absenteeism (cycle 4) (Kamwala)
■ Stolen manhole covers and blocked sewer line and
ceased soakerway (George)
■ Antenatal booking late by more than 12 weeks
(Civic Center)
■ No staff housing near the center, low immunization
coverage (Rusangu)
■ Inadequate maintenance (cycle 1), rise in malnutrition
cases (cycle 2), long waits (cycle 3), inadequate
maintenance of water supply (cycle 4), patient
records missing (cycle 5), (Manungu: Team may have
addressed all these at once and become stuck)
Reducing waiting times was chosen often, in part
because it was used as an example in training. The
clinical problems (e.g., malaria, late antenatal booking)
were relevant and important for the health of patients
and/or the community and were usually chosen because
staff noticed a rise in cases rather than because they
studied statistical trends for the center. About 5 percent
of the problems were identified by soliciting the views
of the users or neighborhood.
Following the Steps of the Problem-Solving Cycle
and Choosing and Using Methods
Most teams correctly used the QA tools and methods,
such as problem prioritizing and cause and effect
diagrams. The use of methods for listing data sources
(data matrix) and for gathering and interpreting data
could have been better, and about 40 percent of teams
did not gather data. The most serious divergences were
not following the steps of the cycle when it was neces￾sary to do so and failing to use the conclusions of a
previous step as a basis for the next one. Some teams
did not use data interpretation to list solutions, but
instead moved on to brainstorming a solution.
Data collection and following all steps pose the main
challenges to teams. Nine teams did not finish their first
cycle. Some did not carry out the steps properly or in
sequence; others became stuck and did not have access
to a coach. Teams were better at following the steps
20
Case Study Number 4:
The Nakoli Health Center (Kabwe District)
Problem Solving
Nakoli health center is a semi-urban clinic with a staff
of 15, all women. It seemed cheerful and well orga￾nized to the evaluation team. In June 1996, the sister
in charge took part in a local five-day training run
by the regional link. The training was on the quality
problem-solving cycle and teamwork, with one day on
standard setting. The sister used notes from the
course to teach all HC staff for two hours for 16 weeks.
From the start she decided that all the staff would
take the training, and she decided that the training
meetings would become team problem-solving meet￾ings. They started on the first problem in late June
1996, and by September 1998 had worked on five
problems and were starting on a sixth. They worked
on the first problema rise in malaria casesfor 15
months. The QA notebook documentation was not
available to the evaluation team for any of the
problems because they had only one copy, but the
problems and steps were documented in exercise
books. When they first met, they identified five prob￾lems: a rise in the incidence of malaria (which they
scored 25), upper respiratory tract infection (24),
diarrhea (23), eye infections (21), and skin diseases
(17). They used the criteria for scoring each problem
correctly and decided to work on the problem with the
highest score: malaria. They stated the problem as,
Nakoli health center recorded increases in the
number of malaria cases, with the lowest figure in
1995 being 163, to 242 in 1996. A rise in new cases
has been noticed since 1995 and has resulted in
increasing morbidity. The improvement should result
in morbidity rates reduced to at least 100 new cases.
The statement used measures, had a target, and
chose to focus on treatment rather than prevention
because the team had more influence over treatment
than prevention.
In common with many other teams, Nakoli did not
change the membership of the team in step 3,
Identifying who needs to work on the problem. The
complex step 4, Analyzing and studying the problem
to identify major causes, consumed time and was
where the team encountered problems. They used the
bubble chart to show possible causes of the problem.
They decided to focus on why people appeared to be
coming back to the clinic with symptoms after being
seen only a few days before. They did a high-level flow
chart of the treatment process and then identified
data they needed to find the main causes of malaria
morbidity. They interviewed 43 patients who came
back within five days with the same symptoms, and
the data showed that most had not completed their
full course of chloroquine. They did not question the
data further to understand why, but went straight to
step 5, Developing and choosing solutions. Like
many other teams, they used brainstorming to list
solutions and then choose among them, and chose
to ask patients to come back every day for a super￾vised drug treatment. They set up a monitoring
system and set standards for return treatments.
The team had established a treatment supervision
system that monitored the number of people super￾vised and the number with symptoms. The data show
that 80 percent in June and 90 percent in July com￾pleted treatment, indicating that the intervention did
have some effect, and that 13 percent in June and 7
percent in July returned within five days with same
complaint.
While looking at solutions to the treatment problem,
they decided to broaden the work to include preven￾tion and went straight to brainstorming ways to
reduce the incidence of malaria cases. They worked
with the community to find stagnant pools and
increase the use of impregnated bed nets. The statis￾tical data showed that the six-month mean cases for
January to June 1996 were 496 and for January to
June 1997, after the preventative efforts, the case
rate was 370. However, at the time of the evaluation,
they had discontinued monitoring the malaria rates,
and the evaluation team calculated that the mean
case rates for January to June 1998 was 726, a
significant rise.
This team was unusual in continuing to work on other
problems: scabies (four months), low fee collection
(one month to solve), congestion at tea time (two
weeks without using the problem-solving cycle), staff
reporting late (one week without using the problem￾solving cycle), and low immunization rates. They were
also unusual in deciding appropriately when to use
the problem-solving cycle and when not to. These prob￾lems were congestion at clinic at 11 am because all
staff were going for tea at the same time (solution:
some staff going for a 10:30 break and some for an
11:00 break), and reporting late for work (solution:
reporting-in book). The evaluation team asked why they
had not stopped the quality work at the time of
de-linkage, like many other teams. They admitted
that some staff was unmotivated, but they had
decided to start another problem-cycle to show
what women can do.
21
where the officer in charge had had coach training or
where the team had been a coach’s training site. The
following assessment covers both teams that had
difficulties and those that completed the cycle.
The evaluation team used the “quality team evaluation
method,” which is part of the semistructured interview
guide (see Appendix D). It scored how well each team
both followed each step and used the method within it.
The scores, presented below, are an average for all of
the 25 centers studied. The team used criteria to give a
score of between 0 and 5 for each item (0 = did not do
it, 5 = could not imagine how to do it better). Where
possible, the team negotiated a score with the problem￾solving team members who were interviewed by
together studying the documentation and asking
questions. Although the teams were trained to follow
the five-step cycle, six are described here. To simplify
the model, step 5 of the original model is divided into
two steps, identified below as steps 5 and 6.
After discussing the criteria for scoring, the evaluation
team found that problem-solving team members tended
to score their work slightly lower than the evaluators
did and did not make the slight allowances the evalua￾tors did for such things as the problem being the first or
little training or coach support. Several team members
and coaches asked for copies of the evaluation team’s
assessment to review their own or another team’s work.
They confirmed the evaluation team’s impression that
no evaluation was done of the teams’ work to learn how
to improve and that there was no system for doing this.
Step 1: Identifying the Problem—
Average Score 3/Range 3–5
1) Listing—Average score of 3: Most teams scored
fairly well on this step because they had considered
a reasonable number of problems: between four and
eight. However, the evaluators could find only four
teams that had consciously included problems that
were of concern to NHCs, and only two had
conducted a user survey.
2) Prioritizing matrix and vote using criteria—
Average score of 4: Ninety percent used the matrix
to prioritize the problems they had identified and
used the prioritizing criteria reasonably well to
score each problem. However, about 30 percent
chose problems that were not the highest priority on
their list. It might be that the team rightly chose a
problem perceived as solvable, rather than one with
a higher score but perceived as too difficult, i.e., the
prioritizing criteria they used may have been wrong.
The evaluators did not verify this. In some cases the
team chose a lower-priority problem because it was
more important to members.
Four teams inappropriately chose to work on more
than one problem at once. One team appeared to
have worked on five at once, with five problem
statements listed together in the storybook as well
as five flow charts. The evaluators were not able to
verify this. Two teams became stuck in later steps
and started on their second problem before finishing
the first.
Step 2: Problem Statement—
Average score of 2/Range 0–4
3) Method for writing a precise statement with
measures and a specific target: Most of the
problem statements followed the guidelines for the
problem statement reasonably well, although about
50 percent did not state measures or a target in
numerical terms. This made it difficult for the team
to evaluate the impact of any solution they may
have implemented.
Step 3: Identifying Who Needs to Work on the
Problem—Average score of 2/Range 0–3
4) Method for selecting the right people for team
or for involving them in other ways to analyze,
collect data, and solve the problem: The evaluation
team found only two teams that had changed the
membership of the team after choosing their prob￾lem: Kalingalinga and Chibombo. Most, however,
had identified others outside the team who should
be involved and had involved them in data gather￾ing or solution identification and implementation.
Step 4: Analyzing and Studying the Problem to
Identify Major Causes—Average Score 1/
Range 0–3
This step is the most complex and was least well
followed. It includes the use of the following methods
for most problems:
5) Bubble chart or other cause-effect diagram
6) Listing of possible causes
7) Creation of data matrix with data gathering
questions, sources, and methods
8) Data gathered and recorded (quantitative or
qualitative)
9) Validity and reliability of data for answering the
problem (e.g., sample for interviews, trending
quantitative data, or validity of statistics)
22
10) Data analysis and interpretation
11) Use of data to determine root cause
Most teams did not follow the sequence or use the
methods correctly, resulting in the low score. In
general the cause-and-effect listing was done well,
but most teams either did not gather data or did not
correctly decide which data to gather to be certain
about the real cause. When they did correctly
decide which data to gather, they did not use it to
find the real root cause.
Step 5: Developing and Choosing Solution—
Average score of 2/Range 0–3
12) List of possible solutions: Most teams listed and
assessed solutions, but not systematically.
13) Link between solutions and the use of data: A low
score for this step resulted because nearly all teams
failed to gather data that they might have used to
decide the real root cause.
14) Choice of solution: What was the method used
and how good was the choice? Most teams used
objective criteria to choose between possible solu￾tions and did follow the methods taught in training.
Step 6: Implementation and Evaluation of Quality
Improvement Work—Average score of 2/
Range 0–3
15) How well was the solution implemented (plan and
persistence)? Most teams did not plan the imple￾mentation of a solution in detail, and the evaluation
team saw only five written records of the Plan￾Develop-Check-Act cycle used for this purpose.
16) Were data gathered again to check the effects of
the solution? Few teams carried out before-and￾after intervention measures, and for those that did,
the validity of the comparisons was questionable.
17) What difference did the solution make and to
whom (from data or judgement)? Without data in
most cases, the evaluation team had to judge the
results and ask team members and others their
judgements. Many did not evaluate their work in
any systematic way.
18) How well did the team follow up six to 12 months
later or keep monitoring? The evaluation team
found no problem-solving team that performed
follow-up, although a few did set standards and
quality indicators and monitor these routinely.
Difficulties the Teams Met in Following the Steps
and Using the Methods
Teams experienced most difficulties collecting and
using data to identify root causes. Teams of four or
more, in centers with over 10 staff, with an able coach
on the team or regularly available, and with reference
and training materials, did not encounter serious prob￾lems. This was especially so if they were visited by link
facilitators, even if they were affected by morale prob￾lems. Other teams experienced the most difficulties in
getting access to a coach; getting quality reference
materials and keeping them at the center and available
to all; and in using certain methods, such as deciding
which data to gather and then gathering reliable and
valid data. Many teams had problems implementing
solutions, usually due to lack of resources (e.g., no
drugs). In some cases this difficulty was because the
problem selection method was not used properly to
identify a problem solvable by the team. Often the
teams recognized they were having difficulties and did
ask for a coach visit; a significant number had stopped
working and were “waiting for the coach to visit.”
If a team is working on reducing
malaria morbidity and collecting data,
it is de-motivating when drugs run
out just as they are beginning to see
a downward trend. Drugs are
centrally supplied, but they (drug
stock suppliers) do not have a quality
program.
(Coach)
The evaluation team asked team members and coaches
about any difficulties they experienced using the steps
and methods. To do this, they used the same list and
scoring as the one described above (Appendix D). Each
item was given a “difficulty score” of 0 to 5 for the
level of difficulty experienced and a score of 10 for the
team reporting that the step or method was so difficult
that they were not able to do it. The evaluation team
termed the total score the team’s “difficulty index.”
Of the teams that started using the problem-solving
cycle, about half had followed each step in the correct
23
sequence. Of those that did not follow the steps, most
missed data gathering, often because of difficulties
creating the data matrix and then gathering data
(discussed below).
One frequent “linking failure” was teams’ not using the
possible causes list (item 6, step 4) to decide which data
to collect (item 7, step 4). The most common and seri￾ous “linking failure” was failing to use the data interpre￾tation (end of step 4) to decide the list of solutions (step
5). For the few teams that reached this stage, most
tended to brainstorm solutions without considering the
data they had collected. Many seem to have forgotten or
never recognized that the purpose of data collection is
to confirm the “real root causes.” This may be because
step 5 is long and complex. The team believes that
consideration should be given to dividing this step into
cause analysis, and data gathering and analysis.
Some teams’ difficulties are explained by lack of coach
support. Nearly all of these difficulties could be over￾come by frequent visits by capable and trained coaches.
The team estimates that 40 percent of the centers with
teams did not have a coach on the team or frequent
visits from one, and this is the single most important
factor causing teams to get stuck or waste time.
Team Performance in Gathering and Using Data
Some teams tried to collect and use data; about 60
percent actually performed data gathering. Of these,
some used qualitative data from interviews (e.g., of
patients). Most used quantitative data from routine
statistics or from their own special methods (e.g., for
waiting times, some used a note attached to the medical
record that the patient carried in the “journey” through
the HC. The staff noted departure and arrival times at
stops along the journey).
About 50 percent of the teams who collected data
encountered problems, making it difficult to use the
data to select a solution. Most did not have reference
material about how to gather data, and many links and
some coaches only had the Franco et al. manual
(Franco et al. 1997), which does not describe the data
matrix with data questions that some teams had used.
Most teams that tried collecting data had difficulties
completing the data matrix, especially in deciding
which data gathering method or source to use. There
were problems in using routine statistics, such as
unavailability of a full time set or because population
denominators for indicators were from census statistics
that underestimated the population size. In the latter
cases, the team’s work was often delayed by having to
organize head counts to get more reliable data. One link
reported that the census data for the district showed
215,000, but the head count found 365,500.
Problems in data collection and use were the most
serious deficiencies observed. In most cases, not
collecting data made it difficult for teams to identify
real root causes. This made it difficult for them and the
team to objectively evaluate the results of their solution
interventions.
Mastering the use of data and QA tools would improve
with appropriate coach support. These difficulties and
our assessment of the use of the methods show that
there is room for improving the competence of team
members to use and choose the methods. Competence
levels should be improved in following and linking the
steps of the cycle. This involves fully understanding
why the steps should be followed and being able to
decide when to diverge from them. Competence in data
gathering and analysis should also be improved.
There are different ways to raise competence, but the
most effective in Zambia appeared to be “learning by
doing” when coaches visited to help a team work
through the cycle.
Results and Progress of the Teams According to the
Self-Monitoring
The work of some teams was documented through
the report of the link facilitators to the CBoH and put
together in tables such as those in Appendix E. Those
that did not document their results were assessed in
terms of their ability to: (a) define a solution, (b) fully
implement a solution, (c) show evidence of an impact
on the problem (e.g., before-and-after measures), or (d)
report that they thought they had solved the problem.
Five teams achieved measurable changes in quality,
and about eight teams reported that they had achieved
significant quality improvements but did not have data
to show it. About 60 percent of the problem-solving
cycles did not produce results that were perceivable
within six months or that could be attributed solely to
the team’s intervention. In some cases the problems
chosen could be only partially solved by the team.
Because of the lack of documentation, the evaluation
team could not find many results from problem-solving
efforts, but this does not mean that the teams’ work was
24
ineffective or without benefit. It was also difficult to
determine whether results were due to the teams’ efforts
or to other factors.
The side benefits of teamwork went beyond solving a
problem. There were many indirect benefits, where
people used their quality skills for other purposes. An
example was the use of problem identification by
health centers and districts to set their action plans and
budgets. Other benefits are below in the summary of the
strengths of the teams and their work.
Duration of the Problem-Solving Cycle and How It
Could Be Shortened
It took the teams a long time to finish the first cycle.
The average duration of the first cycle for teams that
met monthly was from less than one month to 12
months. This dropped to seven months for the second
and subsequent cycles. Only six teams completed more
than one cycle, and the duration depended on the type
of problem, so these figures should not be generalized.
The cycle time might be reduced through refining the
problem-solving methodology. Feasible measures to
reduce the cycle time include: teams gaining skills in
deciding which problems are solvable and focusing on
them, regular coach visits or access to a coach’s advice,
more training on how the steps of the cycle link to each
other and when to use simple methods, training on how
to use existing protocols and standards as a solution,
and training and support on how to use already
collected data (e.g., HMIS) rather than carrying out
special data collection.
The cost-benefit of team-based problem solving
should be studied. The evaluation team did not make a
systematic cost-benefit assessment of the teams’ work.
Such an assessment would determine how best to use
scarce resources. The evaluation team estimates that an
average team, meeting monthly for two hours with eight
staff on the team, “costs” 192 hours or four person￾weeks. The findings show that benefits of this time, in
terms of changes to quality, are not startling and often
not provable, although there are a few exceptions.
However, the indirect benefits are significant and are
noted below under “strengths.” If both indirect and
direct benefits are counted as the “return on investment”
of four person weeks, then the cost-benefit could be
judged as “reasonable” and has the potential to be much
better, especially if teams were to focus on waste
problems.
Cost-benefit, however, is judged in relation to alterna￾tive uses of time. An important question is, What is the
cost-benefit of problem-solving teams compared to the
cost-benefit of other quality methods or other uses of
time (e.g., developing diagnosis and treatment skills)?
Further research would clarify when the problem￾solving method is best.
Strengths and Weaknesses of the Problem-Solving
Teams
Side benefits to the problem-solving efforts are numer￾ous. Although evidence of measured improvement to
quality was scarce, several direct and indirect benefits
emerged from the teams and their work, including:
■ Some teams’ achieved measurable improvements to
community health (e.g., lower malaria incidence),
clinical care process (e.g.., increased immunization),
and patient quality (e.g., reducing waiting time from
two hours to 50 minutes)
■ Reported improvements to facility quality and patient
or professional quality
■ The problem-solving model being used as a simple
and effective way to use quality methods and to
combine standards setting and compliance with
process improvement
■ Team building in centers where cross-disciplinary
cooperation had been weak
■ In the larger centers many staff reporting interest in
the team’s work and very cooperative when asked to
contribute, spreading quality ideas in everyday work
■ Increasing the general competence and confidence of
staff and enabling them to use a systematic approach
to solving problems, which they applied in many
other situations
■ Providing an outlet for frustrated and unrecognized
staff potential (many able staff were attracted to
working on the teams or to coaching, providing an
opportunity for professional development not
otherwise available to them)
■ Capacity building and preparation for management
positions
Team experience and the methods are an excellent
practical training for management
Main Constraints to Teamwork
A consideration of weaknesses also highlights some of
the constraints to the teams’ work. These included:
25
■ Loss of staff from the teams and coaches due to
promotion and transfers, and no system for ensuring
sustainability and development of teams
■ Many not following the methods and steps when
necessary, usually because of inadequate training
and/or lack of coach support
■ Many not being able to define measures in the prob￾lem statement and to use data to determine root
causes, as well as not following up after the solution
intervention to check the results and “hold the gains”
■ The length of time to complete a cycle, even after the
first “training cycle” (average of nine months), espe￾cially for teams meeting monthly (most teams) rather
than weekly
■ Not developing the skills to decide which problems
need team problem solving and the full cycle and
which do not (Example: Kitwe took a year to identify
and implement a solution to collect disposed sharps)
■ The high failure rate of teams, in terms of not form￾ing after a member or officer in charge was trained
(about 40 percent of officers in charge from 127
centers were trained), not finishing the first cycle (9
[36 percent] that started did not finish), and not con￾tinuing on further problems (19 [75 percent] did not).
C. Evaluation of the QA Support
Systems
In the context of the Zambian health sector reforms,
QA activities are being implemented within an inte￾grated framework for service delivery. As a result,
training, supervision, monitoring, and other systems
that support the delivery of essential health services
have a QA component. For example, staff at various
levels of the health system have been trained with vary￾ing levels of intensity in the methods and principles of
QA as part of a broader effort to build capacity at the
local level for ensuring quality of care. Reporting
systems are in place to measure performance and
facilitate more efficient and complete documentation of
QA activities. Existing forms of technical supervision
provide a way to identify quality of care problems and
monitor the effects of QA activities. Finally, the net￾work of trained QA coaches and link facilitators sup￾ports all these systems by providing training and techni￾cal assistance at the district and HC levels. This section
reviews QA training, the QA coaches/link facilitators
network, documentation and reporting of QA activities,
and supervision of quality assurance activities.
1. Training in Quality Assurance
The formal agreement to introduce quality assurance
led to the formation of the QA Unit within the MOH
and the start of QA training activities. Training was
given first to key provincial level staff in all provinces,
then at district and facility levels in three pilot districts.
Initial training in the target districts focused on the
setting and monitoring of standards and was later ex￾panded to include problem solving. QA training varied
by district, but generally consisted of sensitization
workshops (one day to introduce the concepts of quality
assurance), a week-long training in DySSSy, a five-day
training in the development of monitoring indicators,
and 14 days in using QA tools and techniques.
QA concepts and methods were introduced in some
form throughout all provinces before the evaluation,
and only North Western Province had not received the
problem solving training.
QA Training Capacity and Experience at Various
Levels of the System
The QAP has trained many staff in QA, but they are
unevenly distributed.
At the central level, the Directorate of Monitoring and
Evaluation is responsible for ensuring that all levels of
healthcare are sensitized to the concept of QA. Their
role is that of “facilitators capacitating health providers
to self-assess, to measure their performance and compli￾ance to agreed standards, and to respond to clients/users
needs.” (Limbambala and Tembo 1996) This unit
provides training and technical oversight in QA to the
DHMTs and their health centers.
At the regional level, no special training has been given
to staff since the formation of the regional directorates
(one staff member at North Central Region had
received some QA training while at a provincial level
post). The CBoH’s intent had been to provide training
to the regional office, but the ban on workshops, delays
in appointing regional office staff, and the introduction
of hospital accreditation activities may have combined
to preclude such training.
Though training in general does appear to be a function
of the regional office, the position of training specialist
was unfilled. The interest of regional staff is such, how￾ever, that QA would reportedly be included in planning
for 1999 activities.
At the district level, staff have received varying degrees
of training. Though few directors, if any, received any
formal QA training, several DHMT staff have been
26
trained as QA coaches or facilitators or otherwise par￾ticipated in some form of QA training. See Table 4–8
for examples.
In one instance (Kitwe), administrative staff had been
given specific training in standards setting in order to
improve record-keeping practices. Lusaka staff reported
receiving some introduction to QA (or at least similar
concepts) as a part of general district-level, capacity￾building training. Capacity-building sessions were
conducted to create effective leadership, accountability
(including financial management as well as QA
systems), and community partnership within the newly
decentralized districts (Limbambala and Tembo
Undated). When asked about the types of QA training
received, only the Lusaka DHMT said that QA was
introduced during the capacity-building training.
There is no formal system for identifying QA training
needs. At the district office, responsibility for coordina￾tion of training activities (including QA) varied accord￾ing to the composition of the DHMT, though it was
usually said to be the responsibility of the Manager for
Planning and Development. At one district (Kalulushi),
selection and prioritization of training was said to be the
responsibility of an in-service coordinator. In other
districts, requests for training were approved on an ad
hoc basis. Though each district is supposed to form a
staff development committee to identify training needs
and conduct needs assessment, reportedly few have
been formed. It remains unclear how training needs in
general would be identified, though QA would presum￾ably be given some priority.
The identification of training needs is further compli￾cated by the replacement of previous DHMT staff as a
result of de-linkage. Senior staff at this level are
recruited on a contractual basis, as opposed to the
“permanent and pensionable condition of service”
(MOH 1996b) in effect prior to the reforms. This means
that newly hired senior DHMT staff need QA
orientation.
Concern for quality at the district office is such that QA
training activities were budgeted in 1998 action plans,
but they were dropped in some districts due to funding
problems and/or the suspension of training workshops.
Most district teams interviewed hoped to include QA in
their 1999 plans.
At the HC level, the extent of QA training varied by
district. For example, some Ndeke Clinic staff had
received a three-day QA training; all health centers in
Kalulushi District had from one to three staff trained in
QA; in Kitwe, HCs reportedly had an average of three
days of training (some had five days; all the in-charges
received sensitization training); in Kabwe District, all
in-charges in all health centers had been given a five￾day training plus a two-day practicum; in Monze
District, all HCs reportedly had staff trained to serve
as “QA Committees.” It would seem that no uniform
approach to training was adopted at the HC level, but
rather facilitators and coaches devised training plans to
suit their circumstances.
Training needs at the facility level, including selection
of staff for QA training, are reportedly determined
through supervision and on-the-job performance. The
general consensus among the HC staff interviewed was
that more thorough “refresher” training in QA was
needed, particularly given the constraints (below).
Main Constraints to the Effective Delivery of QA
Training
Funding and link facilitators’ time are the main limiting
factors to QA training. HC and district staff and coaches
and link facilitators identified the following:
Priorities: Frequent and recurring drug shortages are
often seen as the priority problem, undermining all
other efforts to provide adequate health services.
Coaches/link facilitators have difficulties discussing
quality of care issues (and thus the need for a QA
approach) when HCs lack such basics.
Funding: Training is always a lower priority when
resources are lacking, and districts reported that in the
first nine months of 1998, they received less than 50
percent of the amount budgeted.
Table 48
Examples of DHMT Training in Quality
Assurance
District QA Coach/ Position/Title
Link DHMT
Kitwe Link HC representative; Ndeke clinic
Coach Manager, Planning and Development
Kabwe Link Acting Manager, Planning and
Development
Other DHMT staff sensitized
Kapiri Coach Acting Manager, Planning and
Mposhi Development
Lufanyama Coach Health Information Officer
Lusaka Link Health Nurse, MCH coordinator
Monze Manager, Planning and Development
Health Information Officer
Other DHMT staff sensitized
Choma Link Health Information Officer
27
Case Study Number 5:
Supporting QA Teams: The Work of a Link
Facilitator
The link facilitator (link) has played an integral role
in the development and success of the Zambia QAP.
To strengthen communications and teamwork between
the QA structures at the central, provincial, district, and
facility levels, QA coaching participants who demon￾strated the best aptitude and enthusiasm were
selected to be links. These health staff were expected
to develop a strategy for QA in their respective districts
that would include the sensitization of district level staff
as well as the coaching of problem-solving teams at
the health centers. Despite the many difficulties faced
as a result of the health sector reforms (i.e., the
de-linkage of staff, recurrent drug shortages, and
severe funding constraints), several links were unde￾terred in their efforts to implement QA activities. In
Kitwe District, for example, a link not only managed to
create and support several QA teams but has infused
QA concepts into all aspects of his work as well as
that of his many colleagues. He experienced the
following training regimen:
Phase I: Five days’ sensitization, including basic
QA concepts, indicator development, and standard
setting
Phase II: Five days’ standards setting (DySSSy)
Phase III: 11 days’ problem solving and coaching,
including practice
As a best participant, he was chosen to be a link
after the Phase II training and invited back for Phase
III problem-solving training. After the initial training, he
underwent a series of validation checks where he was
observed and evaluated while training staff in other
locations. He joined other link facilitators at three
different provincial sites (Luapula, Copperbelt, and
Eastern Provinces) to train selected individuals from
the DHMTs and health centers. In Kitwe, he worked
primarily as a Clinical Officer at a health center, but
also served as health centers’ representative on the
Kitwe DHMT. He was well regarded within many
professional circles and thus well placed to introduce
QA into district as well as facility organizational struc￾tures. Within Kitwe, he provided QA training to nine
clinics, including eight that were reported to have
active teams. Some examples of the problems he
coached them to address were:
Table 49
Sample Link Facilitator Results
Location Topic Result
Chemwemwe HC Long waiting time in OPD Flowchart, patient￾flow analysis
Itimpi maternity Low delivery rates at facility Redefine problem,
confirm causes
Kamfinsa clinic Low delivery rates at facility Compare antenatal
visits and deliveries
Besides his work with the teams, caring for patients
at the clinic, and activities with the DHMT, the link
was also working to create other supportive mecha￾nisms to ensure that QA activities (and quality of care,
generally) could be effectively monitored. He provided
training to IMCI supervisors who were required to
monitor (and be monitored by) other IMCI health
centers. Staffs were given a three-day basic training
in QA and later instructed to check the teams’
storybooks during supervisory visits. Elsewhere, he
worked to develop a supervision checklist that could
be used by staff charged with monitoring health
centers as part of weekend coverage visits. Despite
the constraints on his time and the economic and
other extreme difficulties being faced by the district,
the link has made an admirable start with regard to
QA. Although it remains to be seen what the ultimate
outcome of this work will be, at the very least, the
enthusiasm and support for QA are ensured under
vigil of the Kitwe link facilitator.
Time: Most coaches and links are HC in-charges and
thus have little time to devote to training other HCs.
Transportation and distance: Lack of adequate trans￾portation prevents coaches and links from training and
supervising the HCs under their charge, especially the
more remote ones.
Turnover: Losses or changes in staff, in addition to
problems created by de-linkage, may require training or
retraining of relevant staff in QA, creating additional
burden for those who oversee training.
Leadership: Lack of motivation, initiative, and confi￾dence on the part of some coaches and link facilitators
28
may prevent the start-up of QA activities, despite the
confidence-building activities incorporated into the
training validation process.
Team Approach in Small Facilities: Lack of a QA
training approach appropriate for HCs with too few
staff and/or at more remote locations may require
rethinking of the existing training content. Currently, the
training emphasizes a team-based approach to problem
solving, which relies on frequent and supportive visiting
by coaches or facilitators.
2. Coaching/Link Facilitation of QA Teams
To strengthen linkages between the regions, districts,
and centers, the “best” training participants were chosen
to act as link facilitators/coaches. These staff were
expected to develop a roll-out strategy for QA in their
respective districts, including the training and coaching
of problem-solving teams at the health centers.
The Roles of Coaches/Link Facilitators and
Constraints to Adequate Coaching
The coaches/links play an important role in the success
of the QA teams. Coaching visits to the HCs are meant
to provide just-in-time training or technical guidance to
the teams and should include a review of the team’s
work, correction or rework if needed, and planning for
next steps. As reported by the HCs, however, most
coaching visits were not regularly planned and held.
In addition, some teams reported never having been
visited by a coach after initial training, despite having
completed several problem-solving cycles. Other HC
teams reported being visited by a link facilitator only
once or twice since being trained in 1996; still others
reported frequent (quarterly or more) contacts by
coaches/facilitators as part of the integrated supervisory
DHMT visit. In all these cases, it remained uncertain
how often coaching visits coincided with the team
meetings and, moreover, how timely the visits were in
addressing difficulties encountered by the teams.
Some link facilitators reported that the motivation
and ability of the teams to work through the problem￾solving process often appeared to be externally driven;
i.e., a visit by the coach or facilitator would prompt a
team to complete one or more steps in the cycle. In
Kitwe District, for example, the link facilitator reported
that his teams could not complete the steps involving
data collection and monitoring of results without his
persistent support and frequent visiting. Similarly, the
Kabwe link felt that several of his teams could not work
without a coach to motivate them to action.
Coaches and links face many constraints in providing
support to teams. Most coaches and link facilitators
who were interviewed generally perceived QA to be an
important and necessary part of the way work should be
done at the HCs, i.e., it offered “a new way of looking
at problems” that hadn’t existed before, “a way for
health centers to identify and resolve problems them￾selves,” and it provided a means for “coordinating how
problems should be solved.” However, several of the
coaches and links could not provide adequate support to
the teams. Some felt they had insufficient time for QA
activities, particularly as many were managers or held
other positions at the DHMT (some equally “busy”
DHMT staff could find the time to support their teams).
Some coaches were inactive because of a lack of fund￾ing or transportation for training and coaching visits.
An October 1997 link facilitators meeting recorded
some suggestions for improving the support to QA
coaches: (a) regular meetings to improve QA skills, (b)
having districts sponsor their coaches, (c) integrating
QA meetings with HMIS quarterly meetings at the
district level (not all areas have HMIS meetings), (d)
integrating QA meetings with managerial skills work￾shops (not all areas have managerial skills workshops),
(e) performing QA activities during supervisory visits,
(f) encouraging regions to support QA links in
supporting coaches, and (g) including link and
coaching activities in routine action plans.
Though local factors, such as staff de-linkage, drug
shortages, and funding problems, were commonly seen
as obstacles to QA activities, the degree of skills trans￾fer and coaching support seems to depend on: (a) the
ability and willingness of district staff to plan for QA,
(b) the resources committed to training and coaching,
and (c) the motivation of coaches and facilitators.
Preparation for QA Coaches and Link Facilitators
A typical QA training sequence for coaches and link
facilitators is: (Phase I) Five days of sensitization: basic
QA concepts, indicator development, basic standards
setting, (Phase II) Five days of standards setting
(DySSSy), and (Phase III) 11 days of problem solving
and coaching, including practice.
The “best” participants are chosen to be coaches or link
facilitators after Phase II training and are invited back
for Phase III problem-solving training. After the initial
training, participants undergo “validation” checks
where facilitators train staff in other districts or HCs or
coaches train two other HCs. CBoH QA staff evaluate
and re-evaluate participants to gauge their willingness
29
and ability to serve at the next level. Links and coaches
reported that this somewhat rigorous validation was
effective in ensuring that adequate QA technical skills
were acquired.
In addition to technical QA skills, the modular training
package for QA coaching (Quality Assurance Project
1996) includes such topics as the role of a QA coach,
team building and teamwork, communicating in teams,
organizing meetings, team decision making, and con￾flict management.
Supervision of Coaches/Links
Supervision of the link facilitators by the CBoH is
reportedly done through: (a) training activities, (b) visits
to the district with or without another link, (c) feedback
after observations, and (d) quarterly meetings for link
facilitators.
The manner and frequency of supervising coaches
differs from place to place. Some links are quite active
in contacting coaches, though this can be difficult for
those who have more than one district (e.g., Choma).
Other ways to monitor coaches are through meetings of
coaches at the district level (e.g., Kabwe) and through
meetings at the province level of district coaches.
Maintaining QA Skills and Knowledge
Apart from the supervision described above, coaching
skills are maintained through learning by doing, valida￾tion courses in other districts, quarterly meetings of link
facilitators, and random checks on the work of coaches
and teams by central staff. In Kabwe, the link facilitator
gives coaches a test before their quarterly meeting to
detect problems that should be discussed.
Region and District Knowledge on the Work of the
QA Coaches/Links and HC Teams
When the link is not part of the DHMT, the districts
know little about QA activities. At the regional office,
QA was seen to include management activities that help
ensure quality health services at lower levels of the
system, i.e., supportive supervision, training or capacity
building, information exchange through HMIS or other
reporting systems, and other ways of maintaining stan￾dards of care (including the development/dissemination
of new standards). Though staff expressed concern for
such quality issues and appeared to have some knowl￾edge of the existence of QA teams, they were generally
not well informed about the work of the coaches and
facilitators and wished to improve communications
regarding these activities.
District staff are better informed about QA activities.
This is likely due to the fact that many more district￾level staff have had exposure to some form of training
(e.g., QA training workshops or district capacity￾building sessions) than regional level staff. Moreover,
some DHMT staff were trained to serve as coaches or
link facilitators to the HC teams.
3. Reporting/Feedback Mechanisms for QA
Activities
System for Reporting QA Activities
Information on QA activities is unevenly shared
among levels of the health system. At the regional level,
specific knowledge regarding the work of the teams,
coaches, and facilitators is limited. Reports of QA
activities are not submitted to the regional office, and
existing reporting forms (e.g., the performance audit
and district progress reports) do not capture information
about the work of the QA teams.
At the district level, a summary report of HC activities
is completed by the link facilitator and forwarded to the
QA staff at the CBoH. The reports list the active teams
and describe the status and results of their efforts. Quar￾terly meetings of all link facilitators are held to dissemi￾nate results and share experiences regarding their team￾work. The CBoH uses the results of these meetings to
identify opportunities for improvement and track team
activity.
At the HC level, the recommended storybook format is
intended to facilitate more efficient and complete
documentation of the team activities. Page headings
correspond with major steps in the cycle. Teams can
thus describe how their problem(s) were identified,
record problem statements, describe the use of tools and
data gathering instruments, and present solutions and
results. A review of available storybooks found many
incomplete or irregularly updated. Moreover, the supply
of storybooks was inadequate.
In addition, problems/difficulties encountered while
implementing the QA steps are not recorded. Such
findings would provide a useful history of the team’s
experience as well as indicate needed improvements to
the problem-solving process.
At present, HCs do not formally exchange information
about the work of their teams, though opportunities for
doing so exist in the form of monthly meetings of
in-charges and among HCs that share a coach.
30
Communities learn of quality improvements at their
HCs through participation in NHCs. In one instance
(City Center), NHC members were actively involved in
a problem solving. Posters or storyboards are some￾times displayed at the HC. Such “visibles” communi￾cate the team’s commitment to quality, call attention to
the need for similar improvements in other departments,
and strengthen advocacy for QA throughout the HC.
4. Supervision of QA Activities
As described by 1999 action plan guidelines (CBoH
1998a), “supportive supervision is part of a larger
system of quality assurance and improvement,” which
is a departure from the more traditional, authoritative
forms of supervision and calls for a more empowering
approach to improving performance. The frequency and
quality of supervision within and between administra￾tive levels effect the performance of staff and program
objectives. Much work has already been done to
improve supervision, such as formal supervision
occurring as a team activity with the goal of assessing
performance and ensuring collective responsibility for
resolving problems. Guidelines for supportive supervi￾sory visits have been integrated into the new HMIS
(CBoH 1998b), and staff recognize that supervision can
be strengthened, not just by increasing the number of
supervisory contacts, but also by improving the focus
and quality of the visits.
Supervision as a mechanism for monitoring compliance
with standards is in Chapter III, Section A.3. In this
section, supervision is further developed as a way to
initiate and evaluate quality improvement activities at
various levels of the health system, particularly at the
HC level.
Types of Supervision Provided by the Regional and
District Offices and How They Support QA at
Other Levels
Opportunities for supervision of QA activities are
numerous. At the regional level the main type of formal
supervision given to the DHMT and to some HCs is the
quarterly performance audit. Discussions with regional
staff convinced the evaluation team that the regions can
influence quality of care at the HCs by including
DHMT clinical staff in the performance audit visit, and
then delegating responsibility for solving problems to
district-level staff. Supervisory staff reported that
during these visits, the focus would often go beyond
that prescribed by the performance audit to more
specific technical areas. For example, the clinical care
specialist might look more closely at maternal health
services and thus help to identify opportunities for
problem solving at the HC. Since many quality indica￾tors are in the hospital subsection of the audit form
(such as “patient satisfaction”), the audit could similarly
contribute to QA activities.
Those HCs that reported receiving formal visits from
the regional office generally felt it was done infre￾quently and was largely nontechnical (though a review
of one performance audit report listed a series of
“clinical problems”). One DHMT said that although
regional supervisors visit, he does not “see any
long-term results of communication with the region”
since the district does not get needed resources.
In addition to the performance audit, unplanned or
informal visiting at the HCs by technical staff from the
regional office was reported. This too could provide
input into the identification and resolution of quality of
care problems, though the evaluation team did not
encounter any facilities where this had occurred.
With regard to supervisory skills and training, regional
staff want to improve their interpersonal skills for inter￾acting with district and HC staff. Some also requested
that observation checklists be developed in all relevant
technical areas to help structure supervisory visits.
Supervision does not always focus on quality and QA
issues. At the district level, supportive supervision, in
the form of a formal integrated team visit to the HC,
was reportedly often used to monitor QA activities and
identify problems to be addressed by the problem￾solving teams. This appeared to be due in large part to
the presence of one or more DHMT staff who had
either been trained as a QA coach or facilitator or had
otherwise participated in a QA training workshop. Four
districts (Kabwe, Kitwe, Choma, Lusaka), all of which
had a trained facilitator/coach on the DHMT, reported
that the supervisory visit was used to monitor the work
of QA teams in implementing the problem-solving
steps, identifying additional QA training needs, and
addressing quality of care issues at the HCs.
In general, HC staff appreciate the need for frequent,
supportive visits from their DHMT, but many want
more frequent formal visits from district staff,
especially regarding technical issues. Though technical
supervision was said to be routinely planned and
performed, HCs often reported less frequent visiting
than was reported by the district (see Table 4–10).
The supervisory visit provides a mechanism for routine
monitoring of service quality. Six of seven DHMTs
interviewed said all HCs were visited every quarter. If
the IMCI checklist was used during the visit, then a
review of Out-Patient Department (OPD) cards was
31
reportedly done to check for treatment and diagnosis as
well as direct observation of patient care (the latter did
not appear to be routinely done, as noted above).
Supervisory visits were also found to contribute to the
work of QA teams, helping identify problems. In
Kabwe, a DHMT supervisory visit included discussion
of findings from a client survey, which led to a team’s
addressing the problem of clients bypassing the HC to
seek care at the hospital. In Nakoli Clinic, the quarterly
supervision visit identified problems with user-fee
collection, prompting the development of a system for
monitoring fees. The new system was replicated at other
HCs. In Choma, district supervisors noted some
discrepancies in the figures reported for malnutrition; a
team problem solving reportedly found an appropriate
intervention.
Most feedback from the DHMT visit was reportedly
given immediately so individual clinical problems could
be corrected quickly. In Kitwe, if common problems
were found across all health centers, then meetings
would be held with all in-charges to discuss
remediation. Sometimes, feedback would be given
during a general meeting at the end of the formal
DHMT visit and would include all clinical staff and
often NHC members. Problems discussed in a meeting
Table 410
Types and Frequency of Technical Supervisory Visits
District Total HCs 1. Frequency of Formal Visits to HCs
2. Frequency of Other Kinds of Visits HC Interviewed
Kitwe 19 1. Formal team visit: 2/month Chimwemwe
2. Unannounced monthly visit
Kalulushi 14 1. Formal quarterly team visit Kalulushi Gov.
2. Unannounced monthly visit
Lufwanyama 13 1. Formal quarterly team visit Chimokunami
2. Unannounced monthly visit
Monze 14 1. Formal quarterly team visit Rusangu
2. Quarterly technical visit
Kabwe 13 1. Monthly formal team visit Mahatma Gandhi
2. Other: Undetermined
Nakoli/Kapiri 18 1. Monthly formal team visit District Hospital
2. Other: Undetermined
Lusaka 22 1. Formal quarterly team visit Kalingalinga
2. Bi-annual special visit
with the NHC, HC staff, and DHMT
included long waiting times, availability
of drugs, the nonsupportive attitudes of
supervisors and staff, and clinic
maintenance.
Other planned and unplanned visits to
HCs were also reported, occurring
between formal quarterly visits and quite
often addressing problems identified
during the formal visit. In Kitwe, supervi￾sors reportedly observed patient flow and
checked patient records. In Kalulushi,
problems that were identified and dis￾cussed during supervisory visits included
the need to correct and monitor staff in
the use of a new chloroquine regime. In
general, unannounced visits by the district
staff were thought to encourage continued
attention to quality.
Apart from the supervision given primarily by the direc￾tor through daily interaction with other DHMT staff,
internal supervision at the district office is done through
the new HMIS self-assessment form. The form relies on
self-assessment data from the HCs. This form was
encountered in only one district, though it was said that
the data generated had been used to “address problems
such as malaria treatment.”2 The region was also said
to have reviewed and discussed the form during the
performance audit, though this could not be confirmed.
Supervision within and between HCs and How It
Supports QA Activities
Within the HCs, routine supervision by the responsible
staff in various units was also found to contribute to
team problem-solving activities. In larger urban health
centers such as Kalingalinga, for example, routine
supervision is done by the in-charges of various depart￾ments. The maternal and child health supervisor identi￾fied the problem of antenatal care clients coming for a
first visit during their second or third trimesters; this
was resolved through a problem solving. At Mahatma
Gandhi Clinic, supervisors reportedly checked OPD
cards and medical records routinely and discovered low
monthly postnatal attendance as compared to antenatal
visits; this was also resolved through problem solving.
2
 The relevant indicator is a composite ratio of new malaria cases per quarter to new malaria cases in the same quarter of the previous year. Disease aggre￾gation forms for individual HCs would have had to be reviewed in order for district supervisors to begin investigating the underlying causes. Direct observa￾tions or some form of record review would then have been needed to confirm that treatment of malaria was indeed a problem. It remained unclear what the
actual fact-finding process was, whether or not it was actually done as part of a QA exercise, and thus how the indicator might have been used to address
the quality of malaria services.
32
Supervision between HCs occurred in the form of a
peer review in which IMCI-trained staff from one HC
monitor and are monitored by another IMCI health
center. Weekend supervisory visits by a team of HC
staff also occur. In Kitwe District, the IMCI supervisors
had received a three-day training in QA and were in￾structed to check the storybooks as part of their IMCI
support visit. It was further hoped that they would be
able to provide support to the QA teams and relieve the
link facilitator of some of this responsibility. In addi￾tion, the weekend supervision was said to contribute to
problem-solving activities at the HCs; for example,
problems with the maintenance of the Oral Rehydration
Therapy corner and refrigeration of vaccines were
brought to the attention of HC staff.
D. The Role of Other Structures and
Sectors in QA Activities
1. The Neighborhood and Health Center
Committees
A 1997 CHESSORE survey describes the participation
of the NHCs in the improvement of the health services
(CHESSORE 1998a). Most such activities relate to: (a)
the rehabilitation of HCs, (b) information and mobiliza￾tion of the community, (c) expression of community
dissatisfaction with health services, and (d) community
surveys.
Since they represent the communities and express
patients’ perspectives on quality of care, the NHCs
should be involved in some QA activities (e.g., problem
solving). However, it is not clear how much they can
contribute without formal QA training. Some problem￾solving teams have included trained members of NHCs
in their work (Civic Center Clinic, Lusaka).
The evaluation team found anecdotal evidence of public
involvement in QA activities. In Kabwe (Mahatma
Gandhi Clinic) patients complained about the shortage
of drugs. The NHC and clinic staff looked at the
medical records and realized that many patients were
coming from outside the clinic’s catchment area. The
NHC decided to realize a census to register authorized
patients and be able to identify the area where eligible
patients reside. Others are now referred to their own
HC.
Some HC committees (Kabwe) were involved in the
development of the 1999 district action plan by a
district health board (DHB).
2. The District Health Boards
The team met with the Kabwe DHB and discussed its
role in improving quality of care. Most board members
are not health professionals, but they have the power to
make decisions affecting the quality of care and QA
activities, such as: (a) approval of budget, (b) approval
of staff recruitment, and (c) identification of training
needs.
The DHB participates in technical meetings with the
DHMT and accompanies it on inspections of facilities.
The DHB mentioned that it worked on the issue of
maternal mortality when an NHC complained that preg￾nant women were dying during deliveries at home be￾cause the HC did not have a delivery room. The DHB
decided to build an extension to the clinic for deliveries.
This issue was an eye-opener for the DHB, which is
trying to address it in other areas.
The DHB also took an active part in distributing the
patients’ rights booklet.
3. The Private Sector
The Churches Medical Association of Zambia
(CMAZ)
The CMAZ coordinates activities between different
church-owned health facilities (31 hospitals, 60 HCs),
trains managers, works to help its members secure
financing, and is an official link between these services
and the government. Most mission health facilities have
mission doctors and nurses but are mainly staffed by
government employees selected by the churches. Those
staff are on the same contract as staff in public health
services.
The association confirmed the evaluation team’s find￾ings that many of its members participated in the QA
training by the CBoH and that districts usually include
mission health centers with the public centers in their
quarterly supervisory visits. The team was told that QA
training by the association is a three-hour session on a
regular management course. The association was inter￾ested in taking a full part in national and district training
and quality activities, but emphasized the limited finan￾cial resources of its members to undertake quality
improvements that require extra resources.
The association will shortly be acting as an agent for
USAID to channel funds for specific health programs to
districts and evaluate the results. This will include a
quality evaluation component.
33
The Faculty of Private Practitioners
This association has 300 registered practitioners,
including 88 members in Lusaka. All members are
medical officers; it is estimated that they see about 1 to
2 percent of out-patients in Zambia.
Although the faculty has no official QA policy, its
members are required to annually attend 10 out of 12
monthly clinical meetings (continuing medical
education) in order to be registered.
The association does not monitor or evaluate the
quality of clinical care.
4. The Regulatory Bodies
The General Nursing Council
The GNC is a statutory body (established in 1970)
responsible for ensuring that members of the public
receive the best possible healthcare. The GNC sets,
monitors, and evaluates performance standards for
nursing/midwifery education, clinical practice,
management, and research.
Its main activities are: (a) inspection of health institu￾tions and schools of nursing and midwifery, (b) mem￾bership on the Accreditation Council, (c) curriculum
development and revision for nursing and midwifery
training, (d) setting entry standards for nursing and
midwifery basic training, (e) issuing certificates to
qualified nurses and midwives, (f) approving training
institutions for nursing and midwifery, and (g) disciplin￾ary actions for students, nurses, and midwives.
The GNC participated in the development of the
indicators used for hospital accreditation.
The GNC communicates standards by: (a) sending
circulars to individuals and institutions, (b) conducting
workshops, (c) taking corrective measures during health
facility inspections, (d) making recommendations to
health facilities during inspections, and (e) holding
meetings with stakeholders to discuss ways to improve
healthcare quality.
The GNC has inspection guidelines for health institu￾tions and schools of nursing and midwifery. Thy focus
on the structure of the facilities, standard operating
procedures, and policies. Most of the judgement criteria
are not explicit.
Monitoring of quality care is a problem because of
financial and material constraints. The council can
achieve the following through its health facility
inspection: (a) meeting with management, (b) checking
for compliance with standards in medical, nursing, and
administrative areas, (c) direct observation of clinical
care, (d) client and staff interviews, (e) inspection of
nursing and medical records, (f) noting whether
resources such as equipment and time are available for
provision of nursing care, (h) ensuring staff have valid
licenses, and (i) correction of malpractice on the spot.
The Medical Council of Zambia
The MCZ inspects private hospitals and clinics, using a
checklist called “inspection certificate,” which focuses
exclusively on the structural aspects and not on the
assessment of the care. It also approves the curriculums
for all paramedical training institutions except for
nursing schools: clinical officers, dental technicians,
and laboratory technicians.
5. The Patients
Several surveys informed the evaluation team on the
patients’ perspectives on the quality of care
(CHESSORE 1998a). The results are consistent with
the international literature on patient satisfaction. For a
patient, the most important features are: (a) the avail￾ability of drugs, (b) the politeness of the staff, (c) a short
waiting time, and (d) a physical examination. This last
feature is interesting since it relates to the process of
care, usually considered by professional staff to be too
complicated for patients to appreciate. One survey
revealed that only 38.7 percent of patients thought that
they were properly examined (MOH 1994), while 67
percent thought that staff had poor clinical skills
(CHESSORE 1997).
One survey, done before the health sector reform and
the QAP started, assessed the perception of the quality
of care by the beneficiaries (The World Bank 1993). It
looked at urban/rural differences and found that: (a) in
rural areas the basic minimum level of resources was
lacking (especially staff) and patients complained about
drug shortages and staffing inadequacies, (b) in urban
areas, the main problem was the mismanagement of
resources; patients also complained about drug
shortages.
A patients’ rights booklet describes the level of quality
of health services that people are entitled to as well as
the categories for fee exemptions (CBoH 1996). It was
distributed to all hospitals, districts, HCs, and consum￾ers’ associations. It states standards but does not quan￾tify them, leaving the quality gap difficult to measure.
How well patients know their rights is undocumented.
34
6. The Training Institutions
The Nursing School
The University Teaching Hospital (UTH) graduates
some 60 registered nurses per year. The principal tutors
from the school reported that since 1990 the nursing
curriculum had not been revised or reviewed to inte￾grate standards of care. At that time the review panel
consisted of representatives from WHO, GNC, the
Department of Post-Basic Training, MOH, and other
programs. Although another review was planned for
October, 1998 staff felt that the process should be both
more systematic and frequent. Tutors participate in
programs such as Safe Motherhood and IMCI, or go for
periodic “updates” or training, then disseminate techni￾cal knowledge (from their own notes/material) in their
courses. A mechanism for standardization of this new
knowledge is an examination panel made up of princi￾pal tutors. Among those interviewed, however, the
consensus was that not all tutors have enough knowl￾edge on new topics. Overall, new knowledge is im￾parted to students on an informal and irregular basis.
Another way new knowledge is acquired/shared is
through field practicums. To acquire skills in delivery of
primary healthcare at the district level, nursing students
may be sent to the field with costs (fuel, accommoda￾tion, food, and transport) paid by the nursing school.
The district gains free labor in the form of registered
nurses who are at a higher grade of training than many
HC staff.
Action plans for 1996 included activities like intern￾ships for following up students after they left the institu￾tion, but they were never implemented due to funding
problems. The GNC reportedly monitors nursing
standards through their periodic evaluation of hospitals
and nursing school accreditation activities.
Tutors were interested in including QA in their manage￾ment curriculum. MOH or other specialists are invited
to speak as part of a two-week management course
given separately from basic nurses training. It was
thought that this might be a way to introduce students to
QA. The felt need, however, was to integrate QA as part
of preservice and post-basic training curriculums.
The School of Medicine
It was unclear whether clinical quality standards were
taught at the school to medical and other post-qualifica￾tion students. QA did not seem to be taught, but quality
was beginning to be addressed by faculty in academic
journals (Simms 1998). Formal links do exist between
the School of Medicine and the CBoH, with the Dean
on the Board of CBoH. Improving the number and
quality of links between the health system and the
school would help to ensure that the students become
familiar with the standards of clinical care. The school
also could develop its input into formulating and revis￾ing care standards and protocols to ensure that they
conform to the latest research in developing countries.
The school expressed an interest in developing a
Masters-level program in Health Service Quality,
with the faculty teaching prequalification students and
running in-service quality training. The school wished
to introduce QA programs and faculty and felt that this
could complement the training run by the CBoH and
help to build a long-term QA capability in Zambia.
IV. Detailed
Recommendations for
Strengthening QAP
This chapter assembles the recommendations made
above, detailing the most important ones and listing
some options for their implementation when
appropriate.
A. General Recommendations
The evaluation team feels that a program as important
as the Zambian QAP justifies more frequent reviews,
perhaps every two years. Also, regular monitoring of
QA activities and their effects would help in collecting
the documentation necessary to continuously improve
the program.
1. Policy Recommendations
A national QA policy that includes the private sector
should be developed. It should describe the vision,
strategy, and objectives of Zambia’s QAP. This would
allow the development of an action plan that would
facilitate the implementation, monitoring, and evalua￾tion of QA activities. The design of such a policy
should not be restricted to the actions undertaken by
CBoH, but should include the nongovernmental institu￾tions that have a role in QA. Such a policy would help
define the QA roles and responsibilities of various
stakeholders in the health system.
2. Strategic Recommendations
The vertical integration of QA into all levels of the
health system should be strengthened by creating or
reinforcing the links between the QAP and:
35
■ The regions (or provinces), by building QA capacity
in the regional team and adapting the performance
audit instruments; this requires sensitizing and
training the regional teams
■ The districts, by building QA capacity in the DHMT
and adapting the performance monitoring instruments
for supervision; this means that DHMTs should be at
least sensitized in QA and some members trained
■ The hospitals, by bringing QA into their management
board and services. QA has not yet been introduced
at the hospitals, and the new accreditation program
intensifies the need for ways to meet the attendant
new standards
The horizontal integration of QA into other directorates
of the CboH must be strengthened by creating or
reinforcing the links between the QAP and:
■ The Directorate of Health Services Commissioning,
through the implementation of the hospital accredita￾tion program. The QAP should participate in provid￾ing technical assistance to the hospital boards and
measure the impact of accreditation on the quality of
in-patient care
■ The Directorate of Systems Development, through
the development of standards, clinical guidelines, and
job aids. This directorate should coordinate the
development of clinical guidelines and job aids that
best fit the needs of health providers
■ The Information and Health Systems Research Unit
of the Directorate of Monitoring and Evaluation,
through: (a) the development of operations research
on QA and quality of care and (b) the monitoring of
the use and impact of the HMIS on the work of the
QA teams.
The integration of QA within the private health sector
and parastatal institutions must be strengthened by
creating or reinforcing the links between the QAP and:
■ The regulatory bodies, such as the GNC and the
MCZ, through their involvement in the development
and communication of standards and the quality
performance monitoring system. These entities are
already inspecting training institutions and would
benefit from the QAP approach to monitoring clinical
performance
■ The training institutions for medical, nursing, and
paramedical staff, through their participation in the
development of standards and their inclusion in the
preservice and post-basic curriculums
■ The private associations, such as the Faculty of
Private Practitioners, through their involvement in
developing and communicating standards and
assessing quality in the private sector
B. Recommendations on Standards
of Care
1. Recommendations on the Development of
Standards
The development, adaptation, and revision of clinical
care standards should be defined in an official policy.
The process of developing standards necessitates that
one structure (e.g., the Directorate of Systems Develop￾ment) coordinate the work of groups of technical
experts, providers, and users. The involvement of health
providers from all levels of the system would ensure
that standards are realistic and accepted by their users,
while experts’ involvement would guarantee their scien￾tific validity. Evidence-based medical literature is key in
developing standards.
These standards should be based on research and an
assessment of the health providers’ needs for guidance
through job aids. The field of job aids development is
complex and more research should identify the factors
that influence their use by health providers. If they are
used, their potential to improve quality performance is
enormous.
The development/adaptation of standards must be
consistent with the development of other materials,
including training materials, pre- and in-service training
curriculums, job aids, and any instrument used to
measure compliance. In other words, when new stan￾dards are developed, this should trigger a set of changes
in the other mechanisms to communicate the standards
(pre- and in-service training), as well as the criteria on
which performance is being assessed and therefore the
instruments to do so (supervision checklists, etc.).
2. Recommendations on the Communication of
Standards
A national strategy to communicate standards that does
not rely solely on classroom training should be devel￾oped. It should draw on principles for behavior change
that would improve clinical practice. The effective
communication of standards induces a change in clini￾cal practice, which is not easily done by training alone.
The communication strategy should include a combina￾tion of interventions based on behavioral science. This
36
is also a topic for research since little is known about
the factors that would cause health providers to adopt
them.
The curriculums of Zambian nursing and medical train￾ing institutions should include the new standards
through regulatory mechanisms and the involvement of
stakeholders. This would reduce in-service training and
its costs. Preservice training is often overlooked and
comes last, if ever, in a strategy to introduce new
standards of care. This leads to a paradoxical situation
where young graduates are targeted for retraining as
soon as they start work.
The continuous reinforcement of standards is key to an
effective communication strategy. Districts and officers
in charge should be encouraged to use all opportunities
(e.g., during supervisory visits and technical meetings)
to reinforce standards of care. When regularly reminded
of the standards, health providers do better, and the
supervision visits and technical meetings should be
organized to emphasize QA and quality of care.
C. Recommendations on
Performance Monitoring
The assessment of quality performance should include
measuring health workers’ compliance with process
standards through direct observation of the delivery of
care and not focus only on inputs and outcome stan￾dards. Performance audits and supervision visits pro￾vide opportunities to observe clinical performance and
provide on-the-job training, so mechanisms are in place
for this recommendation, but it will require redesigning
the forms and re-thinking the entire strategy, from iden￾tifying the skills needed to understanding the con￾straints. This should be a research topic since resistance
to direct observation of care seems pervasive.
A specific strategy should be designed to explore the
causes of poor performance to determine whether it is
caused by lack of competency or something else.
Before identifying training as the answer to poor perfor￾mance, problem-solving teams should explore the root
causes and regularly test clinical competency to identify
in-service training needs. Other performance deficien￾cies might better be addressed through problem solving.
A formal recognition and reward system based on
quality performance should be established to create
incentives to improve quality. Rewards should be deter￾mined on the basis of effectiveness and acceptability to
district health boards. The staff appraisal system should
be redesigned to assure that quality performance is
linked with career promotion and that the measurement
is objective and fair.
D. Recommendations on QA Activities
Investigate methods for improving quality in HCs with
five or fewer staff, using the evaluation team’s evalua￾tion methodology (Appendix D) if appropriate. If the
CBoH wishes to form problem-solving teams in small
HCs, further investigation is needed of the conditions
required for these teams to succeed and whether these
conditions are feasible. Most of the QA program
resources have been be devoted to building team
problem-solving capacity. The QA Unit or others
should investigate the levels of quality in small centers
to determine which QA methods would be appropriate
for them.
Continue using the problem-solving model, but with
simplifications and with changes to training. The team
problem-solving method can be effective in HCs with
over10 staff under certain conditions. It should continue
at these centers but be simplified for those who receive
less training. The last step of the model should be
divided into two parts: “implementation” and
“evaluation.” Step 4 could be broken into two steps:
“initial problem analysis” and “data gathering and root
cause location.” This would emphasize the importance
of data collection. Each trainee should have a copy of
simple guidelines for team problem solving and also
use a storybook.. The training materials and sessions
should recommend choosing a simple problem for the
first cycle, involving users or NHCs, following steps
in succession and linking them, and gathering and
interpreting data.
A simpler, modular five-day training package for
district training should be developed for those who
will not be coaches; it should incorporate the above
simplifications. Its developers should draw on the
experience of links and coaches who have designed
five-day or 20-hour session courses, because resources
are not available for longer training. The course should
emphasize both the linkage between the steps and the
data gathering and interpretation to determine “real root
causes.”
For the first cycle, teams with little training should learn
by doing, using its first problem-solving cycle as a
training exercise. A coach should attend each meeting
to help the team learn by doing and to show good
practices in documentation, how to use the methods,
and how to link the steps. Identification of the priority
problems could be done during the QA training.
37
The evaluation team’s evaluation system (Appendix D)
should be piloted as a method for helping teams learn
how to be more effective, assessing teams, and giving
them feedback. Teams should be taught how to use this
system for self-assessment and to guide a peer review of
the work of other teams.
Links and coaches should develop skills to decide when
to use a team problem-solving cycle and when not to.
They should then train and advise teams on how to
make this decision.
NHCs and consumers of healthcare services should be
involved in QA problem solving, and patients should
drive the quality program. Pilot efforts should be
evaluated to generate recommendations for increasing
involvement.
All teams should document their work in storybooks,
and the documentation should be available. At least one
copy should stay in the health facility, and coaches
should use it to monitor team progress when they
cannot attend meetings.
Other methods to improve quality for different circum￾stances should be investigated. While the approach
taken by CBoH has achieved a great deal, different QA
methods are needed for different settings, such as small
facilities. The QA Unit should become more familiar
with other methods and assess which might be appropri￾ate in different facilities and for different types of ser￾vices (disease prevention, drug supply). An assessment
should be made of the value of training and problem￾solving teams compared to other quality improvement
methods, such as clinical training, patient education and
counseling skills, on-the-job training, supervision of
clinical skills, protocol and guidelines development,
standard setting and indicator development and moni￾toring, and clinical case review and clinical audit.
E. Recommendations Regarding the
QA Support Systems
1. Recommendations on QA Training
To ensure priority for QA training, a training needs
assessment should be done as part of the yearly action
plans. In reviewing these plans the CboH or regional
office should ensure that QA training activities have
been adequately planned and budgeted, and district
plans should include coaching or link facilitator visits to
HCs.
CBoH should develop enough QA trainers and experts
at the national level to provide adequate technical
support to all levels of the system. Two trainers per
regional or provincial level team could suffice. The
objective is to have the capacity to ensure responsive,
consistent training support.
Relevant professional regulatory boards (GNC and
MCZ) should take QA training and incorporate QA
concepts into preservice and post-basic training
curriculums.
All professional staff at the regional and district offices
should take some QA training, within resource con￾straints. Training should cover standards setting,
indicator development, and problem solving. QA could
also be incorporated into other management training.
The DHMT should identify candidates for QA training,
in particular staff at the HC level, who could become
in-house coaches or facilitators to one or more HC
teams.
At least two people per HC should be trained in QA
(including the in-charge). The district should then
ensure that there are enough trained coaches to cover as
many facilities as possible given available resources
and geographical access.
2. Recommendations on Coaches and Link
Facilitators
At least one staff on the DHMT (i.e., a district-level
coach or link facilitator) should have responsibility for
QA activities in the district. This person should partici￾pate in the district team supervision visits to the HCs
and include QA in the integrated supervisory approach.
QA training should further strengthen capacity in QA
planning, effective communication and teaching, and
supportive supervision. Furthermore, support visits by
the central level to coaches and link facilitators should
be intensified (at least early on) to further ensure that
training efforts actually do result in QA activities.
District offices and coaches should take advantage of
existing opportunities(e.g., district-level meetings and
other training workshops) to update their QA training
and share information on QA activities and quality
issues.
3. Recommendations on QA Documentation and
Feedback
Existing reporting systems should include reports of
QA activities; timely feedback should be given to the
appropriate levels; and information should be shared
between and within levels. QA staff at the CBoH should
help determine what information on QA activities may
be useful at each level and how it should be reported.
38
QA staff should ensure complete documentation (e.g.,
in storybooks) of QA activities and effective use of the
documentation to monitor the work of teams. Sufficient
storybooks should be available to enable this.
4. Recommendations on Supervision of QA
Activities
Regional offices should ensure that DHMTs are provid￾ing adequate and frequent supervision to the HCs. Since
supervision is planned and budgeted as part of the an￾nual action plan, some interim review of achievements
could evaluate where supervision problems may exist
(e.g., quarterly visits are not occurring, supervisory
reports are incomplete).
Each district should plan and budget for an adequate
number of supervisory visits and ensure that they occur.
Staff suggested that the minimum number of visits
would include quarterly visits plus four others per HC
per year. Visits should initiate and support QA problem￾solving activities and, in the spirit of supportive super￾vision, allow for adequate feedback and local resolution
of problems.
The person(s) responsible for QA activities at the
DHMT should be involved in quarterly supervisory
visits to HCs. Many QA coaches and facilitators are
DHMT staff, which seemed to facilitate the integration
of QA into their supervisory duties. These quarterly
visits could then be followed by visits in support of
teams.
F. Prioritizing the Recommendations
The long list of recommendations results from the
evaluation methodology. This kind of methodology
produces detailed results, and the team believes it
would do the same in different settings. These results
are not unusual and are consistent with findings in other
countries.
Of all the recommendations in this report, the ones in
this section should be first in Zambia’s efforts to
improve healthcare quality. They are divided into two
categories: those that can be acted upon immediately
and those that need more analysis of their causes and
operations research.
1. Priority Recommendations for Improving the
QA Program
The evaluation found the lack of QA expertise in most
of the DHMTs hampered the work of the facility teams.
When no district team member has had QA training, no
one at the district level takes ownership for quality
improvement in the district, and no one supports,
through coaching, the work of the teams. The teams
have a hard time implementing QA methods, and they
do not have time to teach and coach others if they are in
charge of a health center. At least two DHMT members
per district (one being the District Medical Officer)
should be trained in QA and take responsibility for QA
activities.
Despite its tremendous efforts and remarkable success
in covering the whole country with a network of
coaches and trainers, the CBoH remains weak in its
capacity to sustain and expand the QA program,
because only two people at the central level are in
charge of QA. While “quality is everybody’s business,”
if there are insufficient people to do the job, it is not
done. The dedication of the two QA staff is remarkable,
but the central QA team needs to be strengthened to
build QA capacity in districts and monitor the progress
of the QA activities nationwide.
A national QA policy would boost the interest in QA
and health sector reform and enable progress despite
de-linkage and the collapse of the drug supply system.
Milestones covering the next five years with a clear
strategy to institutionalize QA would also help, making
it easier for the CBoH partners to design their support
for improving healthcare quality.
Currently, the focus of supervision is not quality of
care, but changing supervisory processes could have a
tremendous impact on quality. It should focus on quality
monitoring and improvement through the definition of
standards for supervision. Supervisory visit should
facilitate the direct observation of the delivery of care,
feedback the results, and help teams solve quality
problems or improve healthcare processes.
A capacity for QA training must be built through a
network of certified QA trainers and the introduction of
QA into preservice curriculums. This would build
self-reliance and ensure QA’s sustainability.
2. Topics for Operations Research
The evaluation team is puzzled by the reluctance of
supervisors to directly observe the delivery of care and
of health workers to use job aids. Research into these
areas should be undertaken as soon as possible.
Other research topics are: (a) ways to increase the cost￾effectiveness of QA training, (b) standards development
and the best communication strategy for improving
clinical practices, (c) workers’ perceptions of the scien￾tific validity, feasibility, reliability, and clarity of the
ITG standards, and (d) factors that influence the pro￾ductivity of the problem-solving teams.
39
V. Conclusion
There is growing evidence of the impact of QA
methods on the quality of care in resource- constrained
environments, where simple solutions such as
re-training of staff or the supply of additional resources
have failed. The richness of the QA experience in
Zambia provides lessons that will benefit not only the
health sector reform in Zambia, but also other QAPs.
While much remains to be done, the enormous number
of discrete changes resulting from the work of the QA
Unit and the Directorate of Monitoring and Evaluation
has had a tremendous impact on the way patients
interact with their health system and demand quality.
Health providers’ behavior is also changing under the
positive influence of the attention districts are paying to
their work and the empowerment mechanisms that have
been established through decentralization.
The key lessons learned from Zambia can be summa￾rized as follows:
■ Support systems, such as coaching and QA training
capacity, must be well established at the district level
in order to assist the teams. In a decentralized system,
the QA program should first target the districts so that
team ownership for QA activities will develop
■ Numerous factors influence the productivity of the
problem-solving teams. The tools developed for this
evaluation (difficulty index and team failure index)
proved to be useful, and coaches could use similar
materials to assist teams
■ The health workers’ resistance to job aids that would
improve their compliance with standards and supervi￾sors’ reluctance to directly observe the delivery of
care are barriers to better healthcare quality. More
research would determine the causes of these
behaviors and assist in their improvement
■ A detailed documentation system of the QA program
would help in monitoring the QAP’s impact and
making adaptations
QA is more than tools and methods. It is a spirit that
comes only from a mentality change, where clients
make demands and express their expectations, where
health providers question their own performance, and
where managers make client-oriented, data-based
decisions to constantly improve healthcare systems. It
cannot happen overnight. It took five years to initiate
change and the emergence of a quality culture in
Zambia.
40
Work Cited
Central Board of Health (CBoH). 1996. Patient’s
(client’s) entitlements and responsibilities in accessing
health care services in Zambia. Lusaka.
———. 1998a. 1999 district action plan: District
health planning guide for district health office and
health center planners: 1999. Vol. 2.
———. 1998b. Health Management Information
System: Procedures manual for health centers.
Directorate of Monitoring and Evaluation.
———. 1998c. Link Facilitators’ Reports.
Central Statistical Office. 1998. The Zambia situation
analysis 1997: An assessment of the functioning and
quality of reproductive and child health services in
Zambia. Lusaka.
Centre for Health, Science and Social Research
(CHESSORE). 1997. Quality of care factors in the
Zambia health services. Lusaka.
———. 1998a. Preliminary fieldwork survey briefing
on the organisation, functions and responsibilities of
the health centre committees in the Northern Province
of Zambia.
———. 1998b. The performance of health services in
Zambia. Lusaka, 1995–96.
———. 1998c. The performance of health services in
Zambia. Lusaka, 1997–98.
Franco, L.M., J. Newman, G. Murphy, and E. Mariana.
1997. Achieving quality through problem solving and
process improvement. Quality Assurance Project,
Center for Human Services, Bethesda, MD.
Heydelberg, E., T. Mubonda, and J.M.N. Tembo. 1997.
Review of HMIS/FAMS/ITG cascade training. Mission
Report. Lusaka/Amsterdam.
Limbambala, M.E., and J.M.N. Tembo. Undated.
Quality assurance, monitoring and evaluation.
Limbambala, M.E. and J.M.N. Tembo. 1996.
Discussion paper on QA of laboratory services.
Ministry of Health (MOH). Undated. Certification of
hospitals: Proposed minimum standards to be met by
referral hospitals levels 1,2,3, and 4 (UTH) in order to
be certified by the Clinical Advisory Committee of the
Central Board of Health. Quality Assurance Unit.
———. 1993a. Factors leading to inappropriate use
of western medicines for acute diarrhoea and dysentery
by consumers and health care providers in Mbala
District, Zambia. Health Systems Research Unit.
———. 1993b. Healthcare quality assurance manual.
Lusaka.
———. 1993c. Source reference standards in clinical
and nursing care, diagnosis and environmental health
services. Ministry of Health, Zambia.
———. 1994. Study of factors contributing to poor
quality of clinical care in health institutions, and
establishing the availability, utilization and
appropriateness of standards in Zambia. Quality of
Health Care Project.
———. 1996a. Health sector reform review. The
Participatory Assessment Group, Lusaka.
———. 1996b. National strategic health plan 1995–
1999. 3rd Edition.
Quality Assurance Project (QAP). 1996. Quality
assurance coaching. Center for Human Services,
Bethesda, MD.
Republic of Zambia. 1997. Immunisation coverage
1992 and 1996: Zambia demographic health surveys:
A review.
Simms, P. 1998. Towards clinical audit in the
University Training Hospital and health services of
Zambia. Zambian Journal of Medicine and Health
Sciences. 2:20–21.
The World Bank. 1993. Quality of care beneficiary
assessment study. Zambia health sector study.
Tropical Disease Research Centre (TRDC). 1997.
Guidelines for malaria management, prevention, and
control in Zambia. Lusaka.
World Health Organization (WHO) and UNICEF.
1995. Integrated Management of Childhood Illness.
Geneva, Switzerland.
A1
Appendix A:
Selected Sections of the Scope of Work and
Terms of Reference
Purpose of the Mission
The purpose of the mission is to evaluate the Quality
Assurance Programme in Zambia in order to make
recommendations that would help the Central Board of
Health and its partners make decisions to adapt and
improve its strategy for improving the quality of care.
The evaluation will seek to answer in-depth the specific
questions detailed in the scope of work on some com￾ponents of the QA activities carried out in Zambia,
rather than to try to address, in a short period of time,
all components of an ideal model for a QA program.
However, the team is encouraged to make recommenda￾tions on QA activities not currently undertaken and that
the team thinks could benefit the Zambia health system.
Given the parallel work of the Joint Commission on
accreditation of hospitals, the evaluation team will not
address this issue.
Overall Organisation of the Mission
An evaluation team of six experts will be constituted:
three international experts (one process improvement
specialist, one health systems analyst and one standards
specialist) and three local Zambian experts (one qualita￾tive data specialist, one quantitative data specialist and
one cost analyst). One of the three international experts
will be recruited from URC/CHS and will be the team
leader.
The total duration of the mission will be four weeks:
■ Field work : eighteen workdays of the core evaluation
team in Zambia to collect and analyse data and to
present the draft of the final report in a workshop;
■ Finalisation of the report: five days of the team leader
at URC headquarters to finalise the report.
As preparation of the evaluation, the CBoH will gather
relevant documents for the team to review and will
organise a presentation of the health sector reform and
QA activities in Zambia at the beginning of the mission.
The mission will use mainly, but not exclusively, the
following methods: meetings with key people and insti￾tutions, review of documents and records, and group
working sessions. Given time constraints, the collection
of primary data through surveys is not expected. The
consultants will work in subteams of two:
■ The standards subteam will be made of the interna￾tional QA expert and the local expert in quantitative
data analysis.
■ The process improvement subteam will be made of
the international expert in Total Quality Management
and the local expert in qualitative data analysis.
■ The support systems subteam will be made of the
international expert in support systems and the local
expert in cost analysis.
We would focus on one or two locations and all three
topics will be studied in depth at the same time both in
urban and rural settings. This will allow linking the
information obtained by each consultant. However,
during their data collection work, each subteam will
work independently with the appropriate level of the
health system. The evaluation team will meet regularly
in order to exchange information, identify problems
encountered for the completion of the work, and moni￾tor the progress according to the work plan, under the
facilitation of the team leader. The evaluation team will
work according to the schedule presented in Table 1,
the details of which will be finalised during a team
planning meeting upon arrival in the field.
A2
Scope of Work
1. The standards subteam will document the way
standards are developed and communicated and how
compliance with standards is measured and used for
decision-making to improve the quality of care. More
specifically, the standards subteam will:
■ Identify the documents that present the standards
for technical and managerial functions performed
by health facilities;
■ Document the ways the standards are developed
(who was involved, what was the adaptation
process, etc.);
■ Document the ways the standards are communi￾cated to the health providers and other staff
supposed to apply them (training, meetings, etc);
■ Assess the level of knowledge of the existence of
the standards and of the standards themselves, as
well as the attitude of the staff about the standards;
■ Assess how standards (including clinical guidelines
and Standard Operating Procedures manual) are
used by staff;
■ Document the existence of a revision/adaptation
policy for standards;
■ Document the role of the regulatory bodies in the
development of standards;
■ Identify the mechanisms by which compliance with
standards is measured;
■ Document how the information on compliance
with standards is used to improve quality of care;
■ Identify the constraints in the development,
communication and use of standards;
■ Suggest options for improvement of the develop￾ment, communication and use of standards; and
■ Investigate any other relevant question identified
by the evaluation team.
Data collection methods used by the standards subteam
will include, but will not be limited to: i) meetings with
the public and private sectors institutions in charge of
developing and communicating standards, ii) in-depth
interviews with health providers, and iii) review of
documents and appropriate reports.
2. The process improvement subteam will document
the work of the facility-based problem-solving teams,
through collection and analysis of data on the struc￾ture, process and results of their work. More specifi￾cally, the process improvement subteam will:
■ Determine the level of competence of the staff in
problem solving techniques;
■ Document the type of problems identified by the
teams, and their relevance;
■ Measure the average duration of a problem solving
cycle and suggest ways to shorten it;
■ Validate the progress made by the teams on the
basis of their self-monitoring;
■ Document the way the different steps of a problem
solving cycle are carried out;
■ Describe the problems that the teams face in
defining and analysing the problem, as well as
identifying solutions;
■ Assess the appropriate use of QA tools such as the
flowchart and cause-effect diagrams;
■ Assess the way data are used for identifying a
problem and defining a solution (sources of data,
collection, analysis, appropriateness of data on
quality of care issues, constraints);
■ Identify the constraints to the work of the problem￾solving teams and at which level they can be best
addressed;
■ Suggest options for improvement of the work of
the problem-solving teams; and
■ Investigate any other relevant question identified
by the evaluation team.
Data collection methods used by the process improve￾ment subteam will include, but will not be limited to:
i) focus group discussion with the problem-solving
teams; ii) in-depth interviews with individual staff
members; iii) review of any record that documents the
problem solving activities; iv) interview with the link
facilitator; v) interview with the coach; vi) supervision
of problem solving activities; and vii) population based
survey, if needed.
A3
3. The support systems subteam will document the
functionality of the systems that support the quality of
care improvement activities, through data collection
and analysis of the supervision, coaching, QA train￾ing systems and documentation of QA activities, as
well as their recurrent costs. The team will focus on
the systems that support the QA activities, not the
support systems to the clinical and managerial work
of the health providers. More specifically, the support
systems team will:
■ List the existing support systems to the QAP in
Zambia;
■ Identify the support systems that are missing for
the QAP in Zambia;
■ Identify the standards according to which the
support systems have been established;
■ Identify the specific problems that each support
system is facing;
■ Evaluate the level of integration of the support
systems into the activities of the health staff at
various levels;
■ Identify the constraints in the operations of each
support system;
■ Suggest options for improvement to each support
system;
■ Identify the amount of resources needed for the
work of the problem-solving teams, including time
and cost;
■ Identify the amount of resources needed for
maintaining the support systems, including time
and cost;
■ Identify the amount of resources needed for
setting and communicating standards, as well as
measuring compliance, including time and cost;
■ Assess the costs involved in implementing the
recommendations made by the other consultants;
■ Suggest options for improving the efficiency and
reducing the cost of the QAP; and
■ Investigate any other relevant question identified
by the evaluation team.
Data collection methods used by the support systems
subteam will include, but will not be limited to:
i) focus group discussions with the central, regional and
district health teams in charge of the support systems;
 ii) in-depth interviews with health providers supposed
to benefit from the support systems; iii) review of
any record that documents the support system; and
iv) interviews with supervisors, coaches, QA trainers
and facilitators. Data collection methods used by the
cost-analysis expert will include, but will not be limited
to: i) meetings with individuals involved in the account￾ing of QAP activities; ii) in-depth interviews with
appropriate staff; and iii) review of documents and
appropriate reports.
The core evaluation team will identify the strong points
of the QAP, as well as the constraints and areas for
improvements. The core team will suggest several
alternatives for each improvement that will be presented
as detailed recommendations. Recommendations will
be made in the context of the Zambian Health Sector
Reform and, as such, will meet acceptability and
feasibility criteria.
A4
Standard Procedure for
Screening a Patient at OPD
1. Ensure particulars of client are recorded,
i.e. Full names and address.
2. Take down complaints and duration.
3. Ensure vital signs are taken.
4. Brief history of current illness.
5. Ensure findings of examinations are recorded.
6. Always indicate your diagnosis or provisional
diagnosis if you are not sure.
7. If laboratory investigations are ordered ensure that
the results are recorded on the card.
8. Prescribe rationally taking into consideration the
illness, cost and efficacy of the drugs/s.
Standard Procedure for
Admitting a Mother in Labour
1. Welcome the mother and make her comfortable.
2. HISTORY TAKING
Using MF110 (Labour Ward Admission Form) take
and record the following:
■ Social History, e.g. name, address, age
■ Family History
■ Medical History (previous illnesses & operations).
■ Previous obstetric history.
■ History of Labour – Time of admission.
3. OBSERVATIONS
Observe and record the following:
■ Temperature
■ Blood Pressure
■ Pulse
■ Respiration
■ Weight
■ Urinalysis (protein Sugar, Ketons)
Appendix B:
Protocols Developed by the
Kabwe District Health Office
4. EXAMINATION
Physically examine the mother from head to toe and
record the following:
Abdominal Exam Vaginal Examination
HOF Vulva
Lie Vagina
Presentation Cervix
Position OS
Descent Membranes
FHS Presenting part
Contractions Station
5. REFER to KGH if the following risk factors are
observed:
■ Poor obstetric history, e.g. three (3) or more
consecutive abortions/still births.
■ Very short stature (150 CM or below)
■ Very young Maternal Age < 15 yrs.
■ Elderly primigravida > 35 yrs.
■ Grand multipara
■ Pre-existing medical conditions e.g. Hypertension,
cardiac Disease, Diabetes.
■ Multiple Gestation
■ Abnormal Lie
■ History of difficult or prolonged labour
■ Previous operative delivery
Standard Procedure for
Referring a Mother in Labour
■ Record all your findings
■ Vital signs
■ Abdominal
■ Vaginal Examination
■ State reasons for referral
■ Empty Bladder
■ Commence I.V.I.
■ Pre-parative Care i.e.
■ Catheter
■ Shaving
■ Escort the patient to the hospital with all her records
■ Properly hand over at the hospital
■ Do not leave before handing over!
A5
Health Centre Supervision Checklist
1997
Date:_____________ Health Centre_________________________________
In - charge_________________________ Supervisors_______________________________
Part I - Core Health Facility Activities
A. Facilities, grounds and buildings
Walk around the health centre with the in-charge and answer the following questions:
YN 1. Are the grounds around the health centre and staff houses free from waste?
YN 2. Is there a functioning and clean toilet for staff and patients at the facility?
YN 3. Is there adequate seating and space for waiting patients?
YN 4. Are all of the rooms in the health centre clean?
¨ floors swept
¨ trash put in trash boxes and not left out
YN 5. Are all linen materials which are in use and in storage clean?
YN 6. Is there an ORT corner fully functional with the following present—
¨ table, seating for mother and child
¨ potable water (_ if supply appears adequate)
¨ 2 large cups (500 ml), 2 medium cups (250 ml)
¨ 1 tablespoon (10 ml), 1 teaspoon (5 ml)
¨ ORS sachets (_ if supply appears adequate)
¨ the ORT register is complete
YN 7. Private consultation room for confidential counseling
(Family Planning, STDs, etc...) and private physical exam (gynae, STDs)
B. Records, Reports and Wall charts
YN 8. Is the following information displayed on wall charts or maps—
¨ map of health centre catchment area displaying—
m boundary of catchment area m communities
m roads m markets
m health centres m NHMTs
m CHWs m trained TBAs (Traditional Birth Attendants)
m rivers, springs and other major water sources
Appendix C:
Examples of Supervision Checklists Used by Districts
A6
o vital statistics for the catchment area—
m total population for current year m under-fives
m under-ones m women of childbearing age
m expected pregnancies in current year
o immunization monitoring chart correctly filled in and up-to-date showing—
m the number entered for measles vaccinations is the same as on the MF-47
m the cumulative numbers are added correctly; and
m the point is plotted correctly to correspond with the cumulative vaccinations
m current estimate of measles vaccination coverage
o Is the first-antenatal-attendance monitoring chart displayed, correctly filled in and up-to-date?
m the number entered for first antenatal visits is the same as on the MF-47
m the cumulative numbers are added correctly; and
m the point is plotted correctly to correspond with the cumulative visits and the month
9. What is the current estimate of first antenatal attendance coverage for this area? ___%
YN 10. Is there a copy of the MF-47 on file for last month, and has it been sent to the district?
C. Review of the Outpatient Register for the past month
YN 11. Is there a registration book in which diagnoses are listed?
If Yes —
How many of the patients seen in the last month had more than one diagnosis written in the
register?___
a.) of the last 10 cases of “diarrhea” or “gastroenteritis” or “A.G.E.”, how many were treated with
ORS? ___
b.) of the last 10 cases of upper respiratory tract infection (URTI) or cold/not pneumonia, how many
were treated with antibiotics? ___
D. Review of the Antenatal Register for the past month
YN 12. Is the antenatal clinic register correctly filled in and up-to-date?
YN 13. Are there notations in the antenatal register for high-risk pregnancies, and are
 special follow-up activities specified for these women?
YN 14. Have the following antenatal services taken place in the past quarter?
¨ outreach antenatal clinics?
¨ community participation regarding maternal care and referral?
¨ community discussions on danger signs of pregnancy and delivery?
YN 15. Is there a register of RPR results for antenatal patients?
E. Review of the Tuberculosis Treatment Register for the past month
YN 16. Is the tuberculosis treatment register correctly filled in?
Note: Answer “No”, if data missing or incorrect. Explanations should be given for missing data.
 17. How many TB patients have defaulted in the past 3 months?______
YN 18. Do records reflect that defaulters were visited at home?
A7
19. How many sputum positive reports at two months were not followed up?_____ (number)
20. Number of new patients started on treatment in the previous 3 months? _____
21. Number of patients completing treatment in past 3 months? ______
22. Number of TB patients not responding to treatment referred to the district? _____
23. How many patients are receiving DOTS treatment? _____
F. Financial
24. How much money did you collect from user fees, since the last supervision visit?______
YN 25. Is there a financial committee that monitors expenditures and are minutes of meetings available?
YN 26. Does the total amount of money collected on user fees last month correspond with the total amount of
the receipts in the receipt book?
YN 27. Is the cash from user fees for last month available or if it was collected, can it be
confirmed by the District Accountant that it has been banked?
YN 28. Does the number of receipts issued last month for user fees match with the number
of user fee paying patients according to the outpatient (+/- inpatient) register(s)?
YN 29. Does the staff know which patients are exempted from paying user fees?
¨ Children under 6 years and patients above 65 years
¨ Treatment for chronic diseases like TB, diabetes, hypertension
¨ STD, antenatal and other MCH services
¨ If patients have truly no means to pay (approved by the Dept. of social welfare)
G. Services provided
YN 30. Are the following services available more than once weekly? (Supermarket)
¨ childhood immunization
¨ antenatal care
¨ family planning
YN 31. Is there a UCI outreach programme? If yes, how many times last month?____
How many sites last month? ______
YN 32. Is there an AIDS home-based care programme functioning?
If Yes— how many patients are presently receiving treatment? ____
¨ records complete
What supplies are being distributed in past month days?
H. Personnel
33. How many staff and of which cadre are working at the health centre?
___ Doctor ___ Environmental Health Technician
___ Clinical Officer ___ Laboratory Technician
___ Registered nurse ___ Classified Daily Employee
___ Enrolled nurse ___ other: ________________________
___ Community Health Worker ___ other: ________________________
A8
YN 34. Was there a staff meeting held last month? Note: To answer “Yes” there must be minutes of the
meeting, including names of the participants
YN 35. Were there other health facility committees which met in the past 3 months?
Specify which (housing, drugs & therapeutics, discipline etc) and verify minutes
I. Equipment, Supplies and Stationary
YN 36. Are each of the following items of equipment present and in working order?
o salter scale/weighing bag
o adult scale
o blood pressure cuff
o clinical thermometer
o foetal stethoscope
o stethoscope
o timer for health worker seeing children
YN 37. Is there a dental tray present?
YN 38. Are supplies of the following available in adequate quantities?
¨ water for washing hands
¨ potable water
¨ soap for washing hands
¨ fuel for steriliser stove
YN 39. Are adequate supplies of the following forms/stationary available?
¨ outpatient register
¨ tally forms for UCI
¨ MF-47
¨ Notifiable disease report form
¨ receipt books
¨ OPD cards/record books
YN 40. Are laboratory services available?
If Yes — is the following equipment available?
¨ microscope ¨ haemoglobinometer
¨ centrifuge ¨ reagents for blood films (1 mo supply)
¨ cell counter ¨ urine dipsticks (1 mo supply)
¨ glass microscope slides (1 mo supply)
J. Vaccine Supply and Cold Storage
YN 41. Has the vaccine refrigerator maintained an acceptable temperature, and is its present temperature
between 0°C and 8°C?
YN 42. Has the refrigerator temperature chart been filled out twice daily for the past month?
43. For how many days in the last 3 months does the stock books show each of these vaccines out-of-stock?
Measles:___days; DPT:___days; Polio:___days; BCG:___days; TT:___days
¨ Vaccine stock book not current
Y N 44. Is a 2 week supply of fuel available?
11. Is there a vaccine stock book?
o steam steriliser
o steriliser stove
o measuring tape (for measuring fundal height)
o vaccine carrier
¨ ice packs
¨ wash stand and basin
¨ vaginal speculum
¨ sterile syringes (1 mo supply)
¨ sterile needles (1 mo supply)
¨ cotton wool
¨ IUD kits
¨ Children’s clinic cards
¨ ante natal cards
¨ EDP report forms
¨ blank stock cards
¨ TB cards
A9
K. DRUGS AND CONTRACEPTIVES
45. stock card recorded = days o/s
present actual last month
Child health/Malaria oYes oNo oYes oNo ___ days
Chloroquine tabs oYes oNo oYes oNo ___ days
Cotrimoxazole tabs oYes oNo oYes oNo ___ days
ORS sachets oYes oNo oYes oNo ___ days
Pyrimethamine-sulfa tabs oYes oNo oYes oNo ___ days
Vitamin A capsules/tabs oYes oNo oYes oNo ___ days
Maternal health/Family Planning
Methylergotamine injection oYes oNo oYes oNo ___ days
Condom oYes oNo oYes oNo ___ days
Contraceptive pill oYes oNo oYes oNo ___ days
Depoprovera oYes oNo oYes oNo ___ days
STDs/Tuberculosis
Benzathine penicillin oYes oNo oYes oNo ___ days
Ethambutol tabs oYes oNo oYes oNo ___ days
1 mo supply oYes oNo
Isoniazid + Ethambutol tabs oYes oNo oYes oNo ___ days
1 mo supply oYes oNo
Pyrazinamide oYes oNo oYes oNo ___ days
1 mo supply oYes oNo
Rifina oYes oNo oYes oNo ___ days
1 mo supply oYes oNo
Other
IV Fluids and giving sets oYes oNo oYes oNo ___ days
Ferrous Sulphate oYes oNo oYes oNo ___ days
Folate oYes oNo oYes oNo ___ days
YN 46. If there is a maternity facility, the following drugs, at a minimum, will be present in
adequate amounts for the deliveries which occur in this unit—
¨ gentamicin ¨ Vitamin A
¨ procaine penicillin ¨ tetracycline ointment
¨ anticonvulsants ¨ lidocaine 2%
¨ antihypertensives ¨ amoxicillin
¨ BCG ¨ ampicillin injectable
¨ mebendazole
Y N 47. Are drugs handled in an appropriate manner?
¨ How are new drugs stocked when they arrive? (_ for: stock rotation)
¨ What do you do with expired drugs? (_ for: return to district)
¨ Are the any drugs on the floor? (_ for: none)
A10
L. MALARIA/CHILD HEALTH
(including UCI, diarrhea, pneumonia, nutrition and malaria)
48. Observation of Assessment of the Sick Child 2 Months to 5 Years
Health Worker’s Name _________________________ CO ZEN EHT RN MD CDE
Health Worker trained with 11 day MCI course Yes No Age of child: ___ months
¨ Does the health worker greet the mother?
Does the health work ask about or does the mother volunteer—
Does the health worker examine for—
Danger signs
¨ not able to drink or breast feed? ¨ convulsions?
¨ vomits everything? ¨ lethargy or unconsciousness?
Cough or difficulty breathing
¨ for how many days? ¨ count breaths?
¨ raise the shirt? ¨ look for chest indrawing?
Diarrhoea
¨ for how many days? ¨ offer fluid or observe breastfeeding?
¨ is there blood in the stool? ¨ skin pinch of the abdomen?
Fever in the past 24 hours
¨ for how many days? ¨ examine for stiff neck?
¨ has Chloroquine been given at home for this illness?
Immunization
¨ ask to see immunization care? ¨ due for vitamin A?
Feeding (if under 2 yrs or very low weight)
¨ do you breastfeed your child? ¨ does the child take any other food?
¨ if yes, how many times in 24hrs? ¨ if yes, what foods or fluids?
¨ how many feedings per day?
49. Observation of Treatment and Counseling — 2 Months to 5 Years
Drugs prescribed Does health worker correctly explain—
____________________ o Dose o Frequency o Duration
____________________ o Dose o Frequency o Duration
____________________ o Dose o Frequency o Duration
____________________ o Dose o Frequency o Duration
YN 50. Does a health worker ask any open-ended questions to determine whether the
mother understands how to give the medicines prescribed?
A11
YN 51. Does the health worker advise—
o Increase the frequency of meals or breastfeeding
o Reduce or stop other foods other than breastmilk
o Begin or increase the frequency of complementary foods
o Give food that is thicker or enriched (e.g. with sugar, oil, ...)
YN 52. Does the health worker advise on when to bring the child again?
Y N 53. Does the health worker advise mother to return with child immediately for:
o Develops a fever or fever does not go away
o Drinking poorly (if child has had diarrhoea)
o Blood in the stool (if child has had diarrhoea)
o Breathing fast or difficult ( if child has been coughing)
o Child becomes worse for any reason
Part II Specialised Health Centre Activities
M. Community Partnership/Health Communication
54. How many NHCs are in the catchment area of this health centre? ____
and how many are active? ____
YN 55. Was there a meeting with all the NHMCs in the catchment area last month?
¨ if yes are there minutes available
¨ if no, why did meeting not take place?
56. Describe one activity carried out by/with NHCs in the last quarter:
_____________________________________________________________________________________________________________________________________________________________________________________________________________________________
57. What is the total number of CHWs (active and inactive) in the catchment area? ___
How many of these are active? ____
58. How many CHWs submitted any reports in the last 3 months? ___
Note: confirm that reports are on file
YN 59. Was each active CHW supported at least once in the last quarter?
Note: To answer, “Yes”, there must be a report of the support visit.
60. What is the total number of active trained TBAs in the catchment area? ___
61. How many active trained TBAs submitted any reports in the last 3 months? ___
Note: Confirm that reports are on file
YN 62. Was each active trained TBA supported at least once in the last quarter?
Note: To answer, “Yes”, there must be a report of the support visit.
63. Approximately how many times did staff spend outside the health centre in the last month providing
outreach services on one of the 6 health thrusts? ____ times
A12
64. How many different sites did staff visit outside the health centre in the last month to provide outreach
services on one of the 6 health thrusts? ____
Note: ask about which sites were visited and which community groups participated
65. What development NGOs are functioning in the catchment area, and what programmes are they
undertaking?
66. What activities are being conducted by the health centre in conjunction with NGOs?
N. ENVIRONMENTAL HEALTH
Sanitation and waste management
YN 67. Does the health worker know—
¨ the population of the catchment area?
¨ the number of households in the catchment area?
¨ the average number of persons per household?
¨ how many households have pit or VIP latrines?
¨ how many households have flush toilets? (where applicable)
¨ how many house holds have refuse pits?
YN 68. Are there promotional programmes underway on excreta disposal in this area?
Y N 69. Does the health worker maintain statistics on faecal borne diseases in this area?
70. How many new pit latrines have been constructed in the past quarter?
71. How many health education meeting on hygiene has he held in the past month?
______(check diary for number)
YN 72. Is there uncollected or undisposed waste lying around the area?
¨ if Yes, has the health worker suggested any alternative methods of disposal to the community?
(_ if yes)
¨ if Yes, what is this alternative?
____________________________________________________________________________________________________________________________
YN 73. Is the excreta and waste disposal system at the health centre adequate and safe?
YN 74. Is there a malaria control programme being conducted?
If Yes, what is included in the programme?
75. How many visits to the community has the health worker conducted in the past month in conjunction
with a community representative to check on sanitation issues? ________
Water
YN 76. Does the health worker know—
¨ the number of households served by protected wells or boreholes?
¨ the number of households served by communal taps?
¨ the number of households with mains water connections?
¨ the number of villages with protected wells or boreholes?
¨ how many villages are within 0.5 km of a protected water source?
A13
YN 77. Does the health centre have a stock of water treatment chemicals for
 emergencies?
YN 78. Does the health worker regularly check the chlorine level of the water supply to
 the health centre (check his or her record book)?
79. How many NGOs or cooperating agencies have water activities in this area? ____
YN 80. If these are present, does the health worker attend their meetings? (check diary)
YN 81. Is the health worker using any IEC materials to protection of water sources?
YN 82. Has the health worker conducted any meetings about water supply sources with
 communities without protected sources? (Check the diary)
83. How many water sources has the health worker inspected in the preceding quarter about which he or
she has written a report? ______
O. OBSERVATION OF FAMILY PLANNING SERVICE DELIVERY
YN 84. Did the health worker display the following actions
¨ greeted the patient in a friendly manner
¨ encouraged questions
¨ ensured privacy
¨ provided a health talk about family planning
¨ carried our screening or management of STDs
YN 85. Were the following clinical procedures carried out?
¨ TT status checked
¨ if needed TT given
¨ blood pressure checked
¨ patient checked for anaemia
¨ legs checked for oedema or varicose veins
¨ weight checked
¨ abdomen palpated
¨ RPR done during this pregnancy
P. HEALTH CENTRES WITH A MATERNITY FACILITY
General observation
YN 86. Does the facility have the following—
¨ delivery room
¨ telephone or radio in working condition
¨ transport facilities for patients (where applicable)
¨ adequate light (hurricane light minimum)
YN 87. Did this health centre maintain the following records for recent deliveries—
¨ partograms filled in properly
¨ blood pressure checked and recorder four-hourly
¨ foetal heartbeat checked hourly
¨ vaginal examination done four-hourly
A14
Supplies and equipment for maternity and postnatal services
YN 88. The following delivery equipment is present
¨ chitle forceps ¨ infant laryngoscope
¨ episiotomy scissors ¨ bag & mask for neonatal resuscitation
¨ suture needles ¨ neonatal mucus extractor
¨ needle holder ¨ clinical thermometer
¨ artery forceps ¨ tooth forked forceps
¨ cord scissors ¨ incinerator
¨ ring (sponge) forceps
YN 89. The following consumables are present in adequate amounts for the deliveries which
occur in this unit—
¨ linen/cloth to dry the baby ¨ IV giving sets
¨ gauze/cotton wool ¨ disinfectant
¨ plastic sheeting ¨ partograph forms
¨ cord clamps/ties ¨ syringes
¨ suture material ¨ needles
¨ maternity pads ¨ gloves
Observation of services in a postnatal clinic
YN 90. The following activities were carried out by the health worker—
¨ checked the date of delivery by card and by asking the mother
¨ verified how many postnatal visits had been made after this delivery
¨ carried out a physical examination on the mother including—
m abdomen m eyes/tongue
m vagina m breasts
m blood pressure
¨ examined the baby for any abnormalities, colour, weight, activity
YN 91. The following health education was provided to the mother—
¨ family planning
¨ immunization
¨ breast feeding
¨ growth monitoring
A15
Ask yourself the following questions:1
A. ORGANIZATION
1. Is there a map of the catchment area that shows all the information that exists in the catchment area?
- Roads
- Villages
- Sub-Catchment Boundaries
- Markets and Stores
- Schools
- Churches
- Rural Health Center
- Health Posts
- Catchment Area Boundary
2. Is there a “schedule of activities” for last month?
*Review it before you mark “yes”.
3. Was there a staff meeting held last month?
*To answer “yes” there must be minutes of the meeting which include the names of participants.
4. Was there a meeting with all the TBAs/CHWs in the catchment area last month and health neighborhoods.
*To answer “yes” there must be minutes of the meeting which include the names of the participants.
5. Is there a copy of MF 47 on file for the last month?
*See the copy.
6. Is there a working:
- Refrigerator
- Clean
- Water bottles for temperature
- Maintenance
7. Is the fridge temperature being maintained?
Charting of feezer temperature twice a day.
- Between 0 – 20o
 C
- Cold space temperature between 0 - 8o
 C
9. Do you have enough vaccines in stock?
- BCG
- DPT
- POLIO
- MEASLES
10. Are the vaccines valid?
Any expired vaccine in stock?
11. Is there a vaccine stock book?
Do you have a vaccine stock record usage for the past week?
12. Is there a constant supply of electricity?
1 This second example of supervision checklists was untitled.
- Water Source
- Lake
- Streams
- Springs
- UnPro Wells
- Protected Wells
- Fishing Camps
- Agriculture Extension Units
- Graveyards
- Population distribution
A16
13. Was the longitudinal Immunization Register used last month?
*See the dates-of-immunization in the Register.
14. Are the data in the Immunization Register for last month the same as the numbers recorded in the immunization
section of MF 47?
*Select one dose of any vaccine given to children <1 and check each page of the Register in the column for that
dose. Add up all immunizations given for that dose during the previous month. If the number recorded on MF
47 for the last month is the same as your calculate number, mark “yes”. If the number is different or if the
Register is not used or if there is no copy of MF47 available, mark “no.”
15. Is the immunization monitoring chart filled out correctly for last month?
*Study the “immunization monitoring chart” to see if:
- The numbers entered for BCG and Measles are the same as the number of children <1 on the copy of MF 47
for last month;
- The cumulative numbers for each are added correctly;
- The points are plotted correctly for the cumulative number and lines drawn connecting the points.
*If all three steps are correct, mark “yes” and review coverage status.
B. NUTRITION PROGRAMME
16. Was Vitamin A given out last month to all eligible children and seriously ill patients according to the Vitamin A
Schedule?
*See the vitamin A column in the Immunization Register and the “Treatment” column of the OPD Register and
compare entries to the instruction on the Vitamin A schedule.
17. Was growth monitoring conducted weekly during last month for “at-risk” children <5 years of age?
*See the dates in the Weekly Growth Monitoring Register.
18. Were any “at-risk” children <3 years of age followed-up at home last month for nutritional counseling?
*See the “Home Visit” Register. If any home visits were made to children <1 and nutrition subjects were
discussed as noted in the “Remarks” section, mark “yes.” If there is no Home Visit Register or no visits noted
for children <1 or no nutritional counseling noted, mark “no.”
19. Does the data on MF 47 for last month correspond to the data in the longitudinal (monthly) Growth Monitoring
Register?
*Compare a few data sets – if even one set does not correspond, the answer is “no.”
C. CONTROL OF DIARRHOEAL DISEASES
20. Is there an ORT Corner ready for service?
*To mark “yes” all the following must be present on a table/shelf ready for use:
- One (1) Litre of clean/safe water (covered)
- Two (2) cups (500 ml)
- Two (2) Cups (250 ml)
- One (1) Tablespoon (10 ml)
- One (1) Teaspoon (5 ml)
- Fifty (50) sachets of ORS
21. Are there at least 100 sachets in stock?
*Count them.
22. Are the data on diarrhoeal disease on MF 47 for last month the same as in the OPD Register?
*Count and compare cases <5 years of age.
A17
D. ENVIRONMENTAL HEALTH (EH)
23. Was a “Village Inspection” carried out last month?
*See EH records, registers, reports, etc.
24. Were all appropriate columns in the Village Inspection Register completely filled out for last month?
25. Is the information in the EH Register for last month the same as the data in the EH Report?
26. Was an Iodine Test for household salt carried out last month?
E. TUBERCULOSIS/LEPROSY
27. Is there a TB Register in use that lists the patients currently under treatment?
*NOTE: Remember, it is important that the Leprosy Register and reports are reviewed right after the TB data is
reviewed. Although the “yes” and “no” answers refer specifically to TB programme activities, Leprosy
programme activities must also be reviewed.
28. Is there a copy of last months TB Report on file?
29. Does the data in the report for last month correspond to the information in the TB
Register?
*Compare a few data sets – if even one set does not correspond, the answer is “no”.
30. Is there at least a one month supply of drugs available for each registered patient?
*Calculate a one month supply of drugs by multiplying the number of registered TB patients by:
- Ethambutol 400 mg (60 tablets)
- Isoniazid 300 mg (30 tablets)
*If there are no patients registered, mark “yes.”
F. MALARIA PROGRAMME
31. Is the number of malaria cases recorded on the MF 47 for last month the same as the number of cases recorded
in the OPD Register?
*Add up either new attendances <5 or >5 in the OPD Register and compare with the figure reported on MF 47
last month.
G. MENTAL HEALTH
32. Is there a register available in which all mental health cases and referrals are recorded?
*What was the date of the last case:__________________
33. Is there at least a one month supply of Phenolbarbitol in stock for each registered case of epilepsy?
*To answer this question note the number of cases registered and calculate stock required:
-30 mg TDS x 30 = 90 (30 mg) tabs. per month (x) number of patients =
H. MATERNAL HEALTH
34. Was an antenatal clinic held last week?
*Verify by any record, tall sheet, etc.
35. Was each appropriate column in the Antenatal Register completely filled out for each antenatal visit last week?
*Review selected entries in the Register for last week.
A18
36. Were “at risk” pregnant women correctly identified in the Antenatal Register during the first antenatal visit last
month?
*Review the _____ “at risk condition” columns. If any conditions are ticked, note whether the last “At Risk:
Yes/No” column is filled and special follow-up activities are noted.
37. Is a “yes” or “no” recorded for each new antenatal visit last month in the “Suspected STD” column of the
Antenatal Register (and referral noted if “yes”)?
38. Does the data in the Antenatal section of MF 47 for last month correspond to the information in the Antenatal
Register?
*Compare a few data sets – if even one set does not correspond, the answer is “no.”
39. Is the “First Antenatal Visit” monitoring chart filled out correctly for last month?
*Study the “Antenatal Visit Monitoring Chart” to see if:
- The number entered for first antenatal visits are the same as the numbers on the copy of MF 47 for last month;
- The cumulative numbers are added correctly;
- The point is plotted correctly for the cumulative number and the line drawn connecting the point.
*If all three steps are correct, mark “yes” and review first-antenatal-visit coverage status.
I. DELIVERIES AND POSTNATAL CARE
40. Were the weights of all live births recorded in the Delivery Register for last month?
*If even one weight was not recorded in the Register, mark “no.”
41. Is the information on deliveries recorded on MF 47 for last month the same as the data in the Delivery Register?
*Compare a few data sets – if even one set does not correspond, the answer is “no.”
42. Was Tetracycline Eye Ointment put in the eyes of all newborns delivered in the health facility last month?
*Review the Delivery Register for last month.
A19
Appendix D:
Guides for Interviews Conducted by the Evaluation Team
QUESTIONS TO BE ASKED AND ISSUES TO BE ADDRESSED
BY THE STANDARDS SUBTEAM
The purpose of the evaluation will be presented and the concept of standards will be explained. We will reinforce the fact
that standards have to be developed, communicated, and accepted by the users. We will look at the documents on stan￾dards of care (clinical guidelines) as well as the management standards (standard operating procedures). We will identify
the documents that present results of surveys aimed at measuring performance. We will try to present the results in a table.
If possible, we will try to link the measurement of performance with problem solving activities (i.e., did a problem solv￾ing methodology lead to an improvement in performance and was poor performance identified as a problem?). We will
look at the integrated guidelines for frontline health workers and will try to make a judgement on their scientific validity,
reliability, feasibility and clarity according to the users’ perspectives. We will ask health managers and health workers to
look at their measure of performance over time and try to interpret a change in performance.
Date:
Region:
District:
Structure:
Persons met (Name and function):
Questionnaire #
The Development and Adaptation of Standards
Questions for the Central Level
■ How are clinical standards of care developed or adapted in Zambia? (The integrated guidelines, the IMCI, etc.)
■ Are you using evidence-based medicine to develop national standards?
■ Who was involved in developing standards?
■ Are the users involved in the development/adaptation process?
■ Is there an official policy that defines the process to develop standards?
■ How are standards for management of health services developed?
■· What are the problems/constraints you encountered with the development/adaptation of both types of standards?
■ How best would you address these problems?
A20
■ Which resources were used and how much was spent to develop/adapt the standards?
■ How are clinical standards revised/updated?
■ Is there a policy that describes the process to revise standards?
■· In which documents are the official standards of care and SOP presented?
■· Are standards developed from a patient perspective? (patient’s charters, patients rights)
Questions for regional and peripheral levels
■ What would be your definition of a standard?
■ Have you been involved in the development and/or adaptation of clinical standards of care? If yes, what exactly did
you do?
■ Have you been trained in developing standards? If yes, did you use your skills?
■ Which problems did you encounter?
■ How best would you address them?
■ What documents on standards for clinical care do you know?
■ Do you have these documents (clinical guidelines, standards operating procedures)?
■ How do you use them? How helpful are they? Clear? Feasible? Do you agree with them?
1. The Communication of Standards
Questions for the central level
■ What is the role of the regulatory bodies and clinical associations in the communication of standards?
■ What is the cost of disseminating the standards? (IMCI)
■ How did you communicate/disseminate the different types of standards to the health providers and managers?
■ Is there an official strategy for disseminating standards?
Questions for the providers and managers
■ How did you learn about the standards?
■ Are you regularly reminded of the standards? Through which mechanisms?
2. The Measurement of the Compliance with Standards of Care
at Health Facility Level
Questions for health managers
■ Is measurement of performance someone’s full time job?
■ Who is in charge of it?
■ Which mechanism(s) did you set up to measure performance of health workers?
■ Which standards do you use? Do you measure compliance with standards?
■ Which instrument/tool do you use to measure compliance with standards?
■ What are the indicators of performance that you collect?
A21
■ How do you use the performance results?
■ Are there any rewards for good performance and sanctions for poor performance?
■ When performance is not satisfactory, what do you do to understand the underlying reasons?
■ Do you measure competence?
■ How is feedback on performance given to health providers?
■ How often do you monitor performance?
■ Do you focus more on low performers?
■ Do you share performance results among health workers/facilities?
■ Which problems/constraints do you encounter to measure performance?
■ How would you best address these problems?
■ What is the cost of measuring performance?
Questions for health providers
■ How is your performance assessed and by who?
■ Do you measure your own performance?
■ How do you measure your own performance?
■ What are the benefits of knowing your performance?
■ Did measuring your performance help you to improve your practice?
■ What are the consequences of poor performance?
■ What is your incentive to improve your performance?
■ Which changes would you suggest to measure your performance?
A22
QUESTIONS TO BE ASKED AND ISSUES TO BE ADDRESSED
BY THE PROBLEM SOLVING SUBTEAM
Semi-structured questionnaire with team evaluation and “difficulty index”
Part 1: Basic information about training and teams in the districtfor interviewing coaches,
link-facilitators, or district officers
In the district, how many people have been trained in quality assurance and how long was the training and what
did it cover?
Coach training. Then what training did they do, of whom, and how long, covering what?
How many problem-solving teams have been formed in the district?
How many really followed the 5 step cycle and used the methods properly?
Where Staff at centre total No. Staff in team No. Coach in team (Y/N) and O in C Y/N)
1) Coach? Y/N. O in C Y/N?
2) Coach? Y/N. O in C Y/N?
Other teams, Etc.
When did the team start meeting? Stop actively meeting?
1)
2)
Did any teams finish their first problem. Which teams finished a second cycle?
1)
2)
Did any do more cycles?
How many are active now – when and where?
What were the type and relevance of the problems identified by the teams?
Health centre Title of problem Problem significance Problem appropriateness
for a team and 5 step cycle
(Time taken to complete the cycle)
1)
2)
Part 2: Quality problem-solving teams – facility visit
Facility Name: Informant (s): (coach or trainer (primary informant))
1)
2)
Documentation on the teams work available?: None, Notes (partial, complete), QA storybook.
A23
Question informant at the facility to validate the information given by the coach about:
Staff at centre total No. Staff in team No. Coach in team (Y/N and O in C?)
Training in quality the coach/trainer and others have had:
14 day, QPST 5 day, QPST 3 day Other
Subjects of training:
The month/year the team start meeting m/y stopped actively meeting Finished the full 5 step cycle? Y/N
(Verification e.g., Check documentation)
What were the type and relevance of the problems identified by the team
Title of problem Problem significance (0-5) Problem appropriateness for a team & 5 step cycle 0-5
(Time taken to complete the cycle)
How were the different steps of the problem solving cycle carried out and were the methods chosen and used
correctly?
Step and methods Rating & Justification for rating
Step 1 Identifying the problem
1) Listing
2) Prioritising matrix and vote using criteria
Step 2 Problem statement
3) Precision, measures, target specified
Step 3 Identifying who needs to work on the problem
4) Were the right people on the team, or involved in other ways to analyse,
collect data and solve the problem? (e.g., Who should have been involved,
were people added to the team, were others involved (e.g., community?)
Step 4 Analysing and studying the problem to identify major causes
5) Bubble or other cause effect diagram
6) Listing of possible causes
7) Creation of data matrix with data gathering questions, sources and methods.
8) Data gathered and recorded (quantitative or qualitative)
9) Validity and reliability of data for answering the problem
(e.g., sample for interviews, or trending quantitative data,
or validity of statistics)
A24
Step and methods Rating & Justification for rating
10) Data analysis and interpretation
11) Use of data to determine root cause
Step 5 Developing and choosing solutions
12) A list of possible solutions
13) Link between solutions and Step 8 (especially 8), 10) and 11))(how well
does the list relate to the data and determination of root cause)
14) Choice of solution – what was the method to choose, and how good
was the choice?
Step 6 Implementation evaluation of QI work
15) How well was the solution implemented? (Plan and persistence)
16) Were data gathered again to check the effects of the solution?
17) What difference did the solution make and to whom (successful)
( data or judgement?)
18) How well the team followed up 6m/12m later or kept monitoring?
Total Score %(Out of 90)
Which problems did the team meet in defining and analysing the problems and in identifying and implementing
solutions?
Step and methods – what were the difficulties? Difficulties experienced (0-5) and comments
Step 1 Identifying the problem
1) Difficulties in listing problems
2) Difficulties in using a prioritising matrix and in voting using criteria
Step 2 Problem statement
Difficulties in formulating a precise statement with measures and a
target specified.
Step 3 Identifying who needs to work on the problem
4) Difficulties choosing and getting the right people on the team,
or involved
Step 4 Analysing and studying the problem to identify major causes
5) Difficulties making a bubble or other cause effect diagram
6) Difficulties in listing possible causes
7) Difficulties in creating a data matrix, with data gathering questions,
sources and methods.
A25
Step and methods – what were the difficulties? Difficulties experienced (0-5) and comments
8) Difficulties in gathering and recording data
9) Difficulties in getting valid and reliable data to answer the
problem (e.g., sample for interviews, or trending quantitative
data, or validity of statistics)
10) Difficulties in data analysis and interpretation
11) Difficulties using data to determine root cause
Step 5 Developing and choosing solutions
12) Difficulties listing possible solutions
13) Difficulties making a link between solutions and the last step 4
 (especially sub-steps 8, 10 and 11) (how well does this list relate
to the data and determination of root cause?)
14) Difficulties choosing a solution – what was the method to choose,
and how good was the choice?
Step 6 Implementation and evaluation of QI work
15) Difficulties in implementing the solution? (Plan and persistence)
16) Difficulties gathering data again to check the effects of the solution?
17) Difficulties following up 6m/12m later or continuing monitoring?
Total “Difficulty index” DI = % (out of 95)
How well did the teams perform in gathering and using data?
Refer to the assessment above:
1. Did they collect data?
2. Data gathered before their intervention?
3. Data gathered again after their intervention?
4. Data gathering method or source validity and reliability for the problem? (other sources/methods better?)
5. Data analysis and graphing?
Strong points? Weaknesses?
What was the level of competence of staff in choosing and using problem solving methods?
■ Make a general assessment how well the team performed in choosing and using the methods (0-5).
■ Why would you give the score you have given – what did they not do or do, or do well which justified this score?
■ What would help improve their competence in choosing and using the methods and following the steps?
A26
What were the results and progress of the team according to their self-monitoring?
Any measured changes to quality after solution implementation:( the difference and for whom)?
If not, did members of the team or others notice changes to quality? What changed?
How could they convince others that this was due to the quality work of the team and not to something else?
Were there any indirect benefits of the teams work and their meeting together?
What was the average duration of the problem solving cycle and how could it be shortened?
Note how long each cycle took :
1st cycle from to length: 3rd cycle from to length:
2nd cycle from to length: 4th cycle from to length:
How could the time be shortened?
How cost-effective was the team?
Try to make an assessment of the value of the benefits, for the costs (mainly in staff time).
Could this time have been used to better effect (opportunity cost)?
Resources used
What were the amount of resources needed for the work of the problem-solving team (time mainly)?
Problem 1: How many team members How many meetings to solve the problem
How long was each meeting on average Total time in person hours:
Problem 2: How many team members How many meetings to solve the problem
How long was each meeting on average Total time in person hours:
Problem 3: How many team members How many meetings to solve the problem
How long was each meeting on average Total time in person hours:
Problem 4: How many team members How many meetings to solve the problem
How long was each meeting on average Total time in person hours:
Benefits
For patients
For neighbourhoods
For staff
Indirect benefits
Ratio of resources/costs to benefits
Why do you think teams were not formed in some places, or did not finish their work?
■ Why do you think they did not form after people have been trained? Who was trained? Will a team form if an Officer
in Charge is not trained? How much training?
■ Why do you think they did not finish the first problem cycle?
■ Why do you think they did not start a second problem-cycle?
A27
What predicts the success or failure of a Quality problem-solving team?
In your view, which of these are the most important for ensuring the team follows the steps, finishes a problem gets re￾sults, and goes on to other problems:
■ Coach in team? Y/N or visiting regularly? (how many times and duration between visits)
■ Facility No of staff (at any time less than ten?)
■ Team size – numbers?
■ How many staff left the team?
■ Frequency of meetings (any long breaks?)
■ Officer in Charge on the team? Supportive but not on team?
■ Problem statement poor or too complex?
■ Diverging from the steps inappropriately?
■ What training did the team leader have? (Length and content)
■ Other member’s training?
■ De-linking?
■ Other?
What was good and bad about the team and its work?
How could the work of the team be improved?
Lessons and advice for others?
Do’s and don’ts?
What should we recommend which would help you most in your QI work?
Risk of Team Failure Index and Scoring Method
1: Risk of failing to form a team
Factors and conditions predicting no team formation Risk of Failure Score
There are less than ten staff at the facility, and the team has (<10= score 10 (10-25, score 4, 25-50 score 1,
not been a coach training site. over 50 score 0)
There is no coach in the team, or their visits are over one (no coach, score 5, visits over 1 month score 4, visits
month in frequency. for every meeting, score 1; coach in team, score 0)
The officer in charge does not have training, is not in the team, (No officer in charge in team score 4, no active support
or where the o in c does not actively support the team’s work score 5)
There are less than 4 people in the team (less than 4 score 5; 4 score 4; 5 score 3; 6 score 2; 7
score 1; and 8 score 0)
There is less than 5 person days training represented in the (less than 5 days score 5; 5-7 person days score 3, 10 or
team (e.g., one person with 5 days or over training, or five over person days score 0)
people with one days training)
There is poor morale (e.g., connected with de-linking) or a (Poor morale and unprofessional culture as judged by
culture that does not value professionalism experienced clinician visitor, score 5; good morale and
professionalism, score 0)
A28
2: Risk of not successfully completed a first problem cycle
Factors predicting failure to finish the first cycle Risk of Failure Score
50% of team members leave during a year (50% and over score 5)
Meetings are less frequent than once a month and there are no (Less frequent than one month, score 5, monthly
gaps of longer than two months between meetings score 3, twice weekly score 1, weekly score 0)
The team chooses a problem which is broad or complex for a (If so, score 5)
first problem, or does not precisely define its problem statement,
or works on more than one problem at a time
The team chooses and works on a simple problem that does not (If so, score 5)
need a full 5 step cycle.
The team spends more than four months on one step or had not (If so, score 5)
followed the steps of the cycle when it is appropriate to do so.
And in addition, the earlier factors:
There are less than ten staff at the facility, and the team has not (<10= score 10 (10-25, score 4, 25-50 score 1,
been a coach training site. over 50 score 0)
There is no coach in the team, or their visits are over one month (no coach, score 5, visits over 1 month score 4, visits
frequency for every meeting, score 1; coach in team, score 0)
The officer in charge does not have training, is not in the team, (No officer in charge in team score 4, no active support
or where the officer in charge does not actively support the score 5)
team’s work
There are less than 4 people in the team (less than 4 score 5; 4 score 4; 5 score 3; 6 score 2; 7
score 1; and 8 score 0)
Over 50% of the team have left and there has not been (over 50% left score 5)
retraining, or coach visit-training.
There is less than 5 person days training input (less than 5 days score 5; 5-7 person days score 3, 10 or
(e.g., one person with 5 days or over training, over person days score 0)
or five people with one days training)
There is poor morale (e.g., connected with de-linking) or a (Poor morale and unprofessional culture as judged by
culture that does not value professionalism experienced clinician visitor, score 5; good morale and
professionalism, score 0)
3: Risk of not following the cycle-steps and using the methods correctly
Same as above, and add
The everyday workload of team members did not increase considerably for longer than two months. (If so score 5-10
according to severity).
4: Similar risk factors predict whether the team will continue for more than 2 cycles
Investigate :
a) Interaction of factors – where one makes up for or reinforces another.
b)Are conditions related to relatively higher quality anyway (e.g., competent officer in charge)
A29
QUESTIONS TO BE ASKED AND ISSUES TO BE ADDRESSED
BY THE SUPPORT SYSTEMS SUBTEAM
For Regional Directorate Location :
Staff interviewed :
Supervision (ask if Standards Team does not do)
1. What technical supervision is currently done? (informal and formal), For each type of supervision mentioned, who
does it, who is supervised, how is it done, how often; why is it done?
2. Is supervision budgeted for? Can you estimate how much it costs to provide a supervisory visit? What % of the
budget is for supervision?
3. When was last visit (DATE ________)? what type? duration of visit? outcome?
4. What tools are used to structure visits? ie. checklists (confirm)
5. Generally what happens as a result of the visit : do supervisors comment on problem solving, give feedback; take
action? How? Examples of how supervision contributed to team problem solving at HC
6. Is visit or outcome documented/reported (examples on quality of care)? to whom? how often?
7. Do you feel that supervisors have adequate supervisory skills/training? specify skills
8. How are supervisors chosen? supervision standards or guidelines; job descriptions (check)?
9. Do you feel that the supervision adequately addresses quality of care issues at the District? Why?
10. What are the main strengths of the supervision you provide? Weaknesses? Other areas for improvement and any
recommendations?
Training
11. Are you (region staff) given any QA training? what? when? from whom?
12. How have you applied the QA skills/topics learned in your own work?
13. Do you feel that these types of QA training are useful/ important/adequate? (specify)
14. Main problems/areas for improvement/needs and recommendations
Coaching/Facilitation of QA
15. Do you know about the QA activities in this region? get general impressions regarding work of link facilitators,
coaches, teams
16. Have you heard or read the quarterly link facilitator reports or other reports of QA activities at the health centers?
Any actions taken as result? (examples)
17. Do you provide any support to the HC, Links, coaches for QA activities (financial, administrative, transport, other)?
18. Problems/recommendations
A30
Mechanisms for reporting/documenting QA activities and monitoring quality
Staff interviewed:
19. Do you compile reports/documents about the QA activities? What is done with these reports? How often reported?
20. Is feedback ever given back to the District/HC about this work? Examples.
21. What other ways are QA activities/results reported?
22. New H/MIS piloted here? other H/MIS or monitoring system? SKIP TO END if none.
23. Do you use it to monitor performance/quality of care at the districts? How? Check indicators.
24. Do you use these indicators to identify areas for improvement/districts with poor quality of care? How? How
prioritise among problems identified?
25. Has frequency of collection/compilation been established for each indicator (as per H/MIS guidelines)? Who does?
Check tables.
26. Has an analysis procedure for each indicator been established (including thresholds/targets)? Who does?
27. Does the Region provide feedback to DHMT/ HCs on quality of care as a result of these indicators? How? When?
28. Main problems/areas for improvement/needs and recommendations.
For DHMT Location :
Staff interviewed :
Supervision
1. What technical supervision is currently done? (informal and formal), For each type of supervision mentioned, who
does it, who is supervised, how is it done, how often; why is it done?
2. Is supervision budgeted for? Can you estimate how much it costs to provide a supervisory visit? What % of the bud￾get is for supervision?
3. When was last visit (DATE ________)? what type? duration of visit? outcome?
4. what tools are used to structure visits? ie. checklists (confirm)
5. Generally what happens as a result of the visit : do supervisors comment on problem solving, give feedback; take
action? examples of how supervision contributed to team problem solving at HC
6. Is visit or outcome documented/reported (any examples on quality of care)? how often?
7. Do you feel that supervisors have adequate supervisory skills/training? specify skills
8. How are supervisors chosen? supervision standards or guidelines, job descriptions (check)?
9. Do you feel that the supervision given is adequate to address quality of care issues at HCs? Why?
10. What are the main strengths of the supervision you provide? Weaknesses? Other areas for improvement and any
recommendations to improve the supervision you provide?
11. What supervision do you receive from the Regional Directorate? Technical or managerial? How often?
12. When was the last Regional visit (DATE)? Purpose of the visit? What was done? Any feedback?
13. Do you generally find these visits helpful for addressing quality issues? Why or why not?
14. What needs improvement? Recommendations to help improve the way supervision is done?
A31
Training
Staff interviewed :
15. Who is responsible for training activities at the District? How do you identify the need for training/ how are people
selected to be trained? Is the same true for QA training?
16. Is there a plan for conducting these training activities for the district (check Plan of Action)
17. What QA training have you (District staff) received? when? from whom?
-sensitisation
-STDs setting
-problem solving
-indicator development
-coaching/facilitation
18. Have you been able to apply these QA skills/topics in your work? How?
19. How does the District ensure that QA skills are maintained at the District/HC levels after training? periodic monitor￾ing, re-training etc.
20. What are some areas for improvement/additional needs/ any recommendations
Coaching/Facilitation of QA
Staff interviewed :
21. What are your impressions regarding the QA activities (link facilitators, coaches, teams)
22. Are QA activities budgeted for at the District? Check action plan.
23. Have you compiled the quarterly link facilitator reports or other reports of QA activities at the health centres (check)?
What do you do with them? How are they used?
24. Do you provide any support to the HC for QA activities (financial, administrative, transport, other)? To the Links? To
the coaches?
25. Constraints/recommendations
Mechanisms for reporting/documenting QA activities and monitoring quality
Staff interviewed:
26. Do you compile reports/documents about the QA activities? What is done with these reports? How often reported?
27. Is feedback ever given back to the District/HC about this work? examples
28. What other ways are QA activities/results reported?
29. New H/MIS piloted here? other H/MIS or monitoring system? SKIP TO END if none
30. Do you use it to monitor performance/quality of care at the districts? how? check indicators
31. Do you use these indicators to identify areas for improvement/districts with poor quality of care? How? How priori￾tize among problems identified?
32. Has frequency of collection/compilation been established for each indicator (as per H/MIS guidelines)? who does?
check tables
A32
33. Has an analysis procedure for each indicator been established (including thresholds/targets)? who does?
34. Does the Region provide feedback to DHMT/ HCs on quality of care as a result of these indicators? How? When?
35. Main problems/areas for improvement/needs and recommendations
For Health Centre QA teams Location :
Staff interviewed :
Supervision
1. What technical supervision do you receive from the District? (informal and formal), For each type of supervision
mentioned, who does it, who is supervised, how is it done, how often; why is it done?
2. When was the last visit (DATE ________)? what type? duration of visit? outcome?
3. Were any tools used to structure visits? i.e., checklists (confirm)
4. Generally what happens as a result of the visit : do supervisors comment on problem solving, give feedback; take
action? Examples of how supervision contributed to team problem solving at HC
5. Is visit or outcome documented/reported (any examples on quality of care)? how often?
6. Do you feel that supervisors have adequate supervisory skills/training? specify skills
7. Do you feel that the supervision given is enough to support the HC? Why or why not?
8. Do you generally find these visits helpful? why or why not? What are the main strengths of the supervision you re￾ceive? Weaknesses? Other areas for improvement and any recommendations to improve the supervision you receive?
9. What supervision is given to the HC by the Regional Directorate? Technical or managerial?
10. When was the last Regional visit (DATE)? What was the purpose of the visit? What was done? feedback given?
11. Do you generally find these visits helpful for addressing quality issues? Why or why not?
12. What needs improvement? Recommendations to help improve the way supervision is given?
13. Is supervision carried out within the HC? Why is it done? How? By whom? How often?
- informal
- self-assessment
- peer review
Coaching/Facilitation of QA
Staff interviewed :
14. What are your impressions regarding the QA activities (link facilitators, coaches, teams)
15. Are these QA activities budgeted for at the HC? Check action plan.
16. How do you report QA activities at the HC? What, how often, who prepares? From whom do you receive feedback
regarding these reports? Any actions taken as result? (examples)
17. Does the HC provide any support for QA activities (financial, administrative, transport, other)? To the Links? To the
coaches?
A33
18. How many times in the last month were you visited by a coach? Link facilitator?
19. Any problems/recommendations?
Training (do this section if QIT team does not do)
20. What QA-related training have you received? what type? When? From whom?
–general awareness
–standards setting
-problem solving
-indicator development
-coaching/facilitation
21. Which QA skills/topics have you used in your own work/in the teams?
-STDs setting
-problem solving
-indicator development
22. What problems have you encountered in applying these skills? (i.e., data collection and analysis?)
23. Do you feel that these types of QA training are useful/ important/adequate? (specify)
24. How are new staff individually trained in QA? formal/informal?
25. Have you been given any refresher/in-house QA training since ___?
26. Are you satisfied with the training you received?
27. Main constraints/problems/areas for improvement/additional needs and recommendations
Mechanisms for reporting/documenting QA activities and monitoring quality
Staff interviewed:
28. Do you compile reports/documents about the QA activities? What is done with these reports? How often reported?
29. Is feedback ever given back to the District/Region/CBoH about this work? examples
30. What other ways are QA activities/results reported?
31. New H/MIS piloted here? other H/MIS or monitoring system? SKIP TO END if none
32. Do you use it to monitor performance/quality of care at the districts? how? check indicators
33. Do you use these indicators to identify areas for improvement/districts with poor quality of care? How? How
prioritise among problems identified?
34. Has frequency of collection/compilation been established for each indicator (as per H/MIS guidelines)? who does?
check tables
35. Has an analysis procedure for each indicator been established (including thresholds/targets)? who does?
36. Does the Region provide feedback to DHMT/ HCs on quality of care as a result of these indicators? How? When?
37. Main problems/areas for improvement/needs and recommendations
A34
For Links/Coaches
Name/title :
1. For which District/HC are you the Link/coach?
2. What training did you receive to be coach or link? when?
3. Was training adequate to conduct QA training/coaching activities?
4. What duties do you perform in addition to working as a Link/Coach?
5. General impressions regarding effectiveness or importance of QA activities?
Do you have a schedule for HC visits? trainings to do? ad hoc
6. How many visits to how many teams on average per month?
7. How do links supervise work of coaches? How ensure that skills of coaches are adequate/effective?
8. How do you maintain your own skill levels?
9. What support do you receive for doing this work? from Region, District, HC, donors?
10. Who supervises the work you do as a facilitator/coach?
11. What motivates you to continue with these activities?
12. Constraints/problems/recommendations especially for sustaining QA work?
A35
Appendix E:
Achievements of the Problem-Solving Teams
Link Facilitator Reports
Region: Southwest Number of Districts in Region:
Province: Southern and Western Number of links in Region: 7
Quarter: 3rd 97 Number of coaches: 31 (24 active)
Date: 22 October 97 Reporting: Mr. Chikuta, Mr. Masheta, Mr. Chindongo, Mr. Kachana,
Mrs. Nkatya, Mr. Litebele
Location Activity Problem-Solving or Other Quality Improvement
Gwembe District Hospital and 3 rural Standard setting – district compiling a book of standards;
health centres received incomplete information about exactly what standards this concerns.
QA sensitisation
Mazabuka District 13 DHO staff received Beginning problem solving – identified 11 possible problems.
Health Office (DHO) training in problem Short list done, priority matrix used to identify lack of teamwork as
solving problem; now doing operational definition of problem.
Magoye Rural Health Inadequate water supply: HC staff and patients’ relatives drawing
Center (RHC), water from community borehole – inconvenience, long time to draw
Mazabuka District water, predisposes staff to poor hygiene; formed team with
community to solicit donations, received a submersible pump,
hooked it to HC pipes, now center has piped water morning and
lunch hour.
Reduced teamwork among staff, no transparency, gossiping,
blaming. Treated as low-lying fruit – noted clinical officer was
controlling all aspects of the center (stores, finances, records, etc.).
Met as a team to divide responsibilities, sharing responsibilities,
delegation, now have regular staff meetings and participation by all
staff. Ill feelings reduced.
Mwachipapa RHC, Low immunization coverage – 39.8%. At last report, doing data
Choma District collection, now doing data analysis.
Sikalongo Health Unsafe home deliveries, 12/27 had complications; insufficient
Center, Choma information in report to know over what time data was collected,
District or what the complications are; in analysis phase.
Choma District Increase in malnutrition throughout district – 30% in 1995, 35% in
Health Office 1996. Just beginning data analysis to find out the patterns and
characteristics of the statistic.
6/10 health information forms missing within the office before they
reach the health information officer. Set standards, for forms
routing and handling, monitoring results.
Choma General New QA committee
Hospital formed, had been
inactive
A36
Location Activity Problem-Solving or Other Quality Improvement
Sinazeze RHC, Low immunization coverage – 18% baseline in July. Did bubble
Sinazongwe District chart of possible causes, testing theories of cause with data
collection going on now.
Maamba Rural After becoming aware of a need to focus on clients, discovered that
Hospital, Sinazongwe 15/20 patients in one day got lost going to different departments
District within the hospital. Standards were set for directional signs, posting,
signs put up in July. Now monitored: only 3/20 got lost, these
people could not read. Considering ways to provide way-finding for
people who do not read.
Makunka RHC, Problem Solving, After becoming aware of a need to focus on clients, realized there
Makunka District QA sensitisation was no privacy for antenatal history-taking, low-lying problem,
rearranged patient flow to provide private room.
Ngwezi RHC, Problem solving, Delayed wound healing, beginning analysis; incomplete information
Makunka District QA sensitization available about which wounds, which patients, definition of
“delayed.”
Libuyu HC, Monthly QA committee Had no monitoring of STD (Sexually Transmitted Disease) cases in
Livingstone District problem-solving meetings antenatal mothers and babies, started routine monitoring, noted high
incidence of syphilitic rash in this population, began problem
solving; defining the scope of the problem.
Mahatma Gandhi Monthly QA problem Congestion at MCH (Maternal and Child Health) department,
Clinic, Dambwa, solving meetings baseline waiting time from arrival to departure was 3 hours. Noted
Livingstone District longest waits for patients arriving earliest, problem due to staffing
patterns that put more staff on late morning and afternoon shifts;
rescheduled staff to morning hours, wait now maximum 68 minutes
Livingstone District Trained one coach in Inadequate coaching resources for the district, low-lying problem
Health Office problem solving solved with validation of new coach.
Namwala District Originally, no monitoring of post-natal coverage, began monitoring.
Hospital In July, reported only 13% delivered mothers attended post-natal
care. Acted on easy solutions: educating mothers at discharge,
during antenatal care, when getting BcG vaccine, set appointment
date when discharged from hospital. Recent monitors show 28% of
delivered women attend post-natal.
Namwala District DHO (District Health Previously reported late submission of monthly returns, none
Office), hospital and submitted on time. Set standards for submission with clinics.
RHC staff received July 8/10, August 7/10, September 8/10 submitted on time.
problem-solving training
Masele Clinic, Previously reported standards setting to correct problem of not
Namwala District receiving referral information on patients sent to hospital. Initially
got 5/5 and 6/7 referral letters back (May, June). From July –
September, only 2/15 referrals returned. Looking at the problem
again with hospital and clinic staff.
Itezhitezhi District QA coach trained New district without a QA coach or locally available link; attended
QA Coaching and Committee workshops; coach validated.
A37
Location Activity Problem-Solving or Other Quality Improvement
Kaoma District Weekly QA problem- Congestion at OPD. Flow analysis and data collection showed long
Hospital solving meetings wait was occurring while waiting for lab results. Further data
collection to determine reasons.
Mbanyutu RHC, HMIS training Noted inadequate problem-solving methods for problems
Nkeyema RHC, discovered through HMIS analysis, due to verticalization of
Mayokwa-Yukwa programs of HMIS and QA. Started integrated supportive
RHC, Kaoma supervision with problem-solving instruction during quarterly
District supervision.
Kalabo District Unmonitored UCI coverage for children under 1 year. Had
Health Office standards, began monitoring, note low coverage of 65% in 1996.
Kalabo District Congestion at OPD, flow analysis shows long waiting time of 2
Hospital hours from arrival to leaving, bottlenecks. Redesigned patient flow,
assigned public relations person to help patients with way-finding.
In first 2 months, average wait reduced to 1 hour.
Yuka Mission Problem-solving team No monitoring of drug consumption. Began tracking,
Hospital meetings noted essential drugs not available or run out early. While posing
possible causes, this mission hospital realized they did not get
regular drug supplies from CBoH, requested drugs. They continue
problem solving to arrive at an understanding of drug consumption
patterns and to track receipt of CBoH drugs.
All Kalabo District HMIS training Same problems as Kaoma District with integration of HMIS and QA
institutions problem solving; supportive supervision interventions also done
here.
Region: North Central Number of Districts in Region:
Province: Central and Northern Number of links in Region: 11
Quarter: 3rd 97 Number of coaches in Region: 30 trained (28 active) + 9 local coaches
trained in Kabwe
Date: 22 October 97 Reporting: Mr. Witola, Mr. Wapakula, Mr. Mate, Mr. Sikana,
Mr. Nyrienda, Mr. Mwanza, Mrs. Chisi, Mrs. Musonda, Mrs. Mulenga,
Mrs. Mwindula, Mrs. Ng’andu
Location Activity Problem-Solving or Other Quality Improvement
Nakoli Health Center, High incidence of diarrhea in children, suspected problem with
Kabwe District water sources. Collected data (water samples, source information)
and found water sources innocent in cases. Most diarrhea (7/10
cases) was related to malaria (positive slide, negative water).
Decided not to do massive chlorinating of wells – saved K120,000
per barrel. Re-examining care of malaria patients with diarrhea.
Kabwe District Training local coaches Two coaches were not enough for the district, could not reach the
HCs, links trained for 5 days in standards setting and problem
solving, working in pairs and starting regular coaches meetings in
district.
A38
Location Activity Problem-Solving or Other Quality Improvement
Mumbwa District QA sensitization and Concerned about waiting time in OPD, doing data collection to find
Hospital problem-solving training out what waiting time is. Staff aware of data collection and they are
already changing their habits by reducing absenteeism and working
more efficiently. Team concerned that problem will be resolved just
by monitoring – commended!
Nampubwe Health QA sensitization and Perceived long waiting time for mothers attending under-5 clinic,
Center, Mumbwa problem-solving training measured at 2 hours in small sample. Chose low lying solution,
District started supermarket services. Mothers have expressed that they
prefer the supermarket approach. Team to re-measure waiting
times, but realize that old model of service delivery does not match
new, and data may be questioned.
Kapyanga Health QA sensitization and Long waiting time at OPD, estimated at 2 hours. Collecting data
Center, Mumbwa problem-solving training during patient flow analysis.
District
Sichobo Health QA sensitization and Long waiting time in antenatal clinic, estimate women wait for two
Center, Mumbwa problem-solving training hours before being attended. Collecting data during patient flow
District analysis.
Serenje District Refresher QA training QA had stalled in the absence of the link. There was no monitoring
for DHO, hospitals and of disease management, no assessment of management support to
centers facilities, no assessment of the effectiveness of UCI program. Link
and teams did quality assessment, collecting data from the
community and facilities. Found: irrational prescribing with no cost
considerations; screening skills quite poor with some patients not
being screened, some not being greeted, some not having
temperatures or other vital signs measured; absence of essential
drugs; incomplete history taking; incomplete health records; using
expired vaccines; ineffective communication between providers and
clients; ill-defined referral; irregular or inappropriate staff training.
Are considering at the district level what the priorities are, and will
sponsor teams formed at facilities to address these issues.
Northern Province, QA Committee and QA In September, did province DySSSy and QA workshops; 44 people
all districts Coaching workshops attended DySSSy, 39 QA Committee, 25 coaching. All districts
held have done action plans for 1997 and are doing them for 1998. Each
district has plan for sensitizing DHMTs and rural HCs. Most
committees recently formed have members from the community
such as farmers, shop keepers, church headmen, as well as health
workers.
Nseluka RHC, Poor drug management – drugs finishing before receiving the next
Kasama District kit. Bubble chart being done, unsure if drugs finish because of
appropriate use, or insufficient supply, or irrational use.
Nkolemfumu RHC, Inadequate equipment for cleaning, doing bubble chart.
Kasama District
Chilubula RHC, No orientation for patients; patients told verbally to go to the wards
Kasama District but no knowledge of why. Doing bubble chart, although may be a
low-lying problem.
Poor drug management – many expired drugs in pharmacy, doing
data collection to discover what is expired, for how long.
A39
Location Activity Problem-Solving or Other Quality Improvement
Outpatient Lack of cleaning materials, bubble chart done. OPD recently
Department, detached from the hospital to the district, after coming under the
Kasama district control, adequate supplies and materials were provided.
Mungwi RHC, Congestion in treatment room. Doing bubble chart, anticipate need
Kasama District for patient flow analysis.
Lukupa RHC, Poor drug management – drugs finish before receiving next
Kasama District consignment. Doing bubble chart and data collection to determine if
this is a problem or inappropriate use or appropriate for service
volume.
Milima RHC, Long waiting time; estimated that patients wait 3 hours from entry to
Kasama District discharge; bubble chart.
Tazara RHC, Low family planning acceptance, 159 of 3206 users; unsure what
Kasama District time frame this baseline refers to; doing bubble chart.
Location RHC, Started problem solving, but information not available about topic.
Kasama District
Musa RHC, Long waiting times, estimate 40 minute wait to be seen, doing
Kasama District bubble chart.
Isoka District Standards setting; sensitization Nurses are setting clinical standards
Hospital for their services.
OPD, Isoka Problem solving sensitization Long waiting time at OPD,
District Hospital formulating data questions.
Luwingu District Formed two QA After low-lying problems defined, standards were set or
Hospital working groups; re-emphasized and immediate solutions were applied:
doing problem ID ■ sharps disposal not appropriate - heps bags that were lying in
corridors have been removed
■ incinerator over full – being emptied frequently
■ no rubbish pits – community became aware through QA
committees and churches dug the pits
■ poor patient meals, with mostly cabbage and beans – a nutritionist
was assigned, meals have improved
■ visitors coming at all hours – gate fixed, visitors controlled
■ lack of nursing staff – number of nurses going to the market while
on duty has reduced, adequate staff for care requirements
■ no laboratory screenings for MCH patients – some reagents were
lacking, but MCH have started routine screening after learning
about requirements
■ inadequate latrines for hospital – churches assisting with digging
pits
■ no input from community – community members on QA
committee challenged facility to get client input, suggestion boxes
put up at hospital
■ lack of security at hospital – constructing a wall fence and shelter
for guards
Long waiting time at children’s clinic, approximately two hours
from arrival to departure. Flow chart and bubble done, data
questions posed, data collection done on 15/10/97, conclusions not
yet available.
A40
Location Activity Problem-Solving or Other Quality Improvement
Luwingu District Poor accommodation for health workers, QA committee influenced
Health Office the district health board to start working on the problem; application
made to procure a vacant building, applying for funds under
micro-project unit.
Measles cases managed in the TB ward - construction begun on an
extension ward.
Chilubi District, Poor management of essential drugs – drugs run out after 3 weeks
all 8 RHCs rather than 8 weeks as the standard. Doing flow and bubble charts,
considering if standard is appropriate or not.
Chinsali District Hospital Starting problem solving looking at infections in post-operative
c-section patients; just beginning.
Region: North Western Number of Districts in Region:
Province: Luapula, Copperbelt, North Western Number of links in Region: 13
Quarter: 3rd 97 Number of coaches: 48 (45 active); none yet trained in North
Western Province
Date: 22 October 97 Reporting: Miss Buleze, Mr. Jere, Mr. Chibesa, Mr. Longwani,
Mr. Kilele (no report from North Western Province)
Location Activity Problem-Solving or Other Quality Improvement
Buchi Urban Health Dirty clinic, cleaning supplies are exhausted in two weeks that
Center (UHC), should last a month. Set standards – a book to record cleaning
Kitwe District materials received, an audit of supplies daily, stock control cards, a
cleaning committee formed, a working schedule for cleaning (when
to clean floors, windows, and so on) to avoid misuse. Supplies now
last the whole month, three different supervisors at three different
visits have commended clinic for cleanliness.
Chimwemwe UHC, Long waiting times, data shows most time spent in registry and
Kitwe District screening, longest delays are at the beginning of the work day.
Actions: night shift to pull cards for patients who arrive early, night
staff collect registration fees for early patients, clerks can make
cards for new patients as well as pull cards for old patients at the
same time, patients are given numbers to track who comes first
(rather than getting pushed lower in the stack as more recent arrivals
get earlier treatment), shifts changed for clinical officers so there is
staff on for 24 hours, more staff early in the morning; meeting with
clinical officers discussed ways not to waste time; nurses trained to
help screen. Registry waits reduced from average 36 minutes to 11
minutes; screening time reduced from average of 54 minutes to 29
minutes; pharmacy wait averages 18 minutes. Total reduced waiting
time is from 1 hour 48 minutes to 58 minutes.
Complaints from antenatal care patients they wait too long before
going home. Collecting data to see which part of the process is slow.
A41
Location Activity Problem-Solving or Other Quality Improvement
Itimpi Maternity Underutilization of services – less deliveries compared to booked
Center, Kitwe District clients. Though there does not seem to be a clear problem (nothing
wrong with at-home deliveries if they are safe), staff believes they
need to increase the number of deliveries in the center. Treating as a
low-lying problem, with intensified health education on benefits of
delivering at center, risks that cannot be handled by TBAs; also
educating mothers to begin travel early to avoid transport problems.
Kwacha UHC, Dirty clinic. Procured slashers, burned refuse, procured cleaning
Kitwe District material and a book for receipt and usage monitoring. Clinic and
surroundings now clean.
Kwacha UHC, Drugs run out after three weeks, data collection going on.
Long waiting time in OPD, data collection going on.
Luangwa Peri-urban Long waiting times in the queues in the clinic, did flow and bubble
Health Center, charts, data showed people waited in queues approximately 1 hour.
Kitwe District Actions: ensured each room had a staff member assigned, set
standards for staff punctuality, procured enough equipment
(BP cuffs, scales), scheduled client visits for different days
(primagravidas and multigravs on different days), staff trained in
family planning so there is always a FP person in the clinic. After
interventions, queue waits reduced to an average of 40 minutes.
Ndeke Clinic, Long waiting time in OPD, doing data collection.
Kitwe District
Ipusukilo Peri-urban Long waiting time in MCH, doing data collection.
Clinic, Kitwe District
City Square Clinic, Careless sharps disposal. Got cartons to use for disposal, bought
Kitwe District empty drums for use as dust bins, educated staff on risks of sharps
injury and prevention methods including proper disposal, drew up a
schedule with council refuse trucks to have more frequent refuse
collection (for burning). Sharps are now disposed of properly.
Kamfinsa Prison Low clinic maternal deliveries, need training for staff on problem
Clinic, Kitwe District solving to continue work.
Garneton Clinic, High number of defaulters in child immunizations. Sample from
Kitwe District April – September 1996 had 88% not fully vaccinated. Set
standards for recording vaccinations, vaccination to be available
each day, children to receive all necessary vaccinations at once (not
return on successive days), in-charge to check records daily,
vaccinations to begin within 10 minutes of clinic opening each day,
cards to be checked in under-5 clinic and immunizations to be given
there as needed, do standard health education on the benefits of
vaccination. Plan to monitor during a 6-month period.
Munkanta RHC, Problem-solving training Standards previously set (UNICEF) were not addressing important
Kawambwa District issues, were not objective. Staff are redefining standards.
Kawmbwa District Problem-solving training, Problem selection – poor environmental hygiene in the hospital.
Hospital QA committee formed
Kitwe District
A42
Location Activity Problem-Solving or Other Quality Improvement
Mwense Hospital Problem-solving training, Inadequate supply of supplementary ARI drugs. Reported at PDCA
QA committee formed (Plan, Do, Check, Act) cycle, but no details of problem solving
work available to this report.
Mulumbi RHC, QA sensitization, Low utilization of FP services. 14/372 baseline users. Acceptors in
committee formed June were 32. Unsure of specific actions, the length of time over
which the baseline was calculated, if “32” is total or just June
acceptors.
Fwaka East 8 RHC, QA sensitization Center was formerly in Samfya District and run by the Flying
Milenge District Doctor’s Service – no exposure to QA. Beginning problem solving,
working on low UCI coverage.
Kabongo RHC, Attempt to visit this center failed as coach did not get promised
Milenge District transport (is 135 km from coach, formerly in Samfya district).
Samfya District Problem-solving training Beginning problem solving, “high number of unsafe deliveries at
Hospital home.” No information available on why home deliveries are
thought to be unsafe.
Buntungwa Urban Congestion at OPD, choosing a team
Clinic, Mansa District.
Fimpulu RHC, Increase in complicated malaria cases under 5; up to bubble chart.
Mansa District
Mibenge RHC, Lack of co-ordination within staff; treated as low-lying problem,
Mansa District drew up time table for staff meetings, minutes are written in Bemba
for the sake of the CDE’s
Kabunda RHC, Late reporting for work. Set standard for reporting, and for staff to
Mansa District seek permission from the in-charge if they have other problems to
attend to requiring them to be late.
Paul Mambilima RHC, Congestion on wards caused by relatives staying with patients
Mansa District because there is no shelter for relatives. User fees used to buy
bricks, community has started bringing bamboo for the roof, shelter
to be constructed before the rains.
Muwang’uni RHC, Underutilization of family planning services. No detail on analysis,
Mansa District but have integrated FP with other services and health education is
targeting men. Some men have started collecting contraceptives on
behalf of their wives (links pointed out the risks of using FP
methods without assessment and evaluation, coach to investigate).
Ndoba RHC, Low immunization coverage, starting problem solving, doing bubble
Mansa District chart.
Mansa District Overall low family planning acceptance rate. Supermarket approach
introduced at 4 urban centers, mobile services started. Target for
1996 new acceptors was 46%, now new acceptors are 68%.
Kanyembo RHC, QA sensitization, Identified 16 problems, chose to work on low number of deliveries
Nchelenge District problem solving at the health center, just defining problem.
A43
Location Activity Problem-Solving or Other Quality Improvement
Kambwali RHC, QA sensitization, Center staff had poor interpretation of QA – viewed as examination
Nchelenge District problem solving and scoring points. When district left, staff went to same old
performance. Now realize importance of setting own standards and
local control of quality. Just started problem solving on “long
waiting time at the OPD.” Also bought an Oral Rehydration
Therapy corner kit after realizing it was important to quality, not just
an inspection criteria.
Nchelenge DHO Standards training District has 60 QA standards, which are not yielding desired results
(standards styled on Luapula UNICEF manner). Decided to review
standards, revise them as needed, and use them in RHC monitoring.
Also, working on the problem of low coverage of planned activities
in district action plans; flow chart, data questions done.
Nchelenge RHC, QA sensitization, Beginning to do problem solving, brainstormed 16 problems,
Nchelenge District problem solving prioritized to “high number of home deliveries which are unsafe,”
just doing problem definition.
Kabuta RHC, QA sensitization, Beginning problem solving, brainstormed 13 problems, prioritized
Nchelenge District problem solving to “late reporting by expectant mothers at the RHC for labor,” just
doing problem definition.
Chabilikila RHC, QA sensitization, Beginning problem solving, brainstormed 13 problems, prioritized
Nchelenge District problem solving to “inadequate number of primary healthcare workers, CHW and
TBAs in the health post,” just doing problem definition, link alerted
that this seemed to imply the solution, not the problem.
Chipungu RHC, Coaching Beginning problem solving, prioritized to “over crowding in the
Chienge District wards by inpatients” – just beginning problem definition.
Chienge DHO Report was sent but not received – known to be working on low
family planning utilization in the district, at implementation.
Ndola Urban DHO Few coaches for 20 Clinics submit late monthly returns. 9/20 submitted on time. At
clinics – request CBoH in-charge meeting, set standard for submission before 5th of
train one more following month, clinics allowed to use imprest for duplicating the
forms. First after setting standard had 14/20 submitting on time,
continuing to monitor.
Dola Hill Clinic, Problem-solving and Started to investigate compliance with chloroquine administration
standards-setting but unable to gather data because patient records missing.
training Therefore, began to work on poor record keeping in registry –
patient records often went missing or were misplaced. For one
week, 14 missing, 17 misplaced records (no denominator available).
Set standards for filing, registry entry; noted many went missing
from dispensary, so introduced card boxes in dispensary. Now
patient records are entered in respective registers every day, missing
records down to 8 per week, misplaced down to 6. Will monitor,
and start over on chloroquine problem.
Twapia Clinic, Ndola Problem-solving training Long waiting time at children’s clinic, mothers wait for an average
Urban District estimated to be 2 hours before being attended to. Did flow chart,
now collecting data on current waiting time.
A44
Location Activity Problem-Solving or Other Quality Improvement
Kabushi Clinic, Ndola Problem-solving training Low family planning acceptance. In July, 190/6000 (estimated)
Urban District women of child bearing age in catchment area were family planning
acceptors. Treated as low-lying problem, started FP services in
supermarket approach, more attention to health education on FP
during post-natal and under-5 visits. Increased to 209/6000 by
August 30.
Lubuto Clinic, Ndola Problem-solving training Patient complaints in OPD that they wait too long before being
Urban District attended to. Started problem solving, patient flow analysis, doing
data collection of times at each stage in the process.
Region: South East Number of Districts in Province:
Province: Eastern, Lusaka Number of links in Province: 7
Quarter: 3rd 97 Number of coaches in Province: 37 trained (32 active)
Date: 22 October 97 Reporting: Mr. Chitomombo, Mr. Banda, Mrs. Mupwaya
Location Activity Problem-Solving or Other Quality Improvement
Luangwa District Distributed storybooks No indicators on finance being monitored, set indicators for #
to RHCs and hospital payments made after 15.00; # payments made before approval;
# imprests given before retiring old imprest; monitoring.
Writing and distributing No indicators on maintenance being monitored, note erratic
link minutes to coaches servicing of vehicles, set indicator # times a vehicle undergoes
service in a quarter; monitoring.
Team building for one
day with RHC staff
Antenatal care coverage 26%. Interventions: all RHC and hospital
MCH will conduct follow-ups once a month to pregnant women; in
3rd quarter, coverage is 73%.
Luangwa Boma Clinic Essential drugs run out after two weeks, 50% of patients got drugs
Luangwa District without prescription, 15% got multiple prescriptions. Intervention:
recording of each step in drug use and dispensing, kits now last 4
weeks or more.
Put vision and storyboard Patient long waiting time – impression is that patients average a
in view 40-minute wait from pulling their card to departure, but before
further analysis, will gather baseline data for two weeks to find out
real waiting time and flow analysis.
Estates Clinic, Had solved problem of long waiting time, now experiencing
Kafue District welcome side effects of the actions taken to shorten waiting time.
Had established 7 immunization outreach posts to reduce
congestion, but now also noting an increase in immunization
coverage: in 1996 BCG 50%, measles 50%, DPT 65%, Polio 60%
TT 50% (# immunized / target population). Jan – Aug 97 already
have BCG 85%, measles 75%, DPT 80%, Polio 80%, TT 60%
(against annual targets). Also, had started orienting l nurse to MCH
A45
Location Activity Problem-Solving or Other Quality Improvement
duties, to relieve congestion in that area. Now notice that there is a
side effect of no more shortage of nurses to work in MCH – there is
always someone on duty who is fully oriented. In addition, nurses
who have been trained in IMCI are assisting with screening, further
shortening the wait for screening. Also, note some delays had been
due to waiting for referral to UTH, requested a physician be
assigned. After physician assigned, fewer patients are waiting as
examinations can be done without referral to UTH, and referrals are
done more quickly because the physician has already done a
preliminary examination and studies. There is no congestion in the
clinic.
Neglected wards. All nurses had been working in OPD, only
distributing medications in ward. Realized this was a low-lying
problem, assigned nurse each day to meet medical needs of patients,
make beds, ensure meals are provided. Also assigned cleaning
standards for ward and note there are no more offensive odors
coming from the ward.
Impression that there was an increase in septic wounds which turned
septic after treatment. Noted that only clean, not sterile, items were
used for wound care. Requested an autoclave, set standards for use
of sterile materials in wound care. No formal measure of infection
rates, but are 100% meeting standard that sterile materials are used
for wound care.
Mission Hospital QA sensitization Long waiting time. At problem definition, discovered this to be a
RHC, Kafue District low-lying problem, delays were due to health workers reporting to
work late, there were no bottlenecks in care. Set standard to report
to work on time, in-charge monitoring.
Railway Clinic, Problem-solving training Low post-natal coverage; no data available for baseline level, doing
Kafue District problem definition.
DHO, Kafue District Late monthly returns, monitoring solutions put in place reported last
quarter. 100% compliance with submitting returns on time.
Baseline 21% compliance (3/14 centers)
Mr. Makulu Clinic, all members of staff, not Late antenatal booking previously reported. Little or no
Kafue District just QA committee, improvement, but staff continue to follow standard of screening
sensitized in problem all women for LMP, and booking for antenatal evaluation, no matter
solving what clinic they attend. They will measure the percent of bookings
during early trimesters later in the year.
Long waiting time previously reported reduced from 1 hr 45 minute
to 30 minute average over 10 patients. Monitored 10 more patients,
wait ranges from 30 – 45 minutes.
High number of TB defaulters – Jan – June 22/50 TB patients
defaulted. Just beginning problem solving with problem definition.
Chama District Long waiting times in outpatient care; data questions posed, data
Hospital, DHO collected, being analyzed.
A46
Location Activity Problem-Solving or Other Quality Improvement
Kambombo RHC, QA sensitization and Problem identification – late reporting to work, one hour late
Chama District problem solving common. Standard set by going through individual job descriptions
(different reporting times for different staff) and monitoring by
in-charge. No results available, but in-charge says people are on
time.
Tembwe RHC, QA sensitization and Problems identified and short listed, prioritized to work on low
Chama District problem solving immunization coverage – 30% of target population under 1 year are
immunized. Bubble chart is next step.
Sitwe RHC, QA sensitization and Problems identified and short listed, prioritized to work on shortage
Chama District problem solving of drugs, bubble chart next step.
Madzimoyo RHC, Increase in diarrhea cases, doing bubble chart.
Chipata District
Madzimawe RHC, Low immunization coverage, coverage less than 60%, target is 80%;
Chipata District analyzing high level flow chart.
Kasenengwa RHC, Late reporting for work; standard setting did not change habits,
Chipata District decided to do formal problem solving, preparing data questions after
completing bubble chart.
Msekera RHC, Inadequate screening of patients, doing data questions.
Chipata District
A47
Appendix F:
People Met during the Mission
People met in Lusaka
Central Board of Health
Dr. E. Limbambala Director, Directorate Monitoring and Evaluation
Dr. M. Maboshe Manager, Service Quality and Performance Audit, Directorate Monitoring and Evaluation
Mrs. Joyce Tembo Quality Assurance Specialist, Directorate Monitoring and Evaluation
Miss M. Moonga Clinical Advisor to the Director General
Mr. B. Chita Manager, Directorate Health Services Commissioning
Miss Anne Young HMIS Specialist
Ministry of Health
Mrs. Siame Immunization Specialist
Mrs. Sinyangwe Child Health Specialist
BASICS/Lusaka
Dr. Abdikal Alisalad Child Health Coordinator
Dr. Remi Sogunro Chief of Party
USAID/Zambia
Dr. Robert Clay PHN Chief Officer
Dr. Paul Zeitz Technical Advisor
Center for Health, Science and Social Research (CHESSORE)
Dr. T.J. Ngulube Researcher,
Dr. Mubiana Macwan’gi Researcher, Sociologist
Faculty of Private Practitioners (FPP)
Dr. Hilda Mutayabarwa Member, former Chairperson
Zambia Medical Association (ZMA)
Dr. Hilda Mutayabarwa General secretary
General Nursing Council (GNC)
Mrs. E. Msidi Registrar
Mrs. Dorcas Phiri Training Coordinator
Mrs. Theresa Sikateyo Trainer, In-service Training
Mrs. Rhoda Nthani Examinations Officer
A48
School of Nursing, University Teaching Hospital
Mrs. Margaret C. Maimbolwa Principal Tutor
Mrs. Mulima D. Ketata Nurse Tutor
Mrs. Stella C. Chisuka Senior Tutor
Mrs. Salome M.S. Zulu Nurse Tutor
School of Medicine, University of Zambia
Prof. L. Mukonge Dean, School of Medicine
Prof. A. Haworth Professor of Psychiatry
Prof. Kareshani Head of Department of Anatomy
Dr. Rosana Head of Physiological Sciences
Dr. Shinondo Head of Department of Pathology
Dr. L. Chiwele Head of Department of Community Medicine
Mrs. Kapembwe Senior Assistant and Registrar
Churches Medical Association of Zambia (CMAZ)
Mr. Marlon Banda Quality Coordinator
Dr. Simon Mphuha Health Programmes Manager
Dr. Biemba Secretary General
Medical Council of Zambia (MCZ)
Mr. W.W. Banda Registrar
World Health Organisation (WHO)
Dr. Buoaye WHO-Representative to Zambia
People Met in North Western Region
Regional Office, Kitwe
Mr. Lombe Chipupu Human Resource Specialist
Mr. Patrick Mubiana Health Promotion Specialist
Mr. Chipupu Kandeke Pharmacy Specialist
District Health Office, Kitwe
Dr. Chashi Cleto District Director of Health
Mrs. Mary Sieta Manager Planning and Development, DHMT
Mr. Bruno Chilundy District Coach
Chimemwe Health Center, Kitwe District
Mr. K.Mainda Clinical Officer
Mr. O. Chileshe Enrolled Nurse
A49
Ndeke Health Center, Kitwe District
Mr. Mulemfwe Chipiri Clinical Officer
Mr. Anthony Longwani Clinical Officer, Regional Link, Kitwe District, DHMT
Lukoshi Health Center, Kitwe District
Mrs. Christine Chanda Enrolled Nurse
Mindolo 1 Clinic, Kitwe District
Mrs. Annette Chilimana Acting Nursing Officer
Ms. Chisimba Simfuke Chief Nursing Officer, Nkana Mine Hospital
R.B. Maguswi Senior Nursing Officer – Clinics, Nkana Mine Hospital
D. Bolla Acting Hospital Administrator, Nkana Mine Hospital
Lufanyama District Health Management Team
Mrs. Grace Phiri District Director of Health
Mr. John Lungu District Information Officer, coach
Mrs. Colly Sovi Manager Planning and Development
Miss Josephine Banda Manager Administration
Mr. John Makumani Clinical Care Officer
Chimokunami Health Center, Lufanyama District
Mr. Philip Monoluba Sister in-charge
Mrs. Ruth Chibale Enrolled Nurse
Chati Health Center, Lufanyanma District
Mr. John Makumani Clinical Care Officer
Mrs. Daisy Chibale Family Health Nurse
Kalulushi District Health Office
Dr. Kawesha District Director for Health
Mr. Andrew Sekanika Manager Planning and Development
Kalulushi Government Clinic, Kalulushi District
Mr. Ndemena Laboratory Technician
Miss Idah Kasimbo Sister in-charge
St. Theresa Hospital, Ibenga
Dr. Joop Hanssen Medical Officer in-charge
Sister Christine Tembo Nurse
Dr. Soko Executive Director, Mpongwe Mission Hospital
Inonge Makuyu Kapika Theatre Nurse
A50
People Met in North Central Region
Regional Office, Kabwe
Dr. G.L. Kassanda Regional Director
Dr. A. Simwansa Manager Technical, Support and Performance Audit
Mrs. M. MusondaManager Human Resources and Administrative Systems
District Health Management Team, Kabwe District
Dr. E.J. Sikasula District Director of Health
Mr. H-C Witola Acting Manager Planning and Development, coach
Mr. Henry Fumbeshi Manager Administration
Mrs. Annie Matakala Coordinator MCH
Kabwe District Health Board
Mrs. Gertrude Kacha School Teacher
Mr. Jackson Mukuka Businessman
Mahatma Ghandi Clinic, Kabwe Urban District
Mr. Musukoma Clinical Officer in-charge
Mrs. A. Shaumkali Sister in-charge
Mrs. Sampa Enrolled Midwife
Mrs. Kassanda Enrolled Midwife
Mr. Mwale Laboratory Technician
Mahatma Ghandi Neighborhood Health Committee, Kabwe District
Mr. Henry Diangamo Member
Mr. Maxwell Kaira Member
Mr. Rodrik Kalimbwe Member
Mr. Joseph Kwando Member
Mr. Boniface Matafwali Member
Mr. Agrey M’Kwasa Member
Mr. Joseph Mulowa Member
Mr. Loyd Mwale Member
Nakoli Health Center, Kabwe District
Mrs. Josephine Maala RN, Sister in-charge
Mrs. Bessie Nyirenda Enrolled Midwife
Kapiri Mposhi District Health Management Team
Dr. Victoria Daka District Director of Health
Mrs. M.M.K. Njomwa Acting Manager Administration (DHI)
Mr. M.M. Mukolo Acting Manager Planning and Development
Mr. A. Mulenga Information Officer, QA coach
A51
Kapiri Mposhi District Hospital (1st level) - Quality Assurance Committee
Mrs. C.M.C. Kaziejka RN, Acting sister in-charge
Mrs. Beatrice Kunda RN
Mrs. Silvia S. Maswa Enrolled Nurse
Mrs. Lubinda EHT
Mrs. J. Chibuye EHT
Miss E. Mwale EHT
Mrs. J. Lumwenda EHT
Makululu Health Center, Kabwe District
Mrs. Joyce Happy Mwepa Sister in-charge
Mr. Alick Kalambata Chairman, Neighbourhood Health Committee
Mr. Joseph Ngulube Treasurer, Neighbourhood Health Committee
Mwashisompola Health Demonstration Zone, Chibombo DHO
Dr. Elijah Sinyinta District Director of Health
Mr. Ernest Mutukwa Senior Clinical Officer
Chibombo Health Center, Chibombo District
Mr. Davison Mpukuta Senior Environmental Technician
Bwacha Health Center, Kabwe District
Mrs. Grace Sikazwe RN
Liteta Hospital, Chibombo District
Mr. Hawela Moonga Laboratory Technician
People met in South Eastern Region
District Health Management Team, Lusaka
Dr. Moses Sinkala Acting District Director of Health, Manager Planning and Development
Mrs. Sipatonyana Public Health Nurse, Regional Link Facilitator
Mrs. Mavis Kalumba District Coach (Senior Health education Officer, Council Nutritionist)
George Health Center, Lusaka District
Mr. S. Zulu Clinical Officer
Mrs. Anne Kafuta Sister in-charge
Civic Center Clinic, Lusaka District
Mrs. Josephine Nondo Clinical Officer in-charge
Kamwala Health Center, Lusaka District
Mrs. Margaret Chitenge Enrolled Psychiatric Nurse
Mrs. Lister Mwanza Enrolled Nurse
A52
Kalingalinga Health Center, Lusaka District
Mrs. Sarah Ngoma RN Sister in-charge
Mrs. Mavis Kalumba Senior Health Education Officer/QA Coach
Mrs. Regina Moyo RN
Chawama Health Center, Lusaka District
Mrs. Maclean Ukwimi Sister in-charge
Mr. F.K. Kabaso Nutritionist
Dolly Kazoka Staff member
People met in South West Region
District Health Management Team, Monze District
Mr. Daniel Mukupa Manager Planning and Development (on leave)
Mrs. H. Chama Manager Administration
Mrs. J.Z. Munangandu District Health Information Officer
Mrs. Anne Mutinda Acting Manager Planning and Development
Monze District Hospital
Mrs. M. Simatwa Enrolled Midwife
Mr. Harold Nkhoma Clinical Officer, District Coach
Rusangu Health Center, Monze District
Mr. Mate Samwa Clinical Officer in-charge
Mr. James Natesamwa Clinical Officer
Mr. Colly Mwiindwa Enrolled Nurse
Manungu Health Center, Monze District
Mrs. Petronella Fulilwa Zulu Family Health Nurse
Mrs. M.H. Cheenyu Enrolled Midwife
Mrs. Rodia M. Chipuka Enrolled Midwife
Mrs. H.C. Heenyu Enrolled Midwife
Choma Health Office, Choma District
Mr. Grifin Chindongo Health Information Officer and Link Facilitator
A53
Appendix G:
Systems View of a Quality Assurance Program
Structure
Human resources
■ National QA specialists
■ Health staff competent in
QA at various levels of the
health system
■ Private and public health
practitioners
■ Other staff (decision makers,
accountants, etc..) whose
work influences the QA
activities
Bodies and constituencies
■ QA unit
■ QA committees at various
levels
■ Regulatory bodies
(professional councils or
associations, etc.)
■ Consumers/Patients
associations
■ Health committees
■ Public and private sector
associations
Health facilities
■ Hospitals
■ Ambulatory care facilities
■ Private and public facilities
Equipment
■ Identification of the minimal
medical and non medical
equipment necessary to
implement the standards of
care
Process
Development of a QA program
■ Definition of quality of care
■ QA policy: vision, mission, plan of activities
■ Provision of adequate resources for a
nation-wide coverage
■ Updated knowledge of the current
organisation of the health system
(health sector reform, health sector policy)
■ Knowledge of the quality of care situation
through a baseline survey
■ Mechanisms to review and adapt the QA
policy regularly
■ Monitoring and evaluation of the QA
programme
Training in QA
■ QA awareness, Quality design, Standards
setting, Performance measurement, Problem
solving, Quality management principles,
QA tools and techniques, etc.
■ Pre-service training
■ In-service training
■ Continuing education
Quality design activities
■ Assessment and utilization of community
needs and demands to design new health
services or re-design existing ones
■ Patient’s and population’s satisfaction surveys
Developing Standards
■ Setting and adapting clinical and
managerial standards of care
■ Development of clinical guidelines and stan￾dard operating procedures
■ Review/adaptation of standards
Outcome
Improved quality of care
■ Technical performance of
the health workers
■ Better health facility
management
■ Increased access to care
■ Increased efficiency of care
■ Increased safety of care
■ Improved provider/patient
interpersonal relationship
■ Increased effectiveness of
care
Improved population
satisfaction with health
services
■ Increased utilization of
health services
■ Increased community
involvement in the
organisation of health
services
■ Better appreciation of the
health services
Improved health provider
satisfaction with working
environment
■ Increased motivation
■ Increased productivity
■ Increased morale
A54
Structure
Financial resources
■ Budget specific to QA
activities
Support systems
■ Coaching
■ Supervision
■ QA training
■ QA activities monitoring
system
Process
Communication of standards
■ Training of the health providers
■ Information of the health providers
■ Development of job-aids
■ Distance education programs
■ Any other way to effectively communicate
standards
Establishment of incentive
methods
■ Performance-based reward mechanisms for
health providers and health facilities
Quality assessment activities
■ Definition of indicators to measure
compliance with standards
■ Mechanisms for collection of performance
indicators on a regular basis (monitoring
system, special surveys, HMIS, supervision,
self-assessment, etc.)
■ Methods used for collecting data on health
provider and health facility performance
■ Evaluation of technical competency
■ Use of performance indicators by the health
managers and health providers to re-design
the services
Quality improvement activities
■ Use of a problem solving/process
improvement methodology
■ Application of quality management principles
Benchmarking activities
■ Identification of best practices and best
performers
■ Investigation of factors explaining higher
performance
■ Dissemination of the best practices to other
health facilities
■ Adaptation of successful approaches by
another program or facility
Outcome
Establishment of a QA
culture
■ QA “mentality” of the
providers
■ QA management at all
levels
■ Mechanisms for
expression of satisfaction/
dissatisfaction of clients
with the health system
(demand for quality)
■ QA commitment expressed
at highest levels
QA capacity built
■ In-country QA expertise
sufficient to carry-out the
QA activities of the program
■ Understanding of QA by
decision-makers
Institutionalised QA
activities
■ QA activities carried out
on a routine basis at
appropriate levels
■ QA activities sustainable
(expertise, resource and
commitment are sufficient
to apply, adapt, and further
develop the QA methods)
■ Coverage of the QA
program (geographical
area, staff category, clinical
services, etc.)
A55
Process
Research activities
■ Undertaking of operations research on quality
of care and QA topics
■ Use of the research results to anticipate or
solve a quality problem
Regulation activities
■ Accreditation of health facilities
■ Licensing of health personnel
■ Certification of health personnel
Documentation activities of the effectiveness
of local QA efforts
■ Follow-up mechanism in place for the QAP
■ Record system in place
■ Storybooks and story boards of success sto￾ries and problematic interventions
■ Case-study material
Dissemination activities
■ Dissemination of research results to targeted
audience
■ Dissemination of quality of care assessment
results to targeted audience
■ Dissemination of the status of the QA
programme to targeted audience
Community application of QA
■ Links between health system and population
developed around quality of care issues
■ Community involvement in quality design,
monitoring and improvement activities