QUALITY 
ASSURANCE 
PROJECT
Center for Human Services • 7200 Wisconsin Avenue, Suite 600 • Bethesda, MD 20814-4811 • USA
TECHNICAL REPORT SUMMARY
The Chile Quality Assurance Program:
Final Evaluation
The Quality Assurance Project (QAP) is funded by the U.S. Agency for International Development
(USAID), under Contract Number HRN-C-00-96-90013. QAP serves countries eligible for USAID
assistance, USAID Missions and Bureaus, and other agencies and nongovernmental organizations
that cooperate with USAID. The QAP team consists of prime contractor Center for Human Services
(CHS), Joint Commission Resources, Inc. (JCI), Johns Hopkins University School of Hygiene and
Public Health (JHSPH), Johns Hopkins Center for Communication Programs (JHU/CCP), and the
Johns Hopkins Program for International Education in Reproductive Health (JHPIEGO). It provides
comprehensive, leading-edge technical expertise in the design, management, and implementation
of quality assurance programs in developing countries. CHS, the nonprofit affiliate of University
Research Co., LLC (URC), provides technical assistance and research for the design, management,
improvement, and monitoring of healthcare systems and service delivery in over 30 countries.
T E C H N I C A L R E P O R T S U M M A R Y
The Chile Quality Assurance Program:
Final Evaluation Abstract
This Technical Report Summary
is an abridged version of a
longer publication from QAP,
“Evaluation of the Chile National
QA Program.” The summary
provides an overview of the
Quality Assurance (QA) Program
in the Chilean healthcare system.
The program began in the early
1990s, and, working through a
QA Unit in the Ministry of Health,
developed training modules,
standards, and communication
strategies to support the QA
effort.
The summary details the
process whereby the QA effort
became self-sufficient, the
involvement of universities in the
QA effort, and the methodologies
for standard setting and quality
improvement. It concludes with
several recommendations
relating to management and
leadership, technical functions,
support functions, and program
management.
Table of Contents
I. INTRODUCTION.............................................................................................. 1
II. ORIGINS OF THE CHILEAN QA PROGRAM .................................................. 1
III. OVERVIEW OF THE CHILEAN QA ADMINISTRATIVE STRUCTURES ........... 3
A. The Quality and Regulation Unit ................................................................ 3
B. Health Services .......................................................................................... 3
C. Quality Committees .................................................................................... 3
IV. ADAPTING QA APPROACHES TO THE CHILEAN HEALTH SYSTEM ........... 3
A. Stages of QA Implementation .................................................................... 3
B. Sustainability of the Chilean QA Program ................................................. 5
V. EVALUATION OF THE QA PROGRAM STRUCTURE AND MANAGEMENT .. 6
A. QA Structure and Management in the Health Services ............................ 6
B. QA Structure and Management at Hospitals and Primary
Healthcare Centers .................................................................................... 6
C. Informal Aspects of QA Management: The “Magic” of QA in Chile ......... 6
VI. QA PROGRAM TECHNICAL FUNCTIONS ..................................................... 7
A. Quality Improvement .................................................................................. 7
B. Quality Design (QD) ................................................................................... 8
C. Quality Assessment and Monitoring .......................................................... 9
D. Standard Setting and Dissemination ......................................................... 9
E. Evidence-Based Medicine ....................................................................... 10
VII. QA PROGRAM SUPPORT FUNCTIONS ....................................................... 10
A. QA Training ............................................................................................... 11
B. Monitoring and Supervising QA ............................................................... 12
C. Dissemination and Communication ......................................................... 13
D. Incentive Programs .................................................................................. 13
VIII. RECOMMENDATIONS .................................................................................. 13
A. Program Structure and Leadership ......................................................... 13
B. Technical Functions ................................................................................. 14
C. Support Functions .................................................................................... 14
D. Program Management.............................................................................. 14
Acknowledgement
The authors are indebted to the staff of the Quality Assurance Project
(QAP) for technical and administrative assistance. The willingness of
the Chilean health authorities at the central, regional, and district
levels, especially the Quality Assurance (QA) Unit of the Ministry of
Health, to make themselves available at all times is recognized with
considerable gratitude. The evaluation team acknowledges the
support provided by the United States Agency for International
Development (USAID) to fund the final evaluation and thanks the
Agency for the opportunity to contribute to the success of the Chilean
QA Program.
Recommended citation
Legros, S., R. Massoud, O. Urroz, and E. Kelley. 2000. “The Chile
Quality Assurance Project: Final Evaluation.” Technical Report
Summary 1(2). Published for the U.S. Agency for International
Development (USAID) by the Quality Assurance Project (QAP):
Bethesda, MD.
About this series
The Technical Report Summary series provides concise descriptions
and background information on the Quality Assurance Project’s
technical intervention results, evaluation results, and other project
activities. This summary is derived from a longer report that can be
requested at qapdissem@urc-chs.com or 301-941-8532. For more
information on the evaluation of Chile’s Quality Assurance Program,
please contact slegros@urc-chs.com.
The Chile Quality Assurance Program: Final Evaluation (TRS) ■ 1
Stephane Legros, Rashad Massoud, Orlando Urroz and Edward Kelley
The Chile Quality Assurance Program:
Final Evaluation
Acronyms
DISAP Division de Salud de las
Personas
MOH Ministry of Health
NGO Nongovermental organization
PHC Primary healthcare
QA Quality assurance
QAP Quality Assurance Project
QD Quality design
QI Quality improvement
SATUS Servicio de Atencion al
Usuario
SEREMIS Secretarias Regionales
Ministeriales de Salud
USAID U.S. Agency for International
Development
I. Introduction
The Chilean Ministry of Health
(MOH) asked the Quality Assurance
Project (QAP) to evaluate Chile’s
program for the Evaluation and
Improvement of Quality, or QA
Program. It had been implemented
starting in 1991 with QAP’s technical
assistance and was funded by the
U.S. Agency for International
Development (USAID). The QA
Program now sponsors quality
assurance (QA) programs in nearly
all of Chile’s 13 regions and 29
decentralized Health Services.
The evaluation was conducted from
July 12–30, 1999 by a joint team of
three international consultants from
USAID/QAP and the director of the
QA Program. The MOH and senior
QAP staff provided information and
support. The evaluation team first
constructed a logical framework of
technical areas. Next, for each of
those areas it determined questions,
data sources, method of data
collection, and indicators for the
different questions. Lastly, it decided
whether data would be collected
before or during the evaluation visit.
This summary reports on the
following three areas:
■ QA Program structure and
management
■ QA Program technical functions
■ QA Program support functions
The evaluation team conducted an
extensive document review at the
Quality and Regulation Unit, which
directed the QA Program and which
is part of the Division of People’s
Health (DISAP) at the MOH. In
addition, the team conducted site
visits using structured and unstruc￾tured interviews, self-assessment
forms, and appreciative inquiry.
II. Origins of the Chilean
QA Program
Historically, Chile has been at the
forefront of healthcare service
provision in Latin America in terms
of prevention, primary care, and
medical education. Despite the low
incomes of almost half the popula￾tion, health indicators rank Chile
among more highly developed
countries in terms of healthcare. Its
socioeconomic statistics include:
■ 14 million inhabitants
■ 25 percent of population below
poverty level
■ 95 percent literacy rate
2 ■ The Chile Quality Assurance Program: Final Evaluation (TRS)
■ Life expectancy of 71.4 years for
men and 77.3 years for women
■ $2 billion annual health budget
■ $190 public health expenditures
per capita
■ 6.5 percent of Gross National
Product for health
■ Infant mortality rate: 9.8 per 1000
live births
■ Perinatal mortality rate: 4.1 per
1000 live births
■ Maternal mortality rate: 20 per
100,000 live births
In 1982, the MOH and the World
Bank launched the Health Sector
Reform Program. It decentralized
the public healthcare system—which
serves 60 percent of the popula￾tion—into 13 Health Regions, which,
in turn, were further segmented into
29 Health Services.
In 1989, Chile held its first elections
after 17 years of military dictator￾ship. The new government defined
the role of healthcare as essential
both to improved standards of living
and to achieving economic develop￾ment. The government’s social
policies were based on promoting
equity, social participation, respect
for the dignity of individuals, and
solidarity with poor and neglected
segments of the population. In the
effort to strengthen the public health
sector, quality of health services
ranked high as a priority among both
patients and providers.
Public sector patients expressed
concern over limited treatment
availability and rejection of patients
needing care. They also cited long
waiting periods; delayed care;
inappropriate use of costly special￾ists; and infrastructure-related
obstacles to quality care, such as
physical and structural decay. They
also expressed dissatisfaction with
user-provider relationships.
Concerns expressed by providers
included inadequacies in the work
environment, outdated technology
and infrastructure, inadequate
coordination, lack of incentives, high
work pressure, and lack of team￾work. Hesitation to honestly express
feedback was also acknowledged.
A small group of QAP consultants
worked with members of the Primary
Healthcare Department of the MOH
to design the national QA Program.
They concurred on the following
guiding principles:
■ The effort must be national in
scope
■ Participation in the QA Program
must be voluntary
■ Improvement must be achieved
at all levels of the system
■ A broad array of health stake￾holders must be involved
■ Existing technical and administra￾tive lines of authority must be
respected
Based on these principles, MOH
officials defined the QA Program’s
objectives: (a) raise awareness of
the importance of providing quality
throughout Chile’s healthcare
system, (b) develop a structure to
implement and sustain QA activities,
(c) achieve measurable improve￾ments in quality of care and service
delivery, and (d) improve patient
satisfaction.
The QA Program team, led by Dr.
Gilda Gnecco of the Primary
Healthcare Department, developed
a plan of activities to achieve the
objectives:
■ Organize a national conference
with representatives from all the
regions and Health Services, as
well as from universities, nongov￾ernmental organizations (NGOs),
and the private sector, to intro￾duce QA concepts and methods
and motivate local health authori￾ties to develop their own QA
plans and activities. This confer￾ence was held in July 1991 in
Punta de Tralca
■ Support training of healthcare
providers in QA skills by working
with regions and Health Services
to organize and conduct local QA
training courses
■ Promote and support the devel￾opment of QA committees at the
Health Service and facility levels
to plan and direct local QA
activities
■ Identify and train quality monitors
throughout the country to provide
technical support for QA training
and quality improvement (QI)
activities at the regional and
Health Service levels
Lastly, a major consideration was
that the QA Program had to be
financially and technically self￾sustaining.
The Chile Quality Assurance Program: Final Evaluation (TRS) ■ 3
III. Overview of the
Chilean QA
Administrative Structures
To achieve institutionalization of QA
functions, organizational structures,
such as the ones described in this
section, are needed to plan and
direct QA activities.
A. The Quality and
Regulation Unit
In 1991, the MOH appointed staff,
headed by Dr. Gnecco, at the
Primary Healthcare Department to
direct the QA Program. In 1995, this
became the QA Unit, and in 1997, it
became the Quality and Regulation
Unit within the Division of People’s
Health (DISAP). The new name
reflected its new mandate to
reinforce the regulatory role of the
MOH with a quality focus. In 1997–
98 the Quality and Regulation Unit
led staff throughout the MOH in
defining quality standards, estab￾lishing criteria for their achievement,
and developing indicators to
measure the achievement of
standards in 16 priority health areas.
The Quality and Regulation Unit then
coordinated the development of
standards, criteria, and indicators
that could be applied throughout the
regions, Health Services, and
facilities in addressing healthcare
needs. The resulting document,
Criteria, Quality Standards and
Indicators for National Health
Priorities, represented the first time
that quality standards were defined
at the national level in Chile.
B. Health Services
The Health Services are now the
strongest institutional base for QA
programming. In addition to assum￾ing overall responsibility for provid￾ing healthcare to local populations,
the Health Services administer
secondary and tertiary care facilities
while local municipalities administer
primary care facilities. Health
Services and municipalities make
decisions regarding financing,
program administration, and human
resources, effectively determining
the level of support for QA planning,
financing, and activities.
C. Quality Committees
Quality committees are one of the
most important components of the
QA Program. They are designated
by the Health Services, although
participation is voluntary. The
committees function at all levels of
the healthcare system. Membership
may include top level political
appointees; Health Service officials;
and representatives from local
universities, related governmental
services (schools or day-care
programs), and NGOs. Quality
committees set priorities, assign
tasks, coordinate training and
technical support, and coordinate
information sharing and dissemina￾tion. By October 1998, 98 quality
committees had been formed at the
regional, Health Service, and facility
levels (i.e., hospitals and primary
care clinics).
IV. Adapting QA
Approaches to the
Chilean Health System
At the outset of the QA Program, a
variety of state-of-the-art methods
were explored for improving delivery
of healthcare services. These
methods draw on QAP’s six-step
approach to problem solving and
process improvement to improve
healthcare delivery processes.
By the end of the program’s first
year, a standardized quality assur￾ance methodology emerged from
the process of conducting QA
training. A straightforward QI
process resulted that is flexibly
applied by teams throughout Chile.
The program offers a very structured
way of approaching quality prob￾lems, using guidelines, worksheets,
and prescribed formats for each
step of the process.
A. Stages of QA
Implementation
From July 1991, when the first QA
activity took place, through 1998,
the QA Program developed in four
stages, each with its own objectives
and achievements. The USAID￾funded QAP provided technical
assistance to the program during the
first and second stages. During the
first year, technical assistance
efforts emphasized QA training,
organizing quality committees, and
planning. During that period, the
central QA team within the MOH,
with support from international
experts, initiated activities through￾out Chile and quickly developed a
4 ■ The Chile Quality Assurance Program: Final Evaluation (TRS)
national profile for the program. As
implementation progressed, the role
of QAP advisors diminished. With
time, QA became self-sustaining,
relying only on local resources. This
section provides the details of this
implementation plan.
Stage I—Team Building, Skills
Development, and Project
Development (March 1991–
September 1993)
The QA Program became public in
July 1991 with a national QA
awareness and basic skills training
conference. Convened by the MOH
with the support and participation of
international consultants, the
conference hosted representatives
from the Health Services, universi￾ties, and professional associations.
It presented the objectives and
decentralized implementation
strategy of the QA Program and
motivated leaders from the Health
Services to begin QA activities by
forming quality committees, hosting
training seminars, starting QI
projects, and developing QA plans.
As program team leader, Dr. Gnecco
asked conference participants to list
the most pressing quality problems
at the primary healthcare level. The
next step was to form a central team
comprised of MOH staff and their
international counterparts.
The central team began to develop
six basic training modules and, for
the first two years, focused on
developing QA skills and capacity
in the Health Services. Training in
basic QA concepts was provided to
674 health professionals, covering
more than half of the country’s
Health Services, in the first year.
By September 1993, 2,800 health
professionals had received basic
QA training.
In the second year, the team
continued to build capacity by
stimulating the development of
small, targeted projects aimed at
achieving quality gains through QI
problem-solving methods. Small
committees, with the support of
monitors, began 44 QI projects by
1993 and many more ad hoc quality
improvements occurred without
formal projects.
Stage II—Decentralization
and Institutionalization of
QA in the Health Services
(September 1993–March
1995)
September 1993 brought an end to
outside funding for the QA Program
and the challenge of sustainability.
The central team sought to transfer
this responsibility and authority to
the quality monitors and committees
in the Health Services. The ultimate
goal was to have the Health
Services meet their own needs for
QA planning, training, and
monitoring.
During this stage, the central team
finished the QA training modules
and organized another national
conference on QA to present and
discuss experiences under the QA
Program. By the end of this stage,
more than 5,000 health professionals
had received QA training, and some
250 quality monitors had been
trained. More than 200 QI projects
were underway or completed. Of
Chile’s 29 Health Services, 26 had
QA Programs, and four had quality
units. Some 90 quality committees
had been formed at health center,
hospital Health Service, and regional
levels.
Stage III—Institutionalization
of a Quality Unit in the
Ministry of Health (March
1995–March 1997)
Direct QAP assistance ended in
December 1994, and the QA
Program transitioned to a third
stage: continued decentralization
and the permanent incorporation of
QA functions within the MOH
through creation of a Quality and
Norms Unit in DISAP. It was directed
to:
■ Continue to develop and decen￾tralize QA activities, coordinating
and maintaining permanent
communication with the regions,
Health Services, and facilities
■ Support QA activities in the
Health Services in an advisory
role, emphasizing the develop￾ment of quality policies and plans
■ Respond to requests for training
that would be financed by the
Health Services
■ Develop and sustain national
information-sharing and
dissemination channels
During this period, the Quality and
Norms Unit reviewed all existing
MOH standards and regulatory
documents to identify areas where
standards and regulations were
outdated or non-existent. The Unit
also began organizing groups of
experts (from both within and
outside the MOH) to work on
standards, protocols, and manuals
in areas where they were lacking.
The Chile Quality Assurance Program: Final Evaluation (TRS) ■ 5
Stage IV—Quality and
Regulation (March 1997–
Present)
In March 1997, in the context of the
health sector’s reform process, the
Quality and Norms Unit was re￾named the Quality and Regulation
Unit,1 still within DISAP. The newly
named QA Unit gained responsibility
to help develop the regulatory role of
the MOH with a quality focus. It has
since collaborated with the MOH in
two major areas:
Sixteen national healthcare priorities:
Identified by MOH, they are:
(a) cancer, (b) cardiovascular
diseases, (c) respiratory infections,
(d) mental health, (e) accidents,
(f) immuno-preventable diseases,
(g) tobacco, alcohol, and drug use,
(h) occupational health, (i) tubercu￾losis, (j) dental health, (k) HIV/AIDS
and other sexually transmitted
diseases, (l) diabetes mellitus,
(m) congenital and perinatal
infections, (n) sexual and reproduc￾tive health, (o) malnutrition, and
(p) ocular health.
National Healthcare Priorities and
Administrative Agreements: Working
with directors of major health
programs and units in the MOH, the
QA Unit helped prepare Administra￾tive Agreements, a comprehensive
list of quality standards, criteria for
their achievement, and indicators for
their measurement for the health
priorities.
Administrative Agreements were
established at the MOH to enable
the national delivery of healthcare
services. They represent the first
time that quality standards were
defined at the national level and
were designed to enable the
regions, Health Services, and
facilities to assess and monitor their
progress in achieving standards for
the health priorities. They specify
levels of responsibility for adequate
and timely provision of necessary
healthcare services and the requi￾site resources and technical and
administrative support. They lay out
regional and local plans for service
activities, with corresponding goals
and indicators. Chile’s structuring of
Administrative Agreements regard￾ing national health priorities pro￾vided the legislative foundation for
all national health programs in Chile,
including the QA Program.
Among the many Administrative
Agreements are these two
examples, chosen for their focus
on quality assurance. They were
defined by DISAP in consultation
with the Health Services; these were
for 1998:
■ To improve the quality of care
within the framework of the
national and local healthcare
priorities (Agreement No. 20)
■ To develop regulatory instruments
as a function of national
healthcare priorities and the
design of healthcare information
systems for the monitoring and
evaluation of the activities
defined in the national healthcare
priorities framework
The QA Unit was also given respon￾sibility to review and redefine, as
necessary, the regulatory role of the
MOH, which is ongoing.
B. Sustainability of the
Chilean QA Program
A crucial time in the life of the QA
Program came in December 1994.
When QA management and activi￾ties decentralized, the regions and
Health Services decided to support
both current and proposed QA
projects. This financial commitment
ensured the sustainability of QA
even though external funding
ceased.
In 1999, the Health Services allo￾cated approximately $85,000 to
continue previously launched QA
initiatives and design new ones.
Continued local financing of QA
efforts—including personal contribu￾tions by monitors to provide logisti￾cal support to QA committees—by
the Health Services demonstrates
the sustainability of QA as a major
programmatic focus of the health
system and its institutionalization at
the local level.
Eight additional QA initiatives have
been undertaken within the MOH
and the private sector since the QA
Program began in 1991. These
include the MOH’s Norms and
Regulations Department, which is an
accreditation division, and Servicio
de Atencion al Usuario (SATUS), a
central office focusing on user
satisfaction.
1 Hereafter, this unit is referred to as the “QA Unit.”
6 ■ The Chile Quality Assurance Program: Final Evaluation (TRS)
V. Evaluation of the QA
Program Structure and
Management
The science and methodology of QA
emphasizes teamwork, respecting
the opinions of colleagues and
patients, and specific problem￾solving techniques. Numerous
authors in the QA field have referred
to the phenomenon of team owner￾ship of QA methods and tools as
the “democratization of science.”
This phenomenon is particularly
apparent in Chile. Dr. Gnecco and
her colleagues embraced the QA
training model developed by Lori
DiPrete Brown and Professor Pedro
Saturno, who provided QAP techni￾cal assistance. Dr. Gnecco and her
colleagues from QAP disseminated
QA methodology throughout Chile’s
health sector. About 12,000
healthcare professionals, including
800 monitors, have been trained,
representing approximately 25
percent of the health workforce.
QA in Chile is closely linked to
public health. QA activities tend
to be more prevalent in primary
healthcare centers than in hospitals
and are primarily oriented towards
patient satisfaction and primary
care.
A. QA Structure and
Management in the Health
Services
QA leadership of the Health Services
follows the administrative pattern of
the MOH in that regional directors
are also in positions of responsibility
for the QA Program in their geo￾graphic area. Additional oversight
is provided by representatives
of the President (the Secretarias
Regionales Ministeriales de Salud or
SEREMIS). There are 13 SEREMIS,
one per region.
Relationships between some Health
Services and municipalities are
generaly collaborative and construc￾tive. Although full- or part-time
quality coordinators, incentive
systems, and budgets have been
designated for QA by some Health
Services and municipalities, others
have no QA personnel or devote
little if any of their annual expendi￾ture to QA. The evaluation team
found that committed QA leadership
was an important factor in the
success of QA activities at the
regional level in the majority of the
sites visited.
One case shows the difference
committed leadership can make.
One Health Service received training
in 1991–93 but generated little QA
activity. New leadership in 1996
encouraged professionals who had
received training to participate in
QA; the professionals then trained
other staff and initiated QA projects.
Administrative Agreements commit
facilities to a specific level of
performance for the coming year,
usually a minimum number of patient
services. Alternatively, the agree￾ments align with the national
priorities to reduce waiting time and
improve patient satisfaction. Facility
funding is based on goals stipulated
in agreements; this spawned many
QI projects targeting those goals.
B. QA Structure and
Management at Hospitals
and Primary Healthcare
Centers
Administrative structures and
healthcare facilities vary in terms of
their commitment to QA from one
Health Service or municipality to
another. A formal coordinator may
be designated on a part- to full-time
basis, or not at all. In some facilities,
staff are allotted time for QA and
may receive small incentives to do
them. Regional and facility support
for QA has a visible effect on the
level of QA activities, but its ab￾sence was not the only determinant
of the presence or level of QA
activities in that region or facility.
Some facilities have developed
significant QA initiatives in spite of
inadequate support. For example,
one hospital made strides in QA
programming and achieved several
QIs over the years. Regarded as a
leader in quality, the hospital has
slowed its efforts somewhat because
of a change in its leadership.
Significant variation also exists
between facilities in a Health Service
or municipality in terms of resource
allocation for and level of QA
activities, as well as in the sophisti￾cation of methodology. Many QI
projects focus on improving basic
infrastructure, such as making toilets
accessible for people with disabili￾ties. In a few other instances,
however, teams have focused on
more complex issues of clinical
care.
C. Informal Aspects of QA
Management: The “Magic”
of QA in Chile
The evaluation team became
increasingly aware of the unique￾ness of the Chilean QA effort,
particularly through the dedication
of professionals throughout the
country who are ready to engage
in QA regardless of the level of
The Chile Quality Assurance Program: Final Evaluation (TRS) ■ 7
support they receive. What unites
the Chilean QA effort extends far
beyond formal administrative
channels in the MOH to less formal
but strong bonds formed at the local
level. The team referred to this as
the “magic of QA in Chile” and
believes it was crucial to the
program’s success.
VI. QA Program Technical
Functions
This section reviews the Program’s
improvement, design, and monitor￾ing of quality as well as supervision
and standard setting. Indicators of
evidence-based medicine are also
discussed.
A. Quality Improvement
More than 625 QI projects have
occurred in Chile since the QA
Program began, and the vast
majority were performed according
to established QI cycles. This is a
process that entails: (a) identifying
problems related to quality of care,
(b) studying the causes of the
problem, (c) identifying a solution
or QI, and (d) implementing and
evaluating the QI. Training in using
the QI cycle consisted of a basic
four-day workshop with exercises
to familiarize participants with the
principles, conceptual frameworks,
and strategies of QI, as well as the
components of the traditional cycle
and work team principles. Teams
work on different types of opportuni￾ties for quality improvements:
structural, organizational, clinical/
technical, and others. Some
examples of opportunities
addressed in Chile are:
Structural problems: oblems:
Inadequate signage, need for better
patient waiting areas, lack of proper
hygiene and cleanliness standards,
and need for child care facilities for
patients and families
Organizational Problems: oblems:
Underuse of operating room,
inadequate information network
from the board of directors and the
Intensive Care Unit, inadequate
dissemination of technical nursing
standards in intensive care, and
inadequate information on emer￾gency admissions for patients and
their families
Technical/Clinical and Other: echnical/Clinical and Other:
Inefficient ambulatory surgery ward
management, need for an audiom￾etry service, inconsistent care for
patients with hypertension and
diabetes, unclear admission process
policies and procedures, delay
exceeding 24 hours to receive blood
exam results, high average patient
length of stay in intensive care, high
number of surgery cancellations,
lack of personnel motivation, and
long waiting time in the emergency
ward.
At the end of training, the trainees
are usually asked to commit them￾selves to the completion of cycles,
although no systematic approach to
project completion was in place at
the time of the evaluation. Monitors
follow up on the cycles, providing
encouragement, advice, and similar
kinds of support to the teams.
Basic QI tools include:
■ Nominal group technique, which
allows a team to reach consensus
quickly on the relative importance
of issues
■ Prioritization matrices, which
narrow options through a system
of comparing choices
■ Brainstorming, a common method
for generating a high number of
ideas on a given topic
■ Cause-and-effect and fishbone
diagrams and flow charts, which
provide graphic views of data
■ Force field analysis and pareto
charts (infrequently used in Chile)
■ Storyboards, which tell the story
of a QI project from the start to
post-completion analysis
QI efforts have resulted in many
infrastructure improvements
(e.g., to waiting rooms, bathrooms,
and hospital bedrooms) that un￾doubtedly would have occurred
during modernization. Other im￾provements relate to client focus,
such as allowing parents to spend
more time with their children in
hospital wards and improved
interpersonal skills of physician
and nursing staff.
“The QA methodology brings a
new way of thinking. It helps us
to innovate and to r to innovate and to realize what’ ealize what’s
going on” going on”
A Quality Coordinator
Baseline assessments of QI projects
are rarely performed in Chile prior to
the application of solutions, so it is
impossible to measure their suc￾cess. This is a weakness in the
program and may explain why
leaders and physicians are not more
involved in QA efforts.
8 ■ The Chile Quality Assurance Program: Final Evaluation (TRS)
One rationale offered by QI team
leaders for the lack of baseline
assessments is that after brainstorm￾ing, many teams tend to apply
solutions, because they believe that
the full cycle is too time consuming.
Another explanation is that a team
may be able to spontaneously
assess the problem without using
the prioritization matrices or other
steps in problem selection. About 70
percent of the QI teams in Chile are
using the full cycle, and the rest are
using an abbreviated version. Each
team has its own adaptation.
QI results also could be better
documented and disseminated.
At the regional level, specific tools
are often used to define strategic
plans for QI that are integrated into
the regional health plan. However,
time constraints often limit the
involvement of team members,
making completion of the cycles
and their application to actual
problems rare.
QI teams are organized according
to the problems selected and are
usually staffed by service, ward, or
unit personnel and often include
nurses, midwifes, auxiliary nurses,
social workers, nutritionists, and
administrative staff. A ubiquitous
problem is the under-involvement
of physicians. Generally, physicians
comprise no more than 10 percent
of any team, although Barros Luco
Trudeau Hospital in Metropolitan
region is one notable exception.
Physicians there became
increasingly involved and committed
to these efforts, thanks to the
insightful intervention of the nurse
administrator.
QI team attitude, motivation, and
behavior are influenced by Chilean
cultural and professional ethics.
Team meetings display discipline,
and planning is geared for results.
Although bureaucratic requirements
often interfere with optimal function￾ing, staff turnover was not a major
concern. The program is effectively
national and the number of trained
personnel is high, so team members
transferred to other regions partici￾pate in the teams there.
Another concern is the lack of
structured benchmarking (i.e., the
sharing of information between QI
teams) and the QA Unit’s failure to
develop a system to disseminate
success stories. Exceptions are:
(a) a selected number of projects,
many focusing on clinical or client￾related improvements, have been
disseminated for review and pos￾sible adaptation by teams in other
regions, and (b) The QA committee
at the Health Services level selects
the best projects to be financed
each year. The latter has included
breast-feeding awareness
(in collaboration with UNICEF),
presence of the father during
delivery, parent participation during
pediatric hospitalization, presence
of the family during hospitalization
of terminal patients, and special
nursing in-home service for termi￾nally ill patients.
B. Quality Design (QD)
The evaluation team identified
examples of quality design—the
design of activities for new or
existing processes—that had
occurred, although they were not
“textbook” examples. QAP has
established basic guidelines for
quality design efforts, requiring that
they: (a) be based on assessment of
client needs; (b) link client needs
with new features of a process; (c)
provide a new or substantial
redesign of a system, process, or
product; and (d) include some
failure proofing or test for robustness
of the new design proposed. QD
efforts in Chile included:
■ Five health centers in Rancagua
redesigned the annual, regional
immunization campaign. Now it
has a better distribution of tasks,
a common operational calendar,
and coordination among the
health centers
■ At the regional Guillermo Grant
Benavente Hospital in
Concepcion, a set of integrated
solutions was implemented to
halt a syncitial respiratory virus
epidemic
■ Hospital Barros Luco Trudeau
implemented ambulatory
surgeries, where patients are
discharged on the day of their
surgeries, with interesting
outcomes: a 25 percent reduction
in the number of patients waiting
for surgery, 24 percent of surger￾ies performed on an out-patient
basis, and a 20 percent reduction
in the number of surgery beds
■ In the Metropolitano Oriente
Health Service, an appointment
system for specialists’ visits was
designed to operate by fax
between the healthcare facilities
and hospitals. This facilitated and
accelerated the scheduling
system and appointment confir￾mations. Significant improve￾ments were made in time savings,
patient comfort, mechanisms for
tracking patients through
counterreferrals, and reductions
in no-show visits
■ In a southern health region, a
home care system for adult and
elderly patients was designed to
use the services of retired
auxiliary nurses
The Chile Quality Assurance Program: Final Evaluation (TRS) ■ 9
■ In the San Martin de Quillota
region, palliative care was
implemented
C. Quality Assessment and
Monitoring
The QA Program in Chile engages in
a variety of activities in the area of
quality assessment and monitoring,
from both the patient’s and health
system’s perspectives. Determining
levels of patient satisfaction is an
important step in the classic
sequence of QA implementation and
is evaluated through different quality
assessment tools. User satisfaction
is frequently measured through local
surveys, exit interviews, or focus
groups that enable local decision
makers to define strategies of
problem selection. The QA Unit
developed a diagnostic module in
response to a request from several
regions that had difficulties with
survey management. As a result,
suggestion boxes and books for
registering feedback were regularly
displayed in healthcare facilities.
The frequency of the data collection
varies by region. The evaluation
team observed that few provider￾satisfaction surveys are performed.
With regard to the measurement of
provider and facility performance,
the QA Unit designed a specific
module on quality indicators (input,
process, outcome, sentinel, and
impact). It explains the monitoring
process and provides group
exercises for creating and choosing
indicators. Indicators are also
provided in the Administrative
Agreements, and QI teams in most
facilities are using them. (At
Concepcion Hospital, impressive
use of indicators was seen in the
Intensive Care Unit.) In addition,
indicators are linked with new
standards in each regulatory
document from the MOH.
D. Standard Setting and
Dissemination
Beginning in 1997, the QA Unit
collaborated with health program
and unit directors within the MOH to
select and define healthcare
delivery criteria, standards, and
indicators. The effort resulted in an
official MOH document, Criteria,
Quality Standards and Indicators for
National Health Priorities. It was
intended to enable regions, Health
Services, and facilities to assess
and monitor their achievements in
health priorities on a national basis.
It contains evaluation criteria,
definitions, examples, matrices, and
suggestions of indicators to monitor
all management commitments linked
with the health priorities. It also
offers an instrument to help manag￾ers build an information system to
register, tabulate, analyze, and use
information for decision making. It
establishes the basis for quality
assessment activities and ongoing
monitoring of healthcare delivery
processes at numerous levels. Since
1997, an annual inventory2 of
existing regulatory documents has
been developed and disseminated
to all Health Service-related organi￾zations. It compliments the Criteria
compilation and facilitates the
supervision and monitoring process.
Ten volumes of regulatory standards
addressing Chile’s national health
priorities have been compiled,
published, and sent to Health
Services directors, program chiefs,
hospitals, municipalities, universi￾ties, and the private sector. These
standards represent a thorough
systematization and dissemination
effort as well as a significant
economic investment. In addition, an
MOH budget item includes regula￾tion, which will help ensure that
standards are developed and
adapted. The work is not finished.
Gaps immediately appear when the
national health priorities and
regulation materials are compared
(see Table 1); clinical standards are
still lacking in many areas. This
makes it difficult to evaluate and
monitor progress in those priorities.
The second phase of standard
setting began in 1998 under the
authority of the QA Unit and followed
a specific editing framework. The
objectives of this work were to:
■ Review and redefine, if neces￾sary, the regulatory role of the
MOH
■ Diagnose the areas requiring
regulation
■ Organize committees of experts
to address these areas
■ Standardize the editing of new
standards and other new regula￾tory documents
The QA Unit established national
working groups on healthcare
standards. These groups then
developed both technical/adminis￾trative standards and clinical
guidelines. They are elaborating
standards, protocols, decrees, and
laws in areas that lack up-to-date
regulations. The edition of the new
standards includes two complemen￾tary documents: (a) a checklist for
2 Catastro de Documentos de Regulación: Prioridades Salud País (Register of Regulatory Documents: National Health Priorities)
10 ■ The Chile Quality Assurance Program: Final Evaluation (TRS)
supervisors to perform efficient
monitoring of compliance with the
standards, and (b) a monitoring plan
that includes a complete set of
indicators (input, process, output,
sentinel, and impact) with national
indicators, regional indicators, and a
proposal for local indicators. This
instrument applies only for the new
standards.
Additional goals of the second
phase of standards development
and dissemination are to:
■ Disseminate clinical standards to
the appropriate level of care and
to the appropriate teams
■ Refine instruments for the
supervision of standards at the
national level
■ Establish a library of regulatory
documents
■ Systematize existing regulatory
documents related to the national
health priorities to allow internal
and external evaluation within the
system
The communication of standards
appears to face continuing difficul￾ties. Formal structures to create job
aids, training sessions, supervisory
policies, etc. have not been
established to ensure the use of
standards by practitioners. MOH
organized a meeting in August 1999
to begin addressing this gap.
E. Evidence-Based Medicine
The evaluation team is pleased to
report on two indications of the use
of evidence-based medicine,
although how these initiatives will be
coordinated and replicated is
unknown.
The DISAP director, a neurologist
skilled in evidence-based medicine,
launched an initiative in 1998 to set
standards in neurosurgery accord￾ing to evidence-based medicine
and a peer review process.
At Barros Luco Trudeau Hospital,
physicians are developing local
standards based on principles of
evidence-based medicine. They use
materials from the Cochraine Library
for their research.
VII. QA Program Support
Functions
This section presents the results of
the evaluation team’s review of QA
Program support functions in Chile:
QA training and the training mod￾ules, monitoring and supervising
QA, and dissemination and
communication.
Table 1
 Gaps between Health Priorities and Related Standards
National Health Priorities Clinical Areas with Existing and New Standards
Accident
Congenital and perinatal infections
Cancer Lymphomas, leukemia,
Diabetes mellitus
Cardiovascular diseases Hypertension
Immuno-preventive diseases EPI
Respiratory diseases
Malnutrition Nutrition
Dental health Odontology promotion and prevention
Mental health Mental health
Ocular health
Occupational health Occupational health
Reproductive health MC health, diabetes & pregnancy
Drugs, alcohol, and tobacco
Tuberculosis Tuberculosis
Human Immunodeficiency Virus Acquired Immuno-Defiency Syndrome/Sexually Transmitted Disease
Other Clinical and Management Areas That Have Standards
Adolescent health
Adult health
Epidemiology
Clinical laboratories
(include national policy)
Cholera
Intra-hospital infections
PHC
Health promotion
Dialysis and
transplantation
Blood bank
Rural health
Pharmacy
Health education
Traditional medicine
Environmental health
The Chile Quality Assurance Program: Final Evaluation (TRS) ■ 11
A. QA Training
Training is the fundamental strategy
behind the Chilean QA Program.
QAP led the first training in June
1991 with a Quality Assessment and
Improvement workshop, and 40
percent of the trainees were still
engaged in QA during the evalua￾tion. Generally, training was wide￾spread and designed to work with
the overall MOH strategy of
decentralization.
Approximately 20 percent of the
more than 60,000 healthcare
delivery personnel at the central and
regional levels of the MOH have
received some level of QA training.
Table 2 shows the growth of QA
training, demand for which contin￾ued to increase at least until the
evaluation and probably beyond.
To date, nearly 12,000 people have
been trained. This is not a static
result, but rather a success that is
expanding through local training
efforts and raising awareness of QA
concepts among social and profes￾sional communities.
Chile created its own QA training
modules (see Table 3) as part of
the central QA Unit’s commitment
to ensure that materials are
consistent with the country’s needs.
The modules were developed with
the participation of a range of
healthcare professionals, which
contributed to capacity building
and ownership.
The increasing responsibility
assumed for training at the regional
and local levels has affirmed the QA
Program’s sustainability—a consid￾erable accomplishment. As of this
writing, 37.8 percent of all reported
training was conducted at the
regional or local level. The actual
percentage is probably higher
because local teams do not always
report all the training they conduct.
Furthermore, these efforts are
supported by review at the central
level and by the supervision of
training materials. Most of the
regions have a quality plan, which in
some cases is incorporated in
annual strategic planning, so a
particular unit and its schedule of
training activities can be
documented.
The transfer of responsibility to the
local levels has stimulated creativity
in training strategies, and some
regions have achieved a high level
of development in the training
process. While decentralization
progresses, the challenge to ensure
the quality of the courses increases,
Table 2
Growth of the QA Training Program
Total by the Total by the Total by the
 End of 1995 End of 1998 End of 1999
Trained health professionals 5,254 10,600 10,775
Trained quality monitors 256 615 784
Health Services with quality plans 42% 75% 80%
Quality projects completed 200 625 NA
Table 3
List of Training Modules Completed (1994–99)
Module Numbers Titles
Modules 1, 2, 3 Basic QA Concepts
Modules 4, 6 Develop a Culture of Quality
Developing a Quality Plan
Module 5 Teamwork of Leadership
Modules 6, 8, 9, 11 Vision, Mission, Policies for Quality Plans
Evaluating Quality of Services
Statistical Methods for QA Indicators Development
Module 10 Rights of Duties of Clients
Modules 13, 14, 15, 16 External Client Satisfaction
Assessing Quality Projects
Supervision
History of Quality
Modules 7, 12 Training Monitors
Medical Audit
12 ■ The Chile Quality Assurance Program: Final Evaluation (TRS)
so certifying that training is of high
quality becomes necessary.
From the outset, public and private
universities were included in efforts
to institutionalize QA training in
undergraduate and postgraduate
schools and in clinical services.
A few among many examples of
university participation are provided
here: Approximately 70 percent of
Chilean universities teach quality￾related subjects at various levels.
The University of Valparaiso, a state
university, offers a course on QA in
the curriculum of an Obstetrical￾Nursing School course on adminis￾tration. In 1995, fifth-year students
there began internships in QA
projects through their practice in
primary care hospitals and clinics.
These students are also required to
prepare a thesis on quality-related
subjects with the objective of raising
user satisfaction. Such projects
provide research for university
department chairs and have been
incorporated into healthcare service
design.
At the University of Chile, important
contributions to the field of QA
programming in healthcare have
emerged in phonoaudiology;
nursing; kinesiology; and medical
technology and for postgraduate
teachers of epidemiology, environ￾mental health, maternal-infant
health, and pediatrics. In addition,
monitors have been trained through
the Clinical Services Department at
the José Joaquín Aguirre (teaching)
Hospital.
The QA Unit has trained 76 teachers
at the University of Conception, the
University of Valdivia, Temuco, and
the Nursing-Obstetrics School of the
Pontifical Catholic University. The
last has instituted a Continuous
Quality Improvement Division at the
undergraduate and postgraduate
levels in the School of Medicine and
Nursing. From 1996 to 1999 that
university had a quality plan in the
clinical hospital at the School of
Medicine. Meanwhile, training
programs continue to be developed
in basic, clinical, and preclinical
sciences through university medical
schools.
The QA Unit developed a five-day
basic training seminar in April 1999
with the support of regional monitors
and 10 Health Services. The pilot
plan was to train 70 physicians and
dentists who would then depart for
“social service” all over the country.
These trainees would be placed in
management positions in hospitals
and/or primary care clinics through￾out Chile. This effort involved the
deans of public and private schools
of medicine and dentistry.
QA seminars have been conducted
at professional training schools for
dentists, midwives, nurses, nutrition￾ists, psychologists, physicians,
kinesiologists, social workers,
and paramedical technicians. In
addition, scientific associations
representing specialty areas such
as neurosurgery, gynecology, and
endocrinology are consulted
regarding regulation.
B. Monitoring and
Supervising QA
Supervision within the Chilean MOH
is not formally integrated with QA
features and approaches, although
the QA Program offered training in
supervision in 1998 and created a
specific training module. In fact, the
supervision system is in crisis,
lacking such tools as checklists for
supervisors to use during visits to
healthcare facilities. Supervision
also seems to be poorly integrated
within regional training departments,
so technical training reinforcements
are not available when the need
arises. The QA Unit has initiated
some improvements. For example,
in the Osorno region, supervision is
integrated as a function of quality
management through workshops
where supervisors and program
coordinators design tools. Informa￾tion about this activity should be
disseminated to other regions.
After the initial awareness and
training workshop, potential monitors
receive training in QA theory and
practice en route to becoming
“premonitors.” Premonitors are
trained to facilitate tasks of the
multidisciplinary work team and
become monitors once they form
a QA team at their workplace.
Monitors can then become
pre-instructors and instructors,
who train and provide follow-up
and support to new QA trainees.
The monitors are committed,
cooperative, creative, and innova￾tive. Of the 784 monitors trained by
the QA Unit and the 180 monitors
trained by the Metropolitano Sur
Health Services between 1992 and
1999, 75 percent are still active,
reflecting their dedication and the
support provided by the system’s
administration.
All the groups studied during this
evaluation showed extremely high
levels of motivation for QA projects.
However, the evaluation also found
that QA teams believe they are
accorded insufficient recognition for
their efforts, and they acknowledged
that lack of incentive diminished
their QA-related work. Facility
The Chile Quality Assurance Program: Final Evaluation (TRS) ■ 13
managers with active QA teams
should recognize and reward work
team performance as an important
asset to management; they could
take better advantage of this highly
committed human resource.
Monitors and quality committees
find opportunities to meet in their
work units, as well as resources with
which to implement QA projects. In
so doing, they by-pass organiza￾tional hierarchies to provide training
for directors of services, hospitals,
and primary care clinics. Monitors
are models of solid foundations in
building a system of quality
management.
C. Dissemination and
Communication
Despite obstacles to direct commu￾nication, information flows constantly
through both countrywide networks
and within each Health Service.
The QA Unit has supported this
process through formal and informal
communication, creating some
opportunities for benchmarking.
The QA Program has been thor￾oughly disseminated throughout
Chile. The QA Unit and MOH
communicate continually with all
sectors of the healthcare system.
The program reaches all levels of
the healthcare sector, providing
immediate feedback on system
performance through formal and
informal communications. Success
can be attributed to the timely
exchange of information, to the
response to recommendations about
quality projects, and to receptive
and committed QA leadership.
Both central and regional efforts to
disseminate information on the
program have been established
through participatory forums such as
the Month of Quality and the
National Conference on Health
Services Quality. International
dissemination has been emphasized
since 1992 with participation in 10
international presentations or
conferences.
D. Incentive Programs
Chile has implemented the following
reward or incentive initiatives to
promote quality assurance:
■ The National Prize for Quality in
Public Services
■ The Day of Excellence in Health
■ The Zepeda Award, an incentive
for monitors who participate in
training
■ Local recognition of individuals
and work teams
These initiatives are not formally
integrated elements of incentive
systems, but rather are
acknowledgements of voluntary
efforts and do not typically involve
financial compensation. The dedica￾tion of personnel who contribute
their time and often financial support
has largely made the QA Program
successful.
VIII. Recommendations
In its evaluations, QAP seeks
information that could be useful in
planning and making decisions for
the QA program. While this section
lists several areas for improvement,
the team wants to emphasize that
the staff of the QA Unit has been
impressively committed to improving
the quality of healthcare.
A. Program Structure and
Leadership
The team recommends these further
improvements in organizational
structure and leadership:
■ Promote strategic planning within
the QA Program by focusing on
linking QA Program goals and
MOH policy directives
■ Strengthen central leadership of
the QA Program through support
to the QA Unit: Provide staffing
support and information
resources and create a National
Quality Council to link numerous
quality initiatives in QA
■ Emphasize QA methodologies
and tools as general approaches
for improving management,
especially by integrating QA and
management training for MOH
personnel
■ Strengthen physician involvement
in the QA Program with incentives
and efforts designed to increase
competencies and physician
interest in QA activities
■ Examine opportunities to increase
initiatives in the area of evidence￾based medicine
14 ■ The Chile Quality Assurance Program: Final Evaluation (TRS)
B. Technical Functions
The QA Program is engaged in a
range of technical activities, al￾though to date it has focused on QI
at only the facility level. Although the
technical level of the QA Program is
high overall, the team recommends
the following activities for technical
growth:
■ Continue efforts to link MOH
priority health policies and
objectives with regional and local
QI activities
■ Encourage the development of
larger quality efforts using QD
methodology
■ Consider streamlining QI efforts
for certain straightforward quality
problems when applicable
■ Strengthen capacity to monitor
quality at the facility and team
level so that baseline assess￾ments are conducted regularly
during QA activities
■ Demonstrate and document the
cost-effectiveness of QA activities
■ More regularly integrate client
input and client satisfaction data
into QI efforts
C. Support Functions
Support functions are key in devel￾oping and sustaining any national
QA Program. In particular, the
evaluation team felt that support in
two areas, training and dissemina￾tion, deserves particular attention in
the Chilean program:
■ Involve new directors in promot￾ing and approving the training
plan and in maintaining their
continuity
■ Continue training in basic QA
skills at the facility level. Link
training to the development of a
QA project, perhaps through
informal commitments between
the QA trainers and the trainees.
Involve Chilean health profession￾als already trained in QA in
training efforts
■ Maximize the impact of promo￾tional mechanisms and imple￾ment a systematic benchmarking
system (which is already embry￾onic)
■ More physicians and senior
healthcare leaders should
participate in the annual QA
conference. Allocate funds to
increase physician and leader
participation and to disseminate
conference materials. Comple￾ment the current schedule of
national QA meetings with regular
regional meetings on QA topics in
order to assess program develop￾ment and adapt QA Program
objectives and strategies
■ Use the Internet (for example, the
MOH website) and printed
materials to disseminate QA￾related information
D. Program Management
Project continuity and accomplish￾ment of objectives in the areas of
quality and regulation have only
been possible through increased
and unpaid overtime by the program
director and through the dedicated
collaboration of QA monitors and
committees. Ongoing constraints
imposed by Chile’s current eco￾nomic conditions may preclude
resolution of this problem. The
following should be weighed during
planning:
■ Allocate dedicated resources at
the central and regional levels for
QA activities
■ The MOH should examine the
human resources needs of the
QA Program. This is particularly
important for creating an authority
(an individual or committee) at
the MOH level that could inte￾grate and coordinate the quality
activities. The integration of
quality tasks with teams from
divisions other than DISAP is
difficult because of bureaucratic
tracks in the MOH organization
■ Address the lack of time allo￾cated for QA monitors to work on
the QA projects or to receive
training; specific time must be
allocated to quality activities,
particularly training