Impact
Evaluation
CDIE
United States Agency for International Development
PN-ACP-069 February 2002
REDUCING URBAN AND INDUSTRIAL POLLUTION
IN RUSSIA
A USAID wastewater treatment project improved Moscow’s drinking water and its
companion watershed development project advanced environmental education in the
districts near Moscow. A third project in Volgograd failed to build a viable air quality
monitoring program to curb pollution. Sustainability and replicability were undercut
by failure to plan for replication from the outset and promotion of state-of-the-art
(rather than appropriate) technology. Lack of good baseline data and the inability to
distinguish results stemming from the projects from those related to the economic
downturn made independent assessment of project effectiveness difficult.
SUMMARY
R USSIA HAS EXTENSIVE NATURAL RESOURCES, some of the best trained environmental professionals, and an
unmatched productive capacity. It also has some of the world’s most pernicious environmental prob￾lems—a legacy from the Soviet era. In many of its largest industrial cities, drinking water is substan￾dard and air quality poor. Pollution standards for industrial emissions are set high, but competing federal and
regional environmental laws create barriers to curbing indus￾trial air pollution and halting river and stream discharges. En￾forcement of environmental regulations is weak, and fines for
violations are set unrealistically low. Russia’s emerging market
economy has benefited the environment indirectly: many of
the most polluting industrial enterprises have either drastically
cut production or simply gone bankrupt because they were
unable to compete without subsidies or new equipment.
In 1993, USAID launched a multiproject environmental pro￾gram in Russia as part of an international donor “first-wave”
 Contents…
Background .......................................... 2
USAID’s Russia Environmental
Program ........................................... 3
Project Elements ................................... 5
Impacts ................................................. 8
Project Performance ............................ 11
Lessons Learned .................................. 14
2
Team Leader: Steve Gale
Team Members: Matthew Addison, Mark Hodges, Nick
Wedeman
The authors wish to thank Dr. Erwin Silverstein for his
logistical support.
effort to bolster political and economic support. This
Impact Evaluation examines three of those projects—
wastewater treatment, watershed development, and air
quality improvement. The wastewater treatment
project sought to improve Moscow’s drinking water
by upgrading selected upstream municipal wastewa￾ter treatment facilities and industrial enterprises. The
watershed development project sought to create a wa￾tershed management plan for communities in the
Moscow River Basin, aiming to curb runoff (mainly
animal waste) from commercial agricultural opera￾tions. The air quality improvement project focused
on improving conditions in industrial Volgograd by
developing a model air quality management program.
All three projects were implemented by the U.S. En￾vironmental Protection Agency (USEPA).
In November 2000, USAID’s Center for Development
Information and Evaluation (CDIE) fielded a four￾person team to assess project impacts. Through site
visits and in-depth interviews over five weeks, the team
found strong evidence that the wastewater treatment
project had very positive impacts on targeted treat￾ment facilities. The project also strengthened institu￾tions by using a combination of technical assistance,
in-country training, equipment transfer, and U.S. pro￾fessional study tours. In contrast, efforts under the
watershed development activities were much less suc￾cessful—with the sole exception of developing an en￾vironmental curriculum for schools. The air quality
improvement project did not meet its primary goal of
implementing a comprehensive air quality program
in Volgograd.
The lessons learned from the evaluation are that
USAID and its implementers must 1) assess carefully
the potential for upgrading environmental technol￾ogy in cash-strapped plants, 2) focus more on
planning for replicability from the start, 3) use ap￾propriate (not necessarily state-of-the-art) technol￾ogy, 4) consider the balance between short-term
results and long-term sustainability, and 5) collect
baseline and other appropriate data if it plans to assess
project effectiveness.
BACKGROUND
Since the breakup of the Soviet Union in 1991, Rus￾sia has worked hard to make the difficult transition to
open markets and more democratic institutions. The
journey toward a complete restructuring of the Rus￾sian economy and adoption of wide-ranging political
reforms has been perilous. The 1998 ruble devalua￾tion was a severe blow. However, most experts agree
that over the past 10 years, Russia, despite many set￾backs, has become more open and democratic.
Experts also agree that the environment suffered badly
under the Soviet regime. The Soviets gave environ￾mental protection a high priority in principle but a
very low priority in practice. Industrial growth was
almost always at the expense of clean air, soil quality,
potable water, and ecosystem health. Organic, indus￾trial, and toxic spills were frequent, environmental laws
were not enforced, pollution fines were set low, and
graft and corruption were all too common. The Volga
River, Russia’s largest, is still severely polluted by in￾dustrial waste, sewage, pesticides, and fertilizers. Many
Russian cities have air pollution levels that exceed
World Health Organization and USEPA limits to pro￾tect human health.
Russia’s emerging market economy has been a mixed
blessing for the environment. Many industrial enter￾prises are unable to compete in the open market, rely
on state subsidies, and use decades-old technologies
with few pollution controls. Some enterprises use out￾dated processes and equipment and are poorly man￾aged or undercapitalized. Many of the worst-polluting
factories have gone bankrupt or operate at only 50 to
60 percent capacity. At the same time, democratic
3
reforms have handed regional and local governments
the responsibility for cleaning up long-standing ur￾ban and industrial pollution. However, reform did not
provide authority to enact effective pollution preven￾tion laws and regulations, nor the ability to systemati￾cally collect fines from local polluters. And reforms
did not furnish the funds needed to help polluting
enterprises upgrade (or shut down).
Russia is the largest country in the world—about twice
the size of the United States. It com￾prises 21 semiautonomous republics,
49 regions (oblasts), and other terri￾tories. Russia’s vastness, enormous en￾vironmental challenges, unstable
regulatory situation, and other factors
mean that most USAID assistance is
directed at demonstration projects in
selected sites rather than at wide￾spread implementation.
Russia’s environmental protection
laws can be promulgated by legisla￾tive and executive bodies (federal
level), by citizens (oblast level), and
by local governmental bodies (local
level). Thus, unlike the United States
where state environmental laws (re￾inforced by federal laws) predomi￾nate, Russia often has two or more sets of competing
federal and regional laws and regulations. Even worse,
these laws are not always consistent. The federal Jus￾tice Ministry estimates that 25–35 percent of regional
legislation does not conform to Russian federal law.
Environmental laws and regulations in Russia are
implemented by a wide array of often-competing
government agencies at the federal, oblast, and local
level. Many of the activities performed at one level of
government may also be carried out in parallel at an￾other level, complicating coordination, compliance
and enforcement—and implementation of USAID’s
water and air quality projects, which were focused
primarily at the local level.
USAID’S RUSSIA
ENVIRONMENTAL PROGRAM:
WASTEWATER TREATMENT,
WATERSHED DEVELOPMENT,
AND AIR QUALITY
IMPROVEMENT
In 1992, the Gore-Chernomirdin Agreement provided
for the transfer of environmental technology and fi￾nancial assistance to the Newly Inde￾pendent States of the former Soviet
Union. In response, USAID launched
its Russia environmental program.1
Similar initiatives were mounted in
central and eastern Europe. The pro￾gram was originally designed to bol￾ster political and economic support
for Russia and to complement first￾wave support from the World Bank
and other major donors. It was clear
from the program’s inception that it
was to show tangible results as soon
as possible. Funded in 1993 at ap￾proximately $35 million, it later
reached $128 million. Of the origi￾nal $35 million, approximately $11
million went to USEPA, $9.5 million
went to a U.S.-based consulting firm,
and the remaining $15.5 million funded cooperative
agreements and other related procurements.
The CDIE evaluation concentrated on two water
projects (wastewater treatment and watershed devel￾opment) near Moscow and an air quality improve￾ment project in Volgograd.2
 USEPA began work on
the air quality improvement project in industrial
Volgograd officially in September 1993 and began
implementing the wastewater treatment and water￾shed development projects in spring 1994.
1
Formally called the Environmental Policy and Technology
(EPT) project.
2
Formally called the Russian Air Management Program (RAMP).
The Soviets gave
environmental protection
a high priority in principle
but a very low priority in
practice. Industrial growth
was almost always at the
expense of clean air, soil
quality, potable water, and
ecosystem health.
4
The wastewater treatment project aimed at improv￾ing the quality of Moscow’s drinking water. The strat￾egy was to introduce new approaches to upgrade the
operation of selected wastewater treatment facilities
and industrial enterprises upstream from Moscow in
the cities of Tver (Tver oblast), Gagarin (Smolensk
oblast), and Dimitrov (Moscow oblast). The treatment
project sought to improve operations by decreasing
pollution loadings using a standard operation main￾tenance evaluation assessment. This assessment is a
systematic review of all the factors (including admin￾istrative, maintenance, facility design, operations, and
staffing) that limit facility perfor￾mance. Under this project, Russian
specialists also came to the United States
to observe assessment procedures first￾hand and receive lectures and classroom
training. The project also provided a
limited amount of used and borrowed
equipment. Similar but more mod￾est support was provided to key in￾dustrial plants that discharge their
waste into wastewater treatment fa￾cilities to improve the flow of indus￾trial discharges (so-called pretreatment
programs).
The watershed development project
sought to reduce (or capture and
treat) agricultural runoff into rivers
that influenced water supplied to the
Moscow oblast. Concentrating on
small watershed management practices in the Istra
District (located just west of the capital) in the Mos￾cow River basin, this project concentrated on curbing
nonpoint-source runoff from farms and from two large
agricultural enterprises. The idea was to introduce the
latest management approaches at selected demonstra￾tion sites, followed by informational open houses and
public information campaigns to spread the word to
adopt improved agricultural practices. The activities
at Istra also included an in-country educational com￾ponent directed at school children and a teacher ex￾change program. Both components were designed to
increase public awareness, facilitate replication in
nearby water basins, support local regulations, and
strengthen environmental nongovernmental organi￾zations (NGOs). Other activities at Istra included sur￾veying the environment, polling communities,
conducting livestock enterprise feasibility studies, up￾dating an ecological database, reviewing solid waste
management procedures, and testing water quality.
The air quality improvement project sought to im￾prove air quality in Volgograd. Volgograd was selected
because USEPA ranked it as the sixth most-polluted
Russian city and because of prior posi￾tive relationships among the World
Bank, USEPA, and Volgograd city of￾ficials. The project aimed to develop
and test a model air quality manage￾ment (AQM) program, based prima￾rily on training, technology transfer,
and public awareness. USAID and
USEPA assumed that the model pro￾gram would later be integrated at the
federal level into the Russian environ￾mental regulatory framework and
widely replicated.
Nine major components make up a
traditional AQM program (see box,
p. 5). Air quality monitoring, for ex￾ample, helps establish the relationship
between specific pollution sources and
ambient air quality. The air quality
project conducted monitoring in the Volgograd “tri￾angle” consisting of the Red October Steel Mill,
Volgograd Aluminum, and the Silica Brick Factory.
Another component is emissions inventories/emission
factors analysis, which is designed to improve perfor￾mance, increase accuracy, and lower the costs of esti￾mating actual air pollution emission rates. Thus, the
air quality improvement project was intended to make
the Volgograd emissions inventory more complete and
accurate. Other key AQM components include source
assessments, emissions testing, human health and risk
assessment, and compliance and enforcement
The watershed
development project sought
to reduce (or capture and
treat) agricultural runoff
into rivers that influenced
water supplied to the
Moscow oblast. The air
quality improvement
project sought to improve
air quality in Volgograd.
5
methods and procedures. This last component in￾cluded visible emissions (i.e., USEPA Method 9, “read￾ings by trained and biennially certified observers”) used
in Volgograd to train regulatory staff on alternative
inspection methods. The air quality improvement project
also had activities in the other traditional AQM com￾ponents of increasing public participation, establish￾ing legal frameworks, and providing basic training.
PROJECT ELEMENTS
Environmental pollution abatement and management
projects typically are built around five elements: eco￾nomic policy reform, government regulations and stan￾dards, institution building, education and awareness,
and technological change. Most environmental activi￾ties fall into one or more of these elements, and most
environmental solutions will involve working with
more than one element.
Economic Policy Reform
Economic policy can be an important contributor to
sound environmental management. The challenge is
to get decisionmaking to reflect the external costs of
environmental noncompliance and impacts; proper
pricing of resources, appropriate incentives and pen￾alties, and an understanding of financing constraints
can help. Although neither the wastewater treatment,
watershed development, nor the air quality improve￾ment project was explicitly designed to reform eco￾nomic policy, an understanding of Russian pricing,
incentives, policies, and funding will help interpret
performance, impacts, and lessons learned.
Russia has historically undervalued production inputs.
Soviet central planning, which did not recognize the
role of markets, undervalued production inputs. Sub￾sidizing key inputs, such as oil, coal, gas, or water, can
increase the demand for these production inputs and
the environmental damage associated with their over￾supply and misuse. Artificially low input prices, in￾cluding the price of energy, lead to excessive
consumption per unit of output and high levels of
waste and pollution. Even after independence, Rus￾sian energy prices were still below market levels, and
as a result pollution and natural resource degradation
were greater than they would have been otherwise.
Russia’s inefficiency is clear; in 1997, Russian energy
consumption per $1,000 of GDP was 61,000 Btu,
the highest rate in the world. By comparison, energy
consumption in China was 41,400 Btu per $1,000 of
GDP, and in the United States it was 11,600 Btu.
Russia has long had a system of pollution charges, fines,
and user fees. Pollution charges on emissions or efflu￾ents are defined by law. Fines are five times the pollu￾tion charge for excessive emissions or effluents. These
market-based instruments, which raised 1.3 trillion
rubles ($255 million) in 1996 and 1997, have not
proved to be an effective deterrent. The rates are quite
low and were not adjusted to reflect the huge ruble
devaluation in 1998. Compliance and enforcement
are generally weak. Of the 250,000 reported viola￾tions, 40 percent continue unpunished or unmitigated.
Fees and fines for air and water pollution are collected
by the local ecological fund. In theory, 10 percent is
transferred to the federal government and 30 percent
to the Regional Environmental Committee. Of the
60 percent remaining in the ecological fund, 80 per￾cent (about 43 rubles of each 100 collected) go to
wastewater treatment, and 20 percent go for public
Air Quality Management program
elements at a glance…
■ Air quality monitoring
■ Emissions inventories/emission factors analysis
■ Source assessments
■ Emissions testing
■ Human health and risk assessment
■ Compliance and enforcement methods and
procedures
■ Increasing public participation
■ Establishing legal frameworks
■ Providing basic training
6
education and awareness. In practice, a far higher per￾centage of the amount collected still goes to the fed￾eral government, with little typically returned to the
local level where payments are collected.
Poor financial performance in most industrial sec￾tors—particularly in older facilities—and the need to
protect indigenous industries from lower priced im￾ports preclude aggressive use of fees and penalties at
this time. Russia allows water authorities (vodocanals)
to charge user fees for water treatment services, but
these fees are generally insufficient for full cost recov￾ery. Fines and fees are not sufficient
to finance needed assistance, nor of
sufficient size to serve as a disincen￾tive for illegal, inappropriate, or
harmful environmental activities.
Russia lacks adequate financing for
environmental upgrades. The re￾structuring of government environ￾mental institutions, economic
downturns, and privatization con￾tribute to this problem, one of the
most critical facing Russia. Munici￾palities are heavily dependent upon
federal outlays; most do not have sufficient resources
to provide needed services. Municipal bond or lend￾ing markets are inadequate, as in many developing
countries. A difficult economic and political situation,
uncertain intellectual property rights, and changing
contract law are not conducive to foreign or domestic
investments. Available cash flow is typically used to
finance ongoing operations—especially payrolls. The
ecological funds exist to provide financing to govern￾ment environmental agencies, but they depend on un￾realistically low fines and fees. Moreover, their
capitalization is small relative to the problems, and many
of their resources go to salary and operating expenses.
Government Regulation and Standards
Environmental laws, standards, and regulations—and
the government’s capacity and willingness to enforce
them—are the keys to protecting and improving en￾vironmental quality. Russia has some of the most strin￾gent environmental standards in the world, and some
of the laxest enforcement. Emission standards are na￾tional in scope with no allowance for local conditions.
This rigid approach often results in standards that are
more or less stringent than necessary, and thus pro￾vide either more protection and cost than are neces￾sary—or not enough protection.
The complexity of the laws and their inconsistency
across jurisdictions reduce effective compliance, drain
resources, and undermine the politi￾cal will for enforcement. The waste￾water treatment project designers
recognized the importance of intro￾ducing more rational environmental
laws and regulations, and USEPA es￾tablished a separate task force to deal
with general environmental legal is￾sues. While new measures were largely
postponed because of fast-changing
Russian politics, the wastewater treat￾ment project successfully introduced
the “temporary permit” concept,
whereby a plant would be issued a
temporary permit and not be fined if it was making
progress toward reaching a legal standard. The water￾shed development project did not address changes in
government regulations.
The air quality improvement project sought to
strengthen compliance and inspection by addressing
air pollution regulations and standards. Russia has
some of the strongest emissions standards in the world,
but deficiencies in the existing permitting and enforce￾ment processes make successful enforcement impos￾sible. To offset these deficiencies, the project team
worked with stakeholders in Volgograd to strengthen
the existing system by concentrating on measures with
immediate and practical financial benefits. Good ex￾amples include the introduction of visual emissions
testing (as an enforceable standard) and creative adop￾tion of an oil emulsifying process to dramatically
Russia has some of the
most stringent
environmental standards
in the world, and some of
the laxest enforcement.
7
increase fuel efficiency. The air quality improvement
project also sought to rationalize the existing emis￾sions fee system and strengthen Volgograd’s inspec￾tion programs.
Institution Building
Effective management of Russia’s environment requires
strong institutions that work in close harmony. Strong
institutions require political commitment, effective
enabling rules and regulations, trained and motivated
staff, and sufficient financial resources. Moreover, the
environmental conditions facing both users and regu￾lators make it important that institutions adopt a pro￾active strategy emphasizing collaboration over
confrontation. However, the institutions charged with
managing the environment are often underfunded and
therefore cannot properly staff, train, equip, and man￾age their tasks. Underfunding for environmental in￾vestments typically results from low user charges—and
from inadequate federal budget allocations and real￾locations. Less than 1 percent of Russia’s federal bud￾get is allocated for environmental management and
investment.
The wastewater treatment project confronted basic
equipment shortages and the lack of familiarity with
modern methods in wastewater treatment and facility
maintenance and repair. Most of the wastewater treat￾ment facilities lacked even the simplest equipment—
basic equipment such as flow meters, automatic
samplers, pipettes, dissolved oxygen meters, Sludge
Judges, and colorimeters—that would have signifi￾cantly increased their ability to improve treatment ef￾ficiency and effluent quality. Most labs lacked spare
parts and reagents for water quality testing. The project
provided training for the staff of municipal facilities
and industry and a very modest amount of water test￾ing equipment. USEPA cooperators donated addi￾tional equipment—but not as a planned intervention.
For the watershed development project, the principal
problems were 1) a lack of knowledge of appropriate
animal waste management techniques; 2) a general
public that did not understand animal waste prob￾lems or how to address them; and 3) institutions at
cross-purposes and lacking sound data collection,
measurement, and analytical techniques. In most ru￾ral areas, where water pollution loads are predomi￾nantly household sewage and agricultural waste,
treatment staff had no formal work-related training.
They were not able to make even the simplest repairs.
According to project papers, USEPA’s approach was
to “design the Istra project based on the watershed
management approach as developed/practiced in the
United States.” At some sites, project-provided funds
were set aside for rural staff so they could make minor
repairs and help initiate primary water treatment ap￾proaches. The watershed development project at Istra
also trained staff in some basic environmental animal
waste management methods.
The main air quality improvement project goal was
to strengthen environmental institutions, including the
Center for Environmental Training, the Volgograd
Environmental Services Administration (VESA), and
several local NGOs. The project also worked to de￾velop more integration between VESA and the NGOs
and to increase NGO collaboration.
Education and Awareness
Public education and awareness can broaden the im￾pact of a technical assistance program by involving
more people in support of community-related activi￾ties, thus strengthening local government capability.
Although not the primary focus of the wastewater
treatment project, the watershed development project
focused heavily on learning and awareness by educat￾ing school children, holding public events, and imple￾menting an extensive grassroots outreach program
through newspaper articles and radio campaigns. The
most noteworthy achievements were the development
of an environment curriculum and associated instruc￾tional materials, teacher training, and class activities
at seven secondary schools.
8
The air quality improvement project’s
education and awareness efforts were
directed at increasing public knowl￾edge of environmental matters and air
pollution health effects, as well as
training regulatory, institutional, and
NGO staff on environmental issues.
It also built on the proactive efforts
of Volgograd environmental officials,
city administrators, and citizen advi￾sory groups.
Technological Change
New technologies and techniques can
reduce costs and/or broaden environ￾mental and economic impact. Waste
minimization, byproduct recovery, re￾cycling, and pollution prevention are
complements to abatement and treat￾ment in improving environmental
quality. The wastewater treatment
project used operation maintenance evaluation analy￾sis to identify low-cost/no-cost ways to increase envi￾ronmental quality by improving techniques and
performance at existing wastewater treatment facili￾ties. The watershed development project introduced
geographic information systems as well as new proce￾dures in animal waste handling, storage, and use.
The air quality improvement project introduced new
technology in the form of equipment and training.
Training activities focused on operations and mainte￾nance techniques, and technical training in the op￾eration of laboratory apparatus, ambient air and air
pollutant source sampling devices, and other equip￾ment. The project also provided visual emissions train￾ing, environmental audit support, U.S. air quality
familiarization tours, and exposure to the latest envi￾ronmental management techniques. USAID provided
equipment, including $4 million in state-of-the-art
analytical laboratory equipment, ambient and source
sampling equipment, visual emissions smoke
generators (for training and certifica￾tion), and a prototype steel smelter
ceramic cover (“delta”) for an electric
arc furnace.
IMPACTS
The team assessed the overall project
impact along institutional, environ￾mental, human health, and economic
dimensions. The team’s findings are
discussed below.
Institutional Impacts
Institutional impact refers to the estab￾lishment and strengthening of environ￾mental institutions.
The team concluded that the waste￾water treatment project produced
strong and important positive impacts
on institutions. Prior to the project, authorities con￾ducted only routine or “grab” water sampling at one
point in time. Moreover, locally authorized laborato￾ries were not well-equipped or certified. The team
found strong evidence that the project strengthened
participating institutions at all levels—regional envi￾ronmental committees, water authorities, individual
treatment facilities, and industrial enterprises.
Most in-country training in wastewater treatment was
provided by those who were trained in the United
States on project study tours, through local consult￾ants, or through the Smolensk Environmental Train￾ing Center—a local project beneficiary. Enforcement
institutions (regional environmental committees and
laboratories) were provided with tools and training so
their “water quality findings” could not be refuted by
local industries. Wastewater treatment project train￾ing and equipment allowed the labs to become fully
certified and to monitor major polluters on a con￾tinuous basis. Industry and local government both
The wastewater treatment
project produced strong
and important positive
impacts on institutions.…
The project strengthened
participating institutions
at all levels—regional
environmental committees,
water authorities,
individual treatment
facilities, and industrial
enterprises.
9
reported that this monitoring actually reduced con￾flicts and provided for a more positive working part￾nership between authorities and industries. Another
institution-strengthening accomplishment was the
introduction of temporary permits. Russian officials
in two of the three oblasts visited by the team indi￾cated they had successfully implemented such a per￾mitting scheme after their U.S. study trips, where they
had observed temporary permits in action. For ex￾ample, the Tver oblast Regional Environmental Com￾mittee now issues about 1,000 permits per year, of
which 30 percent are temporary. Seventy-five percent
of those issued temporary permits are reaching legal
standards within the allotted time. The Russians char￾acterized this innovation as invaluable.
The watershed development project also had a posi￾tive, albeit minor, institutional impact by working di￾rectly with government institutions and providing
training. The team concluded that national, regional,
and local government institutions improved qualita￾tively as a result. For example, the Istra school admin￾istration put to good use the training, curriculum
development, and teaching materials. The Moscow￾based Water Design Institute (VodNIIinformproekt) re￾ceived geographic information system and watershed
management training and wove these concepts into
its guidebooks, strengthening the collection and ana￾lytical capabilities of the many local government in￾stitutions involved in data collection. The project also
demonstrated the positive benefits of joint efforts in
watershed management, in contrast to the Russian
practice of separate activities.
At the institutional level, the air quality improve￾ment project provided improved monitoring, com￾pliance, and enforcement tools at the local
regulatory agency level and helped develop a better
informed and more capable roster of NGOs. It also
attempted to influence the regulatory approach at the
federal level by introducing new types of air sampling
and analysis equipment and procedures, including the
introduction of visual emissions as a regulatory
compliance assurance tool. Team interviews and
onsite meetings with project staff, government of￾ficials, and NGO leaders consistently revealed that
the air quality improvement project strengthened
the capabilities, performance, and effectiveness of
VESA, the Center for Environmental Training, and
local NGOs, producing marked cooperation among
these key players.
Environmental Impacts
An environmental impact refers to improved water and
air quality.
The team found that the wastewater treatment project
had a positive impact on the environment. This was
the direct result of introducing new equipment and
training. Wastewater quality increased at all munici￾pal sites where equipment and training were provided,
as documented in USEPA final reports and in onsite
data reviews with facility managers. However, estimat￾ing exactly how much impact there was proved diffi￾cult. There had been no attempt to precisely determine
how much improvement was due to the new equip￾ment and how much was due to new methods with
existing equipment. Compounding the measurement
problem was the general decline in economic activity
that altered wastewater composition and treatment vol￾ume during the life of the project.
The team found a general consensus among govern￾ment officials that direct wastewater improvements
took place at facilities not directly receiving project
assistance through either training or use of new tech￾niques. The Smolensk Environmental Training Cen￾ter received high praise for training nearly 700
wastewater treatment facility operators during the
study period. The Center tracked water quality at many
of these “improved sites”—both before and after train￾ing—and found decreased loadings at those facilities
receiving training. However, the Center did not at￾tempt to separate the improvements in water quality
due to the economic situation from improvements due
to training.
10
The watershed development project
sought to have an immediate environ￾mental impact through various activi￾ties. Project activities included
building dams, manure lagoon retain￾ing walls, and storage facilities. Ac￾tivities also included working with
local livestock operations and dem￾onstrating proper manure applica￾tion. The project successfully
implemented much of what it set out
to do, but the environmental impact
of these interventions was modest.
Factors included the relatively small
size of some demonstrations, the de￾cline in production activity (mainly
agricultural) at others, and the fail￾ure of some interventions to take firm
hold—such as the planting of seed￾lings and sludge treatment upgrades.
By all accounts, the watershed devel￾opment project’s environmental im￾pacts were negligible.
Recent environmental data indicated that Volgograd
ranked as the twentieth most polluted Russian city; it
was the sixth at the outset of the air quality improve￾ment project. Despite several published reports that the
air quality improvement project reduced air pollutant
impacts (stationary-source particulate emissions) by 8 to
12 percent, the evaluation team found that most pollu￾tion reductions actually came from curbing
nonsmokestack or vent emissions—primarily road dust.
However, interviews with facility operators and VESA
staff indicated that fully implementing the project-rec￾ommended measures could have realized approximately
an additional 30–35 percent reduction in emissions. Fur￾ther complicating assessment of the project’s environ￾mental impact, major economic downturns dampened
the Red October Steel Mill’s output and the Silica Brick
facility closed (though a smaller, specialty operation
continued). VESA officials claimed the environmental
improvement in Volgograd was not due to the air
quality improvement project but to
the general downturn in industrial
output and facility closings.
Human Health Impacts
Human heath impacts are those asso￾ciated with disease prevention and
health promotion, reflecting that air
pollution contributes to diseases such as
lower respiratory infections and chronic
bronchitis, as well as respiratory symp￾tom days, hospital admissions, and pre￾mature mortality. Similarly, polluted
water leads to a range of intestinal and
other disorders.
The team concluded that neither the
wastewater treatment nor the water￾shed development project produced
measurable human health impacts. In
any case, measuring the health im￾pacts of either would have been diffi￾cult because the downstream beneficiaries (in Moscow)
were not immediate project participants. In theory,
improvements in the quality of water provided by the
municipal wastewater treatment facilities and indus￾tries would be reflected over time in improvements in
Moscow’s drinking water. However, this assessment
was beyond the scope of the wastewater treatment project,
and local authorities at the treatment facilities were very
hesitant to link changes in their effluents to down￾stream changes in Moscow’s drinking water supply.
The team found few, if any, demonstrated health ef￾fects from the watershed development project at Istra.
The air quality improvement project also produced
no measurable human health impacts, a conclusion
confirmed by most VESA interviewees. The project
initially sought to prioritize some emission reductions
based on the measurement of pollutant exposure in
order of severity and the extent of health risk. How￾ever, this approach was later abandoned. Plant pro￾duction downturns, weak assumptions about the links
Despite several published
reports that the air quality
improvement project
reduced air pollutant
impacts … by 8 to 12
percent, the evaluation
team found that most
pollution reductions
actually came from
curbing nonsmokestack or
vent emissions—primarily
road dust.
11
between pollutants and respiratory illness, and the
upswing in vehicular nonstationary pollution pre￾cluded any assessment of the project’s health impact.
Economic Impacts
Economic impacts are all impacts directly valued in the
marketplace and incurred directly by project participants
(financial impacts) and those borne by additional par￾ties not captured in the financial analysis (e.g., social
and health-related costs).
The team could not determine the economic impact
of either the wastewater treatment or watershed de￾velopment projects. First, there was no solid baseline
data and second, where baseline data existed, it was
not possible to disentangle the impact of the overall
economic downturn from reductions due to improved
water treatment processes and practices. In the end,
the data on wastewater treatment did not permit con￾clusions about how much improvement was due to
reduced inflows from industries cut￾ting back on production and how
much was due to treatment plant
upgrades. There was no measurable
economic impact on project partici￾pants or society from the watershed
development project.
For the air quality improvement
project, the only documented posi￾tive economic impact stems from the
use of a diesel oil (boiler fuel) emul￾sification process at the Red October
Steel Mill—not a planned interven￾tion. Using waste fuel oil emulsifica￾tion dramatically reduced mill fuel
consumption, emissions, and waste￾water discharge, reportedly saving ap￾proximately $2,000 per day in fuel
costs. Based on interviews with resident engineers, the
team concluded that this process had been widely rep￾licated. However, the project’s development and use
of an electric arc furnace ceramic cover (delta), which
had the potential to deliver economic benefits to the
pilot mill (as well as to an additional 600 smelters
across Russia), was never realized due to the inability
to secure licensing for local production and the high
cost of importing finished units from the United States.
PROJECT PERFORMANCE
Project performance is assessed on the basis of effec￾tiveness, sustainability, and replicability.
Effectiveness
Effectiveness is the measure of how well the development
assistance met project objectives.
Overall, the wastewater treatment project was highly
effective while the watershed development project
appears to have been much less effective—with the
surprising exception of its environmental education
component. The wastewater treatment project pro￾moted the transition from one-point
or “grab sampling” to continuous au￾tomatic water sampling, using bought
and donated reconditioned equip￾ment. This technique allowed the pol￾lution control authorities to develop
an industrial enterprise “profile” to de￾termine who was discharging illegally
during nonmonitored periods. As a
result of using automated sampling,
treatment facility discharges were bet￾ter managed—in some cases reduced
significantly—and wastewater treat￾ment facilities were better able to treat
incoming industrial enterprise waste
streams. In the majority of industrial
enterprises visited, improvements
were linked to project-provided equip￾ment and technical assistance. Team
visits confirmed that all three municipal sites had
adopted and were continuing to use project-provided
pollution control techniques and that water quality
had improved.
For the air quality
improvement project, the
only documented positive
economic impact stems
from the use of a diesel oil
(boiler fuel) emulsification
process at the Red October
Steel Mill—not a planned
intervention.
12
Additionally, Russian wastewater officials were exposed
to U.S. monitoring and enforcement methods and
corresponding guidelines. This was accomplished
through U.S. training and by joint industrial inspec￾tions in Russia. Joint inspections also led to better re￾lations between industry and pollution control
authorities. The use of monitoring equipment ended
long, drawn-out discussions about discharges. As a
result, the industries have begun to operate at the le￾gal limit and notify the wastewater treatment facility
if there are accidental discharges or “shock loads.”
The watershed development project demonstration
activities were not broadly implemented. An unstable
farming and livestock sector, the relatively short-term
nature of the project, and a difficult objective—alter￾ing lifetime agricultural practices—all contributed to
only modest changes. The project introduced new
methods, including data collection, measurement, and
analysis. However, these new methods and the water￾shed management guidance manual
produced by the project were insuffi￾cient to produce a better-functioning
watershed—let alone serve as a model
for replication. The Istra project suc￾cessfully trained teachers and created
an environmental curriculum, devel￾oping text for grades 2–5, 6–8, and
9–10, along with other environmen￾tal materials, teaching manuals, slides,
and films. The project also established
a local environmental NGO and
worked with local media to spread the
word on environmental issues.
Overall, the air quality improvement
project had mixed results, even after
the scope of the project was dimin￾ished by considerably reducing its
geographic focus and number of tar￾geted industries. Project training and
public participation accomplishments
far outweighed those resulting from technological
upgrades, equipment installation, or alterations in air
quality monitoring techniques and procedures. For
example, of the nine major components necessary for
establishing an AQM program, the team found evi￾dence of limited effectiveness for only four compo￾nents: air quality sampling, emissions testing, public
participation, and training. The team found no, or
only weak, evidence of effectiveness for emissions in￾ventory, human health and risk assessment, and com￾pliance and enforcement components.
Sustainability
Sustainability is the degree to which a project continues
to provide benefits after development assistance ceases.
Overall, the wastewater treatment project was highly
sustainable. USEPA was well aware of the need for
program sustainability and emphasized training, low￾cost/no-cost methods, and institutional strengthen￾ing. Except in those circumstances where an industrial
enterprise had closed, the team found strong evidence
that almost all wastewater treatment
activities were sustained at a high level
three years after project completion.
Local authorities were able to obtain
funds for routine repairs of the
project-provided or donated equip￾ment. Wastewater efforts also contrib￾uted to financial sustainability as
several water testing laboratories be￾came certified, allowing them to per￾form private sector services for a
fee—a growing source of needed in￾come. Project-supported activities—
such as joint training and automated
sampling—strengthened institutional
capability, which over time made it
easier to resolve discharge-related con￾flicts. Wastewater treatment facilities
and the industrial enterprises they
served even came to be seen as part￾ners in a process. The Smolensk En￾vironmental Training Center’s
continued operation and ongoing outreach effort was
further evidence of project sustainability.
The watershed
development project
demonstration activities
were not broadly
implemented. An unstable
farming and livestock
sector, the relatively short￾term nature of the project,
and a difficult objective
… all contributed to only
modest changes.
13
In contrast, the watershed development project showed
few signs of sustainability. Indeed, officials implement￾ing the watershed development project
rightly concluded early on that sustain￾ing many of the demonstration ac￾tivities would be difficult. Factors
impeding sustainability included high
project staff turnover, very unstable
agricultural production, and restricted
access to Moscow-area markets. The
team observed that some of the
project-built physical structures were
still used by operating companies and
farmers. However, the team did not
find evidence that project-promoted
management and agricultural prac￾tices took hold. Of the project’s two
livestock farms, both deemed major
polluters, one closed during the project
and the other sold off the majority of
its animals. The team concluded that
the project demonstration activities were undertaken
in conditions that did not promise any real likelihood
of sustainability. However, environmental activities fo￾cused on education took root and flourished. Many
of the project-provided educational materials for
schools were still in use; environmental education is
still considered important and worthy of respect.
Local interviews and onsite team inspection revealed
that only two air quality improvement project com￾ponents showed signs of sustainability—and that these
were limited at best. For example, of the AQM sys￾tems introduced in Volgograd, local authorities could
sustain only the few air sampling units that were easi￾est to operate and repair, required minimum mainte￾nance, and had low operating costs. Local authorities
could simply not sustain project-provided continuous
air sampling equipment. The team also observed that
this equipment was not in operation due to a lack of
spare parts, calibration gases, and operator resources.
The team judged that the emissions inventories effort
could be sustainable, provided 1) local authorities re￾view and integrate test data and annual operating
report data into a database and 2) industry truthfully
reports its emissions (experience showed emissions re￾porting was highly variable in accu￾racy and content). The team found
little or no evidence for sustainability
of other project-supported AQM pro￾gram components, including source
assessments, emissions testing (includ￾ing visual emissions testing), human
health risk assessment, and compli￾ance and enforcement. Because the
Russian Government did not officially
accept some U.S. methods and pro￾cedures, the project’s results were not
defensible for compliance demonstra￾tion and enforcement purposes.
Project training functions initially
showed signs of success, but train￾ing revenues were not sufficient to
sustain it.
Replicability
Replicability measures whether assistance provided has
spread to other sites or locations.
Overall, the team found very limited replication of
either the wastewater treatment or watershed devel￾opment projects. Noteworthy exceptions included the
wastewater treatment project’s support for the
Smolensk Environmental Training Center, which went
on to train over 300 people who returned to
nontargeted wastewater treatment facilities. The team
also found evidence that some industrial enterprise
staff were able to move water sampling and monitor￾ing equipment located at one industry to another (Tver
oblast). The project also resulted in equipping and cer￾tifying several nontargeted water quality laboratories.
Of the watershed development project activities, only
those supporting environmental education spread be￾yond Istra. Almost all schools in the Moscow oblast,
for example, now teach environmental education us￾ing project materials. Environmental textbooks are
now in their second edition and are being sold to
schools throughout Russia.
The project demonstration
activities were undertaken
in conditions that did not
promise any real
likelihood of
sustainability. However,
environmental activities
focused on education took
root and flourished.
14
Evidence of air quality improvement project-specific
replication was extremely limited, with the exception
of some boiler fine-tuning and fuel oil emulsification
activities.
LESSONS LEARNED
1. Assess pollution control investments and
incentives carefully.
USAID pollution project mangers need to carefully assess
investments in low-cost/no-cost solutions to industrial
pollution, particularly in situations where facilities and
industries are seriously cash-strapped, and to consider
additional incentives for initiating technology upgrades.
In Russia, project managers emphasized low-cost/no￾cost pollution control methods to improve air and
water quality. In practice, “low-cost” and “no-cost”
are misnomers; there were always out-of-pocket costs
associated with any recommendation. For example,
in resource-strapped and understaffed wastewater treat￾ment facilities, buying even modest water sampling
equipment was not possible, even when such costs were
only a fraction of the operating budget. In-depth dis￾cussions with wastewater treatment facility, steel mill,
and other industry operators revealed that the lack of
even modest capital resources blocked them from
adopting new methods, even when payback times were
short. Faced with mounting operational problems and
industry survival issues, managers were more likely to
use new capital to maintain existing services and pay
personnel.
Low-cost/no-cost solutions may also not prove durable.
Much of Russia’s industrial technology is so outdated
that low-cost/no-cost solutions can only bring tem￾porary and marginal improvement. Interviewees of￾ten commented that only completely new processes
could produce significant environmental improve￾ments. For example, venting and controlling emissions
from Russia’s open hearth steel furnaces would rarely
make them economically or environmentally competi￾tive with today’s electric arc furnaces—simply a little
cleaner. The plants will eventually modernize or close
down; in either event, the earlier USAID low-cost/
no-cost assistance will be lost.
2. Plan for replication from the start.
Pilot and demonstration environmental projects to curb
water and/or air pollution will not be replicated unless
replication is planned for from the outset. The projects
produced little or no replication beyond the three tar￾geted wastewater treatment facilities and key Volgograd
industries. In 1992, USAID chose to work with a very
limited set of pilot and demonstration sites because of
Russia’s enormous environmental challenges, vast area,
unstable regulatory situation, government fragmen￾tation, and project budget constraints. Little time was
spent considering how USAID’s projects would spread
to other wastewater treatment facilities and industrial
enterprises in other cities. Working with pilot sites one￾on-one did not result in replication. When the regu￾latory pressure is either low or nonexistent, other
wastewater treatment facilities and industries will not
feel compelled to adopt new procedures and processes.
Thus, replication plans must be integral to project de￾sign—even at the implementation stage.
Suggestions to improve replicability included placing
stress on the financial benefits. Facility managers, es￾pecially in Volgograd, suggested that providing indus￾try with rate-of-return information could spur interest
in adopting environmental technologies. Simply pre￾senting an industrial enterprise (steel, aluminum, or
brickmaking) with the “facts” did not help the firm
adopt any new technology. Furthermore, in a market
economy (as Russia is becoming), there is no incen￾tive—some would say there is a disincentive—for any
firm to share new technology with others. To over￾come this industrial inertia and achieve replicability,
project planners should carefully and deliberately
present financial benefits, including convincing return￾on-investment information on 1) reduced fees and
fines associated with lowering emission levels, and 2)
increased profits through the recovery of materials that
can be recycled back into the production process.
15
3. Use appropriate—but not necessarily
new or state-of-the-art—methods,
procedures, and technologies.
Project-furnished technology and methods have to reflect
local operating conditions. USAID’s implementers pro￾vided technical advice to improve municipal water
quality operations and industrial air pollution mitiga￾tion. Especially as part of the air quality improvement
project, project designers emphasized getting the “best
pollution equipment.” However, the right equipment
and the right expertise, both necessary for a sustained
impact, did not come together as well
as expected. Why? In-depth interviews
and onsite visits revealed that too little
attention was given to the operating
environment for this equipment.
Understanding and appreciating the
operating environment requires that
project implementers must carefully
examine 1) the availability of trained
staff and other necessary support ser￾vices (including operations, equip￾ment calibration, preventive
maintenance, and repair) and ex￾pendable supplies necessary for con￾tinued operation of the device or
equipment; 2) the physical environ￾ment, including exposure to ele￾ments, pollutants, and abuses inherent to the
application; 3) the regulatory and legal acceptability
of methods and procedures; and 4) the level of preci￾sion, accuracy, and spatial and temporal resolution.
The team found that design and implementation staff
focused too much on state-of-the-art equipment and
not enough on what was needed to make things work
better in Russia.
Volgograd plant interviews revealed that operational
funding beyond the initial capital cost of pollution
prevention equipment was not seriously considered.
Activity designers must ask whether there is enough
funding to obtain initial governmental certification,
cover testing costs, and ensure a product market if rep￾lication is desired.
4. Consider both short- and long-term
tradeoffs.
USAID must be clear whether it seeks shorter term re￾sults, with more immediate and visible outcomes, or longer
term results, with sustainable impacts and more durable
institutional partners. The umbrella environmental pro￾gram was officially launched at a time when the U.S.
Government was moving swiftly to be among Russia’s
first supporters. There were political
and financial reasons to promote early
project successes, and work with se￾lected wastewater treatment facilities
and polluting industries was highly
visible. USAID relied heavily on
USEPA technical expertise to take the
lead because of its demonstrated ex￾pertise and well-established Russian
contacts. Team interviews revealed
that USEPA-led activities started up
later than planned, but USEPA estab￾lished an early presence at key sites.
More lead time for project startup and
host-country needs analysis would
have improved communication be￾tween the U.S. and Russian partners
and otherwise allowed for a more sustainable effort.
Better planning might have produced more widespread
and sustained impacts, but at the expense of higher
U.S. visibility early on and less spending. Durable
partnerships and well-functioning environmental in￾stitutions take time to coalesce and mature to fully
operational status, especially in transition countries
like Russia, where environmental institutions are both
understaffed and underfunded.
Interviews with water and air quality officials con￾firmed the need for onsite engagement, not occasional
visits with concentrated bursts of activities. Institu￾tion building and strengthening require a slow, steady
USAID must be clear
whether it seeks shorter
term results, with more
immediate and visible
outcomes, or longer term
results, with sustainable
impacts and more durable
institutional partners.
16
This Impact Evaluation summarizes a Working Paper prepared by Steve Gale, of CDIE, with team members Matthew Addison,
Mark Hodges, and Nick Wedeman. This Impact Evaluation and the Working Paper (PN-ACM-154) can be ordered from USAID’s
Development Experience Clearinghouse (DEC). To download or order publications, go to www.dec.org and enter the document
identification number in the Search box. The DEC may also be contacted at 1611 North Kent Street, Suite 200, Arlington, VA
22209; telephone 703-351-4006; fax 703-351-4039; e-mail docorder@dec.cdie.org. Steve Gale prepared this Impact Evaluation,
and IBI (International Business Initiatives, Inc.) furnished editorial and production assistance.
effort and long-term, onsite presence. USEPA was
often not able to provide this kind of support. Modi￾fying institutions and changing people’s behaviors re￾quire dealing with problems and overcoming obstacles
before or as they arise, and learning through observa￾tion—which may not take place only when advisors
just happen to be in country. Finally, some interviewees
suggested that the typical kind of leveraging that USAID
works hard to secure was partially undermined by the
lack of a sustained on-the-ground presence in Russia.
5. Collect appropriate data to assess project
results.
Collection of baseline data and other appropriate infor￾mation is needed to assess project impacts. Baseline and
other data were not systematically gathered to permit
an unambiguous assessment of the effects of the vari￾ous upgrades at the three wastewater treatment facili￾ties or key industrial enterprises. It was not possible to
fully disentangle the effect of facility improvements
from the impact of the economic downturn that led
to decreased industrial waste streams, or from factors
that would have produced similar outcomes. Similarly
in Volgograd, sufficient data was not collected to ac￾curately assess changes in pollution emissions, nor were
other standardized measures in place to permit assess￾ment of progress in implementing a model air quality
improvement project. Lack of good baseline data and
the inability to link facility and industry upgrades with
specific project inputs undermined USAID’s ability
to conduct accurate project appraisals.