External End-of-Project Evaluation: 
Health Center Renovation Coordination Project
Final Report 
August 2008 
This publication was produced for review by the United States Agency for 
International Development/Ethiopia. It was prepared by The Mitchell Group under 
Contract No. 663-C-00-08-00409-00 
 
This document was submitted by The Mitchell Group, Inc. 
1816 11TH Street, NW 
Washington, DC 20001, USA 
Telephone: (202) 745-1919 
Facsimile: (202) 234-1697 
To 
The United States Agency for International Development/Ethiopia 
 under 
USAID Contract 663-C-00-08-00409-00 
This is an evaluation report carried out by team members: 
Team leader: Barney Mosby 
Team member: Thomas Scialfa 
August 1, 2008 
Disclaimer: 
The authors’ views expressed in this publication do not necessarily reflect the views 
of the U.S. Agency for International Development or the United States Government 
 
TABLE OF CONTENTS 
CONTENTS 
ACKNOWLEDGEMENTS................................................................................................ 5 
LIST OF ACRONYMS ...................................................................................................... 6 
PROJECT SUMMARY...................................................................................................... 7 
EXECUTIVE SUMMARY ................................................................................................ 8 
Overview ......................................................................................................................... 8 
Key Findings ................................................................................................................... 9 
General......................................................................................................................... 9 
Result 1: Coordination................................................................................................. 9 
Result 2: Technical Assistance.................................................................................. 10 
Result 3: Renovation ................................................................................................. 10 
Recommendations ......................................................................................................... 11 
I. INTRODUCTION......................................................................................................... 12 
II. PROBLEM STATEMENT .......................................................................................... 13 
III. HCRCP OBJECTIVES AND IMPLEMENTATION PLAN..................................... 14 
IV. EVALUATION DESIGN AND METHODOLOGY................................................. 16 
Survey design ................................................................................................................ 16
Data collection tools and techniques............................................................................. 16 
Sampling........................................................................................................................ 16 
Analysis......................................................................................................................... 17 
Expected and unexpected deliverables and work plan progress................................ 17 
Site visits.................................................................................................................... 17 
Federal and Regional level key informant interviews ............................................... 17 
Document review....................................................................................................... 17 
Limitations................................................................................................................. 18 
Analysis of the health-care component of renovation............................................... 18 
Composition of the team............................................................................................ 18 
Document review .......................................................................................................... 18 
V. FINDINGS AND CONCLUSIONS ............................................................................ 19 
Result 1: Coordination and donor leveraging ............................................................... 19 
Result 2: Providing technical assistance ....................................................................... 20 
Result 3: Direct renovation of health centers ................................................................ 21 
VI. QUESTIONS POSED TO THE EVALUATION TEAM AND ANSWERS ............ 23 
Program Management ................................................................................................... 23 
Implementation.............................................................................................................. 24 
 
Monitoring and Evaluation (M&E)............................................................................... 24 
Quality Assurance ......................................................................................................... 25 
Capacity Building and Sustainability............................................................................ 25 
VII. RECOMMENDATIONS AND LESSONS LEARNED........................................... 27 
Coordination.................................................................................................................. 27 
Technical assistance ...................................................................................................... 27 
Management.................................................................................................................. 28 
Renovations................................................................................................................... 28 
Monitoring and evaluation ............................................................................................ 29 
VIII. APPENDICES ......................................................................................................... 31 
APPENDIX 1) SOW for the evaluation........................................................................ 31 
APPENDIX 2) TMG evaluation work plan .................................................................. 48 
APPENDIX 3) Topical outline for the small group discussions................................... 49 
APPENDIX 4) Photos from evaluation team site visits................................................ 53 
APPENDIX 5) List of people met and interviewed...................................................... 69 
APPENDIX 6) Description of field guide assessment document................................. 71 
APPENDIX 7) Levels of renovations with respect to health care ................................ 75 
APPENDIX 8) Summary of key project activities with status notes............................ 76 
APPENDIX 9) Table of key project activities with status and notes............................ 79 
 APPENDIX 10) Summary of the results from site visits ............................................. 85 
 
ACKNOWLEDGEMENTS 
First and foremost we thank Dr. Michael Dejene’s team with a special thanks to Ms. 
Azeb Asmamawaw who handled all the logistics for the team. 
The assistance of Sam Clark as our liaison officer was courteous and appreciated. With 
his help, the team was able to meet with other key individuals involved in health sector 
construction. Special thanks also goes out to Marie-Claire Sow and Michael Rossman of 
the USAID Contracting Office for their help in obtaining key documents needed to carry 
out the evaluation. 
Last but not least we are deeply indebted to the staff at the Federal Ministry of Health 
(FMOH) and the Regional Health Bureaus (RHBs) for their interest, support and 
contributions. We particularly appreciate the help of Rik Nagelkerke and Alemayehu of 
the Program and Planning Division (PPD). 
 
LIST OF ACRONYMS 
ART Antiretroviral Therapy 
A&E Architecture and Engineering 
CAC Crown Agents Consulting 
CDC Centers for Disease Control 
CTO Cognizant Technical Officer (USAID) 
FGOE Federal Government of Ethiopia 
FMOH Federal Ministry of Health 
FHI Family Health International (NGO) 
GAA German Agro Action (NGO) 
GOE Government of Ethiopia 
HC Health Center 
HCRCP Health Center Renovation Coordination Project 
HH Household 
IP Implementing Partner 
IQC Indefinite Quantity Contract 
KI Key Informant 
M&E Monitoring & Evaluation 
MOH Ministry of Health 
MOU Memorandum of Understanding 
MSH Management Sciences for Health 
NGO Non Governmental Organization 
OJT On the job training 
PEPFAR President’s Emergency Plan for AIDS Relief 
PLHIV People Living With HIV/AIDS 
PMP Performance Monitoring Plan 
PPD Program and Planning Division 
PY Project Year 
RHB Regional Health Bureau 
SGD Small Group Discussion 
SNNPR Southern Nations, Nationalities, Peoples Region 
SOW Scope of Work 
STTA Short-Term Technical Assistance 
TA Technical Assistance 
U.S. United States 
USAID United States Agency for International Development 
USAID/E United States Agency for International Development/Ethiopia 
USD United States Dollar 
 
PROJECT SUMMARY 
Project Title & Objective: 
Title: Health Center Renovation Coordination Project (HCRCP) 
Objective: The three main expected project results were: 
• Coordination - Serve as a coordination and synchronization “hub” with national and 
bilateral stakeholders on health center renovations to support chronic disease; 
• Technical Assistance Assessment - provide assistance on renovations to the Federal 
Ministry of Health (FMOH) and Regional Health Bureaus (RHBs), including technical 
design, engineering, procurement and logistics support (100 Health Centers [HCs] in 
total); and 
• Renovation - Provide direct or indirect renovation support at selected health centers to 
support chronic disease management including antiretroviral therapy (ART) (50 HCs in 
total). 
Project Number: 663-C-00-06-00428 
Life of Project: 2 Years 
Implementing Agency: Crown Agents USA, Inc. 
Contract Number: 663-C-00-06-00428-00 
Project Funding: $3,919,940 
8 
EXECUTIVE SUMMARY 
OVERVIEW 
In September 2006, USAID/Ethiopia awarded Crown Agents Consulting USA (CAC) a 
contract to coordinate health center (HC) renovations in four President’s Emergency Plan 
for AIDS Relief (PEPFAR) priority regions (Amhara, Oromia, Southern Nations, 
Nationalities, and Peoples [SNNPR] and Tigray) and Addis Ababa in Ethiopia. 
 
The activity was to provide technical assistance to the FMOH and RHBs for renovation 
coordination, technical design, engineering, procurement and logistics support. Though 
not planned, the project also placed short-term technical assistance at the national and 
regional level to support staffing gaps in support of the Government of Ethiopia (GOE) 
health facility renovation/construction. The HCRCP was also to assess and renovate 
selected health centers directly, so that the facilities can better support chronic disease 
management and HIV/AIDS services, especially antiretroviral therapy (ART). CAC was 
to undertake these efforts by contracting with local Architecture and Engineering (A&E) 
and construction companies. The companies were to assist CAC in the assessment, design 
and renovation of the health centers. CAC was also to coordinate the activities of 
numerous organizations funded by several different international donors. 
USAID requested an end-of-project evaluation to assess the achievements, challenges 
and sustainability of the program to inform follow-on programming. The evaluation also 
responded to USAID’s questions that focused on: 
 
• Program management effectiveness; 
• Implementation: the approach, effectiveness and quality of assistance; 
• Monitoring and evaluation: indicators and the use of monitoring and evaluation 
(M&E) for planning and management; 
• Quality assurance: standards and criteria used for renovations; and 
• Capacity building and sustainability: strategies, priorities and lessons learned: 
guidance for implementation. 
The evaluation also provided answers to the illustrative questions that USAID included in 
the evaluation Statement of Work (SOW) Appendix 1. 
The evaluation methodology was observational and qualitative with some quantification. 
It took into consideration both the engineering and health-care aspects of the renovations 
and incorporated elements of a performance audit. This hybrid methodology was required 
to fulfill the objectives and answer the broad range of illustrative questions USAID had 
outlined in the Statement of Work (SOW) for this evaluation. The evaluation included 13 
site visits in four of the project’s five geographic areas, key informant interviews, small 
group discussions and comprehensive document review. The sampling for site selection 
was based on several variables, including: 1) location (north, south, Addis); 2) status of 
renovation (in-progress and completed); 3) presence of other donors or projects; 4) 
presence of GOE or other partner contribution (leveraged or not); and 5) convenience. 
USAID requested that final site selection be made with CAC assistance. 
Specifically, the evaluation focused on the following tasks: 
9 
• Assessing the progress toward contract targets; 
• Assessing the process used to implement the program’s three key areas of 
intervention to meet objectives; 
• Assessing the effectiveness of the program’s three key areas of intervention; 
• Identifying additional achievements and unexpected benefits; and 
• Identifying project components that could be scaled up beyond the project agreement 
in Ethiopia. 
KEY FINDINGS 
General 
• CAC consistently identified challenges, such as inadequate financial resources to 
renovate the 50 targeted health centers and still assure a reasonable level of quality; a 
lack of interest in this project by Federal Government of Ethiopia (FGOE); and a lack 
of available GOE engineering counterparts. 
• The first year work plan was not approved by USAID until nearly the end of the first 
year, even though CAC had submitted the original work plan and its revisions to 
USAID on time; the second year work plan had still not approved at the time of the 
evaluation, more than six months into the second (last) year of the project. 
• These and other constraints hampered implementation. Some of these challenges 
were addressed appropriately and others inappropriately by both CAC and USAID. 
Examples of the constraints are included in the report. 
Result 1: Coordination 
• The environment in which a health center renovation project was conceived changed 
between project design in FY06 and project implementation in FY07/08. By the time 
project implementation began, there were very few NGOs involved in health center 
renovations, so there was little to “coordinate.” Thus, Result One (i.e., to establish an 
HCR coordination mechanism) could have been replaced and the coordination 
component of the project could have been simply an activity under Results Two or 
Three. 
• CAC was to have developed a health center (HC) renovation tracking system as part 
of Result One. If done well, this could have been useful to the GOE for both 
renovations and new construction. Unfortunately, the tracking system created by 
CAC was not a “system” and did not track HC renovations. It was a poorly 
labeled/documented spreadsheet with some charts that predominantly had information 
about the type of services delivered by implementers rather than information about 
the status of HC renovations. 
• The HCRCP was not implemented in partnership with national government. The 
evaluation team assessed that this was due to a number of factors. Because there was 
a major new GOE health care expansion effort requiring construction of new health 
centers, there was limited buy-in to the HCRCP project as designed. USAID’s 
management support/response to this changed environment was slow -- slow to 
initiate dialogue with the FGOE and slow to adapt the project to respond to the 
current environment. 
10 
Result 2: Technical Assistance 
• CAC developed excellent tools and guidance for health center assessments and 
renovations and there is a high probability that these will be adopted by the GOE. 
These take into account not only the engineering aspects of renovation but also 
address the health care aspects of renovations comprehensively. 
• CAC contracted the minimum required single local architecture and engineering 
(A&E) firm to undertake HC assessments, renovation design and renovation 
supervision. Temporary staff hired by the A&E firm were well trained in conducting 
assessments. However, by the time that this evaluation was conducted, CAC had 
stopped using the A&E firm and was doing the assessments in-house. CAC stated that 
this was because the A&E firm was not performing as expected regarding submission 
of their paperwork. However, the A&E firm stated that this was not the case. 
 
• The environment made it difficult to provide technical assistance (TA), mentor and 
provide on-the-job training to GOE staff of the Federal Ministry of Health (FMOH), 
Regional Health Bureaus (RHBs), zonal or district level. Nonetheless, no evidence 
was found that either USAID or CAC explored opportunities to reprogram the 
budgeted resources with alternative capacity building mechanisms. 
• The model developed by CAC for tendering and contracting the HC renovations was 
efficient and resulted in a reduced time between tender and start of renovations when 
compared to the model used by the GOE. During the project period, the procedures 
used by the GOE were also modified, independent of inputs from the HCRCP, such 
that there is no longer a significant difference between the FGOE procedures and 
CAC model. 
Result 3: Renovation 
• As stipulated in their contract, CAC had introduced a “health” aspect to the 
renovations, which addressed issues of client comfort (waiting areas, privacy), 
workflow for increased efficiency, and decreased risk of exposure to infections, as 
part of the assessment/design processes. 
• The funds provided for HC renovations were not adequate to achieve acceptable 
standards for the renovation of the 50 targeted HC renovations. 
• Some of the fixtures and materials used in the renovations were not of acceptable 
quality. In most of the sites visited, the evaluation team observed problems related to 
non-functional water supplies, leaking roofing and guttering, and doors that were not 
hung correctly, although the framing and foundations were of good quality. 
• The number and quality of supervision visits of the renovations were inadequate. 
Even though several sites had been signed-off and reported as being “completed,” 
there were incomplete renovations or long standing problems that had not been 
addressed, or were sometimes not known by CAC/USAID. This was confirmed 
during the evaluation team’s review of the supervision records, and shown in 
photographs taken by the evaluation team. 
 
11 
• Though the HCRCP did not have a stated objective to improve maintenance at the 
renovated sites, CAC did identify this as a serious weakness jeopardizing the 
longevity and sustainability of the project. They made a nominal effort to incorporate 
improved maintenance into their activities. Given the potentially available budget if 
changes were made in Result 1, resources could have been reallocated to increase 
maintenance-related activities. 
 
RECOMMENDATIONS 
• In future projects, additional financial resources should be allocated to assure a 
standard of quality for the major components of the project, especially technical 
assistance and renovation. There should be sufficient resources to renovate each HC 
selected so that a defined standard is assured. 
• Strong, consistent management oversight by USAID and the contractor should be 
provided for successful planning, implementation and monitoring of the project. Clear 
guidance and the development and monitoring of timely work plans should be used to 
strengthen the project’s management. 
• Contracting for assessments and renovations should be conducted by the government, 
building their capacity to manage larger projects for assessment and renovation. To 
increase the ability of the technical assistance contractor to improve the ability of the 
RHB staff, it is suggested that the TA staff be stationed at the RHBs. 
• The two high quality handbooks for health center assessment and health center 
renovation produced by this project should be diffused widely and promoted as a 
resource for the GOE, other organizations, and other donors. 
• It is suggested that strong technical assistance and a significant training program be a 
part of new projects. Various alternatives should be used to increase the capacity of 
RHBs and other organizations. In addition to training RHB staff in HC assessments 
and renovation, a comprehensive training program should be developed for 
maintenance. 
• Establish a minimum renovation package for each site during project design to help 
establish an appropriate budget and implementation plan. Increase the threshold for 
what is considered a “minimum package” renovation - after operationally defining a 
minimum package (e.g., include in all cases covered waiting areas with benches or 
seats) 
• Renovation materials should be of commercial quality and supervision of renovations 
should be adequate to monitor and assure the quality. 
• Finally, contract administration functions and overall control of the assessments 
should be located in the RHB. 
12 
I. INTRODUCTION 
The purpose of this evaluation report is to provide information to allow USAID/Ethiopia 
to assess the USAID strategic objectives and program results of the Health Center 
Renovations Coordination Project. The SOW for the evaluation included: 
• Assessing the progress toward contract targets. 
• Assessing the process used to implement the program’s three key areas of 
intervention to meet objectives. 
• Assessing the effectiveness of the key project interventions: the approach and 
activities used by Crown Agents to assist the GOE to improve health center 
renovation planning, design and operations for HIV/AIDS and other chronic diseases 
among various institutions, and other cooperating agencies, other donors, HAPCO, 
the Ministry of Health and other Ministries. 
• Identifying additional achievements and unexpected benefits. 
• Identifying project components that could be scaled up beyond the project agreement 
in Ethiopia. 
The workplan for the Evaluation team is included in Appendix 2. 
There are nine regions and two chartered cities in Ethiopia. Addis Ababa and Dire Dawa 
are the two chartered cities. HCRCP is located in the city of Addis Ababa and the regions 
of Amhara, Oromia, Tigray and SNNPR. 
1. Addis Ababa 
2. Afar 
3. Amhara 
4. Benishangul-Gumuz 
5. Dire Dawa 
6. Gambela 
7. Harari 
8. Oromia 
9. Somali 
10. Southern Nations, Nationalities, and People's Region
11. Tigray 
13 
II. PROBLEM STATEMENT 
The number of HIV positive individuals in Ethiopia was estimated at 977,000 in 2007 
and is projected to increase substantially in the next two years (a 30% increase was 
projected from 2006 to 2010). The projected growth in the total size of the HIV-infected 
population in Ethiopia means that efforts to strengthen facility-based services will be 
critical for a successful national HIV/AIDS program. The Government of Ethiopia (GOE) 
Federal Ministry of Health (FMOH) is decentralizing ART and other HIV/AIDS services 
to the health center level. HIV/AIDS services at health centers require adequate 
infrastructure to accommodate the rapid and sizable increase in HIV/AIDS clients at 
these sites. 
The health manpower capacity to engineer, plan and monitor the expansion and 
renovation of facilities is limited. Following the Paris Declaration, the GOE adopted a 
plan to achieve universal access to primary health care, including HIV services, by 2010. 
Under this plan, the MOH has a goal to increase the number of health centers in the 
country from around 630 in 2007 to 3,153 by the year 2010. This ambitious undertaking 
to expand the health infrastructure means that the engineering, construction planning and 
monitoring capacity of the federal and regional health administrations are stretched to the 
limit. The expansion of PEPFAR caused even more pressure on health facility 
infrastructure. A recent assessment identified infrastructure deficiencies as a major 
obstacle impeding sustained progress in achieving ART and PMTCT expansion targets. 
The poor condition of health facility infrastructure also affects the public’s willingness to 
access services. 
When the HCRCP was originally designed, several US Government-funded partners in 
Ethiopia were engaged in health center renovation. By the time that the HCRCP began, 
this situation had changed and only one other organization, Management Sciences for 
Health (MSH), was working on health center renovation. Given the identified 
deficiencies in health centers and the limited number of other donors and implementing 
agencies renovating health centers, the need was greater than initially assumed by 
USAID/Ethiopia (USAID/E). 
HCRCP design included the development of a systematic and structured approach for the 
FMOH/PPD so that it could coordinate its own activities as well as other agencies 
implementing and/or supporting HC renovation and construction. One of the three main 
project results was: “Coordination - Serve as a coordination and synchronization “hub” 
with national and bilateral stakeholders on health center renovations to support chronic 
disease.” 
14 
III. HCRCP OBJECTIVES AND IMPLEMENTATION PLAN 
As per the Country Operational Plan (COP07), HCRCP was to support the GOE, 
coordinate the various renovation and construction initiatives for improved HIV/AIDS 
services in Ethiopia, and carry out selected renovations – filling gaps when other funds 
are unavailable. This activity links to care and support, antiretroviral drugs, ART and 
laboratory services. A Family Health International (FHI) 2006 assessment identified 
infrastructure deficiencies as a major obstacle to reaching ART targets, thereby 
addressing ART service expansion by increasing health centers’ operational capacity to 
manage ART services, including integration into the health network. HCRCP was to 
alleviate some of these infrastructure constraints. 
HCRCP’s three objectives, listed as expected results, are described below. 
Result 1: Serve as a coordination and synchronization "hub" with national and bilateral 
stakeholders on health center renovations to support chronic disease management and to 
leverage United States Government resources with the Government of Ethiopia and 
bilateral agency resources. 
Result 1 was to be achieved by creating a resource hub. As per CAC’s proposal and the 
contract, this hub was to have the following key elements: a) a coordination steering 
committee; b) an HC renovation tracking/mapping system; and c) coordination 
workshops. In some documents, there was also mention of setting up a tender committee. 
It must be noted that although the original proposal and contract stipulated the 
establishment of a “coordination/steering committee,” the approved work plan replaced 
this with development of a “coordination plan.” 
 
Result 2: Provide technical assistance on renovations to the Federal Ministry of Health 
Regional Health Bureaus including technical design, engineering, procurement and 
logistics support. Specifically, 100 HC engineering assessments were to be conducted. 
Result 2 was to be achieved by either: a) adapting available; or b) developing new GOE 
HC renovation guidelines and procedures, including tendering and awarding contracts. 
These guidelines and procedures would then be tested and finalized. The result would be 
useful tools that the FMOH and RHBs would adopt and use. 
HCRCP’s technical team was also to identify RHB counterparts and build their capacity 
in: 1) conducting engineering assessments; 2) supervising HC renovation works; and 3) 
using the HC assessments, renovation guidelines and other tools. 
Result 3: Provide direct renovation support in 50 selected health centers to support 
chronic disease management including antiretroviral therapy. 
Result 3 was to be achieved by the project’s technical team identifying and training at 
least one local A&E firm to conduct these assessments and then to renovate selected sites 
using these renovations as an opportunity to build capacity of GOE.
HCRCP’s health facility renovations were implemented in five regions/cities of Ethiopia 
which are among PEPFAR/Ethiopia’s priority geographic areas: These are Amhara, 
15 
Oromia, SNNP, Tigray and Addis Ababa. Within these areas, the HCRCP engineering 
assessments and renovations were to be done at PEPFAR priority sites. 
Activities under Result 1 were to be done largely by the FMOH; Crown Agents 
Consultants was to provide TA and other support. Those activities related to USG 
renovating agency coordination was to be done directly by CAC, collaborating with the 
FMOH. In other activities (under Results 2 and 3), CAC was to lead joint activities 
working with GOE technicians, NGOs and local A&E firms. Still other activities were to 
be conducted by the contracted A&E firm, after they had been trained. The total 
estimated cost of this contract was 3,919,940 United States Dollar (USD) for two years. 
16 
IV. EVALUATION DESIGN AND METHODOLOGY 
SURVEY DESIGN 
The evaluation methodology was observational and qualitative with some quantification. 
It took into consideration both the engineering and health-care aspects of the renovations 
and incorporated elements of a performance audit. This hybrid methodology was required 
to fulfill the objectives and answer the broad range of illustrative questions USAID had 
outlined in the SOW for this evaluation. 
The survey design is a post-test without a baseline. Although the HC assessments 
documented the HC architectural elements prior to renovations, no baseline 
measurements of provider behavior, work flow, capacity building, etc was made. The 
team used historical recall during interviews and small group discussions to estimate a 
pre-test value for some constructs and used stakeholder perception of future/potential 
impact to estimate the post-test measure for sites that were not yet renovated. For the 
architectural and engineering components, the team provides photos of health centers as 
they existed at the time of the evaluation. 
DATA COLLECTION TOOLS & TECHNIQUES 
Document reviews, observations, key informant interviews and small group discussions 
(SGD) with a mix of health center employees were used. HC clients were not interviewed 
because of time limitations. The topical outline for small group discussions is presented 
in Appendix 3. Photos and diagrams are presented in Appendix 4. 
SAMPLING 
Given the time allowed and the scope of the evaluation, the sample size of HCs to be 
evaluated was limited to 13, including the 2 pre-test sites in Addis Ababa. With this 
particular qualitative research design and limited sample size, the appropriate sampling 
method was one that maximizes the variance. To achieve this, we selected using the 
following criteria: 
• Location (north, south, Addis); 
• Status of renovation (in-progress, completed); 
• Presence of other donor or project (CAC only vs CAC+ other {e.g. other = RPM+); 
• Presence of GOE or other partner contribution (leveraged or not); and 
• Convenience. 
Based on these criteria, and at the request of USAID/Ethiopia, with the participation of 
Crown Agents in selecting the sites, the following sites were visited: 
• Addis Ababa: 1) Shromeda, 2) Kotebe, 3) Bole Woreda 17 
• South: 4) Yergalem (SNNPR) 5) Shashemene (Oromia), 6) Batu (Oromia), 7) Modjo 
(Oromia) 
• North: 8) Adet (Amhara), 9) Bahir Dar (Amhara), 10) Dangla (Amhara), 11) 
Koladiba (Amhara), 12) Addis Zeman (Amhara), 13) Injibura (Amhara) 
17 
For the key informant interviews, the team planned to meet the HC Director who had 
signed the MOU with CAC, and representatives from FMOH, RHB, USAID, Centers for 
Disease Control (CDC), CAC, and other implementers. Please see Appendix 5 for a list 
of people interviewed. The team planned to conduct SGDs in all of the HCs visited, but 
logistic and facility staff workload limited access to staff during the team’s visits. 
 
ANALYSIS 
Seven summary tools were created to help analyze and interpret the data collected, as 
described below. 
Expected and unexpected deliverables and work plan progress 
It is important to know what was expected of the contractor (Crown Agents Consulting) 
in terms of services and deliverables under the contract. Because many factors can 
influence what can be achieved, the approved work plans are usually used for this 
purpose. In addition, to understand the environment, if it has changed and if the response 
was appropriate, the quarterly and semi-annual reports were reviewed. During the review, 
the reliability and validity of reported results were assessed. Lastly, as with any project, 
there are often unexpected services and deliverables; these were also assessed The team 
prepared a summary table and an expanded table showing the project services and 
deliverables as defined in different project documents (winning proposal, contact, 
approved work plans and progress reports) to help in the analysis. The summary table is 
presented in Appendix 8. The expanded table is presented in Appendix 9. 
Site visits 
To analyze the results of our site visits, we created two matrices. The first summarized 
the “health care” component of the visit and the second summarized the “engineering” 
component of the visit. Interpreters were used, and they took notes. Each site visit was 
made by a four-person team: the team leader, team member, and interpreters. We met as a 
team; each person reviewing their notes and looking at the photos taken to complete the 
elements of the matrices with information from SGDs, Key Informant (KI) interviews 
and observation. The team then combined the elements of the matrices into a single table 
summarizing the results across all sites. The summary results of the site visits are 
included in Appendix 10. 
Federal and Regional level key informant interviews 
At the federal and regional levels, the interviews were relatively few and did not require a 
matrix to analyze and interpret the results. The team reviewed interview notes subject by 
subject, and discussed common themes from the interview information. 
Document review 
Many documents were reviewed. The four general categories of documents include: 1) 
contracts/work plans; 2) scheduled and interim/ad hoc progress reports; 3) technical 
deliverables; and 4) A&E designs and documents. For the latter, the team created a 
matrix to summarize information relative to the renovation process (i.e., dates) for the 13 
sites visited, but was not able to get the all of the information in the time allotted. 
Therefore, this was not included in the analysis. 
18 
Limitations 
Time to complete this evaluation was the most limiting factor. The sample of sites visited 
captured variations adequately. The evaluation team judges that the engineering 
component of the evaluation (i.e., observation) is a good representation of the entire 
project. The team was not, however, able to have as many SGDs as intended, which 
means the information we obtained on the health impact of the project is more limited. 
 
Analysis of the health-care component of renovation 
The expected impact of renovations on health care service delivery was analyzed by 
using the descriptions in the PMP, contract and approved work plan. To aid in the data 
collection and analysis phases, the evaluation team developed a classification of three 
hierarchical levels of HC renovation related to health care. A thorough document review 
to determine which level was expected of the contractor was done. Level 3 renovations 
included renovations to the HC site’s original design and functionality plus 
reorganization and/or modifications to the architecture design for improved health care 
delivery. The CAC contract stated that level 3 renovations were expected. These were 
stated in the first 2 of 17 bullets under C.4. Illustrative List of Renovations, which 
include: 
• Add or demolish wall partitions whenever necessary to ensure proper functions and 
facilitate patient flow. Install a patient reception counter at the central location of each 
clinic to facilitate patient flow. 
• Covering areas between multiple structures to improve client comfort and client flow 
to create continuous space for patient counseling rooms, waiting areas, administrative 
offices or secure storage. 
Appendix 7 presents a full description of the three levels of HC renovation. 
 
Composition of the team 
Team Leader, Barney Mosley; Team Member, Thomas Scialfa; Team Advisor and 
Coordinator, Michael Dejene; Logistics Coordinator, Azeb Asmamawaw; and 
interpreters. 
DOCUMENT REVIEW 
The evaluation team found significant differences between the expected 
services/deliverables as stated in the different HCRCP documents being: 1) the approved 
PMP 2) CAC Contract 3) the PY1 Work Plan. The results of the document review with 
respect to project implementation are summarized below. Detailed information is 
provided in Appendix 8. 
19 
V. FINDINGS AND CONCLUSIONS 
RESULT 1: COORDINATION AND DONOR LEVERAGING 
The environment for HC renovations had changed from project inception, and during 
early implementation. This change indicated the need for adaptation of the first and third 
key activities/products under Result 1. The HCRCP was designed at a time when there 
were many more implementers doing various levels of HC renovations and these needed 
coordination. By the time that the HCRCP began, Result One was no longer appropriate 
as it was written, with two exceptions: developing a mapping/tracking system which the 
GOE obviously needs, especially given their new HC scale-up; and USG implementer 
coordination for those few involved in HC renovation. 
The significant change in the environment was identified early in the project’s 
implementation through a survey that CAC initiated. This survey showed that 
respondents (USG, other donor and FMOH) were not doing renovations and not 
interested in being part of a national coordination committee/hub. The activity of creating 
a coordination committee/hub changed to establishing a coordination plan, which had not 
been completed at the time of the evaluation. 
 
Although the environment had significantly changed, USAID did not negotiate the 
HCRCP with the FMOH prior to implementation. It was approximately eight months into 
implementation when USAID completed negotiations and obtained some level of FMOH 
support for the HCRCP. However, USAID did attempt to set up meetings with the head 
of FMOH/PPD numerous times without success, and instructed CAC to proceed with 
Results Two and Three until they could reach an agreement with the FMOH to include 
the renovation of existing HCs in their focus to build new ones. 
Moving forward with activities under Results Two and Three was done with project 
activities by-passing the FMOH and going directly to the RHBs (without FMOH 
knowledge) to begin technical assistance and direct renovations. Under Ethiopia’s 
decentralization initiative, the Regional Health Bureaus (RHBs) have the authority to 
make decisions without the FMOH approval. However, this approach, at least partially, 
undermined the project’s ability to support and provide TA to the FMOH. In the team’s 
discussions with CAC, they stated that, when the time came to begin work in Addis 
Ababa and they needed FMOH involvement, FMOH became quite angry to learn that 
activities had already been done in the regions. Project delays in getting FMOH support 
for the HCRCP contributed to CAC having significant barriers to achieving activities 
under Result One. 
 
Although considerable budgetary resources were available to achieve Result One and 
limited coordination needs were evident, project priority was not shifted to develop other 
activities under Result One, such as development of the tracking/mapping system. The 
tracking/mapping system that was later developed and delivered was reviewed and 
assessed to be inadequate. It consisted of a minimal number of MS Excel spreadsheets 
with unlabeled charts. Further, the charts were unrelated to tracking HC renovations and 
contained information about services delivered at the sites. This may explain the 
reluctance of USG agencies, FMOH, RHB and other donors to adopt and maintain the 
“tracking system.” 
20 
The third key activity/product under Result One was to help FMOH and RHBs organize 
and hold coordination meetings. Given decreased need for coordination and the decreased 
number of donors and activities, this key activity under Result 1 could probably have 
been replaced with another activity or project product. There were, however, several 
instances when CAC effectively coordinated with other implementers and directly with 
HCs so that renovations being financed by implementers were done harmoniously. There 
were also a few times when CAC was able to successfully leverage additional resources 
for HC renovations. Due to the changed environment for coordination, CAC coordinated 
activities to the extent possible. 
In summary, the HCRCP did not meet the expectations for Result One, as written. 
However, the team does not feel that these were realistic expectations (except for the 
tracking system) because of the change in the environment. The team does feel, however, 
that the HCRCP requirements should have been modified. 
RESULT 2: PROVIDING TECHNICAL ASSISTANCE 
Technical assistance was provided primarily through developing a new way to conduct 
A&E assessments, a new model for tendering bids for renovations, and capacity building 
through mentoring, on-the-job-training (OJT), other training, conducting 
assessments/proposing redesign, and the technical review of related documents. 
The environment in which CAC’s TA was to be delivered was not conducive to regular 
and significant TA to the GOE (federal, regional and district levels) via mentoring and 
OJT. This situation was compounded by the incredible amount of construction going on 
in Ethiopia; not just in the health sector but across all sectors. This difficulty was 
identified within the first several months of project implementation. However, changes to 
the HCRCP strategy were not proposed. 
To build the capacity of A&E firms, mentoring and OJT was to be provided to one or 
more A&E firms. CAC did provide mentoring and OJT to one A&E firm. Unfortunately, 
that A&E firm chose outside consultants to receive OJT and mentoring. This meant that 
the capacity might not be available for future work. In addition, CAC is no longer using 
the single A&E firm where they provided TA and mentoring. CAC is now doing the 
assessments with their internal staff. 
The team’s review of project TA to the GOE found that few TA, mentoring or training 
activities were carried out by CAC to the GOE. Providing TA to the GOE in an 
unsupportive environment was admittedly difficult. However, neither CAC nor USAID 
actively explored alternatives to achieving some type of capacity building at the GOE. 
The team found little or no evidence that the project provided TA to the contractors’ staff 
doing the renovations. This TA could have enhanced the quality of the renovations and 
safety at construction sites, including the removal of debris and other unused materials to 
prevent accidents. 
The team’s review of the Health Center field guide found that TA provided for both the 
health care service delivery and engineering components resulted in products that are of 
excellent quality. The Guide is in its final draft form. Given its quality and potential 
usefulness, there is good probability it will be adopted and used by the GOE. 
21 
The Guidelines for HC renovation are also comprehensive and of excellent quality. They 
are in “final draft” form. There is good probability it also will be adopted after final 
revision by the GOE. 
Project TA also resulted in a model for tendering by prequalification. It was developed, 
tested and used by CAC under the project, and is of good quality. The team found, 
however, that the GOE had already made changes to their own tendering approach, and 
now use their model for prequalification independent of the HCRCP project. Thus, the 
GOE already has a new model similar to the one that the HCRCP tested and used. There 
might still be a chance that some of the forms and procedures that CAC developed under 
the HCRCP will be helpful to the GOE. 
It is estimated that at least half of the TA that CAC reported in their semi-annual reports 
was not TA1
. (e.g., a phone call was considered a TA incident). Some of the TA reported 
is not usually considered TA. There is a need for a clarification of operational definitions 
for TA that is approved, and adopted by USAID. There were, in addition, documented 
instances where CAC reported TA under the HCRCP and simultaneously under other 
CAC projects in Ethiopia. 
Additional activities were requested of the CAC such as a review of and commenting on 
Standards of Care for Health Centers, a brief report on health center space requirements, 
a draft report on storage capacity, Assessment for Ethiopian MOH facilities, a brief 
update report on seconding short-term technical assistance (STTA) consultants to FMOH 
and RHB, development of SOWs and job descriptions for the seconded 8 STTA 
consultants, and other additional tasks. The Team noted these and other additional 
activities were done by CAC. Details are presented in the Activity Description Appendix 
(Appendix 8). 
Overall, the evaluation team concluded that CAC met expectations with respect to Result 
2. Based on the team’s experience in similar projects, we feel that CAC/USAID missed 
many opportunities to involve more A&E firms, to offer more short- and medium-term 
off-site trainings and work with pre-service and institutions to train more people at 
multiple levels of HC renovation. The one A&E firm did hire consultants that could be 
contracted again, if they are still available. 
 
RESULT 3: DIRECT RENOVATION OF HEALTH CENTERS 
Overall, the team found that the quality of the fixtures used for renovations was low. 
Residential grade fixtures were used and presented problems. The fixtures were 
frequently non-functional in whole or part in most of the sites that the team visited. A 
rapid assessment of available high quality fixtures found that better quality residential 
and commercial fixtures are available in Ethiopia. 
 
1
 For example: from CAC’s PY1 annual progress report, it is claimed that there are 74 separate “T.A. 
activities” which is actually one – maybe two –but they counted each follow-up phone call, meeting, etc. 
for this purpose. USG standards in project monitoring do not count each phone call to each client/partner as 
an activity. 
22 
The installation of fixtures was inconsistent and of poor quality. This was felt to be 
predominantly due to the fact that the specifications used were not detailed enough. The 
installation processes for water supply and sanitation were also found to be of poor 
quality and were installed inconsistently. For example, in some locations, walls were 
broken out to enclose the pipes, which was problematic. In others they were outside and 
properly secured, yet in others, they were outside but improperly placed and secured. The 
water tanks that were used to replace the existing ones were smaller than the originals 
despite the fact that the renovations actually increased the volume of water needed at the 
site. 
The quality of doors was assessed by the team to be very good, although the installation 
process was poor (e.g., incorrectly hung doors) and inconsistent. Windows were of good 
quality and installed correctly. The quality of framing and foundation material, as well as 
their installation was assessed to be of good quality. 
The quality of roofing and gutter materials was inadequate. The team also found that the 
specifications for these materials were not respected. For example, the locations of the 
downspouts were not appropriate, allowing rain water to pour onto external passageways. 
Given these conditions, the team concluded that supervision was inadequate in assuring 
the quality of renovations. None of the sites had copies of renovation plans. When the 
team inquired, the staff had neither the master plans nor the blueprints of the renovations 
to be done. 
Only a minority of the sites had seen the maintenance check list that CAC had developed. 
Less than half of the sites knew that there was a one-year guarantee on renovations and 
that they should contact CAC if there were any problems. 
Health perspective results of the site visits showed that the cleanliness of the inside of the 
health centers visited was very good. Outside cleanliness was also good. Practical safety 
precautions were not generally taken, such as in some sites where used needles were left 
lying around near an incinerator site. 
Health service staff indicated that the renovations had not helped them provide better 
health services to clients, because most of the HCs reported as being “completed” had 
problems. They stated, however, that if the problems were fixed, it would help improve 
the quality of their services. At two health centers that had little or no problems, they said 
it has helped deliver better services, especially changes to delivery room and water 
supply. 
In summary, the team feels that the project did not meet expectations with respect to 
Result Three. 
23 
VI. QUESTIONS POSED TO THE EVALUATION TEAM AND ANSWERS 
PROGRAM MANAGEMENT 
• Have the resources been effectively managed to achieve program results? 
Although it appears that CAC will achieve Results 2 and 3 of their contract, this was 
only assessed against the two very general performance indicators (100 assessed; 50 
renovated). However, this is not an indication of management’s performance. The 
quality of the renovations of the health centers has been poor and not up to 
international standards. More effective management of funding and project 
inputs/resources would have allowed project results to be achieved. 
 
• Has management effectively planned, implemented and monitored the program’s key 
interventions? 
While the planning of the program was good, implementation and monitoring were 
not effective. The CAC Chief of Party and the Mission CTO were unaware of the poor 
quality of the workmanship on the renovations. There were not adequate site visits 
and overall project monitoring. Also, given that the Year 1 Work Plan was approved 
at the end of the first year and the second year had not been approved more than 
half-way into the Year Two, combined with the low reliability and validity of the 
reporting, the team cannot conclude that the project was managed effectively.
• Have the TA resources been identified, managed and supervised adequately? 
TA was a significant requirement of the CAC contract. The project was designed for 
the CAC to provide TA to GOE personnel at the federal, regional and district levels. 
With only one exception reported by CAC in one region, no TA was provided to GOE 
personnel. This was due in part to the massive construction program underway in the 
health sector which requires the attention of the GOE technical personnel. CAC did 
provide excellent training to the staff of one local A&E firm that subsequently 
designed some fifty health center renovations. This kind of project assumes USAID 
construction, architectural and/or engineering expertise in-house or provided 
through TDY. Moreover, the work plan format does not contain the necessary fields 
(information) to monitor the project, and the reporting formats are not conducive to 
decision-making. 
• Has management identified problems in implementation and addressed these 
appropriately? 
From the renovation/engineering perspective, problems were not identified and 
addressed appropriately. This is demonstrated by the poor quality of the renovated 
health centers. CAC had not identified problems with implementation of activities in 
their reports, but rather identified problems regarding the budget being insufficient to 
reach the targeted number of renovations (50) and problems with the lack of 
coordination. 
USAID addressed issues very late in the implementation. At times, poor decisions 
were made to overcome constraints, creating even more constraints. For example: 
USAID waited a long time to work with the FMOH to negotiate buy-in to the project 
so that CAC could start renovations with the FGOE support. When they did, they 
24 
indicated to the FMOH that it would likely be possible for CAC to support an activity 
that was not yet budgeted (paying consultant salaries to be placed at the FMOH). 
IMPLEMENTATION 
• How effective is this approach/model of coordination, TA and direct renovation for 
supporting the development of GOE capacity to renovate health centers for 
HIV/AIDS and other chronic diseases? 
The CAC approach/model was not effective given that it was done in complete 
isolation of the GOE and has not increased the capacity of the GOE to renovate 
health centers. If, however, the GOE does eventually adopt, and use the guidelines 
developed by CAC, this TA will help with renovations for health centers. 
• How well is this project assisting the GOE to plan and coordinate the activities of 
assessments, planning, technical reviews and renovations done by multiple 
organizations providing health center renovations? 
The isolated approach of CAC in the assessment, design, and renovation of the health 
centers does not assist the GOE or other organizations to undertake health center 
renovations. 
• Is the “engineering assessments’” approach and their implementation adequate to 
address both the engineering and health service delivery needs of health centers? 
Yes. Without exception, the most significant contribution by CAC has been the 
development of the two handbooks; namely, the Health Center Assessments 
Handbook and Health Center Renovation Handbook. These handbooks serve to 
support both the renovations and the health service delivery. 
• Has the amount and quality of the assistance/resources used for the assessments been 
adequate and appropriate? Have they been directed at priority health centers? 
Yes. The quality of the assessments done by both by CAC and the local A&E firm has 
been excellent. However, the appropriateness of CAC undertaking the health center 
assessments directly is questionable. The trained local A&E firm completed fifty one 
assessments at less than a fraction of the cost of the assessments performed by CAC. 
It would have been more cost effective to have the assessments performed by the local 
A&E firm since the quality of their assessments were comparable to those of CAC. 
• Has the direct support to renovate selected health centers been adequate and of 
sufficient quality to meet program targets? 
No. The specifications for some of the materials did not meet acceptable standards 
nor did the quality of some of the construction. 
MONITORING AND EVALUATION (M&E) 
• Has the M&E system adequately tracked the progress using the mandated indicators? 
Yes, technically, because there were only two mandated indicators: the number of 
HCs assessed, and number of HCs renovated. These do not have definitions and there 
are no other processes, output or outcome indicators. However, by USAID’s own 
M&E standards, this M&E “system” is inadequate to track progress. 
• Has the information from the M&E system been adequately used to plan and manage 
the program? 
25 
No. There is essentially no M&E “system”; nor is there a “monitorable” work plan. 
The M&E component did not have useful performance indicators. 
• What indicators of the quality of renovations can/should be incorporated into 
monitoring the follow on project? Is there a need to introduce or change Project 
Results? 
Yes. There are many possible indicators; a list was left with USAID before the 
evaluation team departed. A few of those provided are: number of change orders 
required, number and type of repairs needed in the first 12 months, number of 
contracts for which the retention was paid to the contractor. Yes, there is a need to 
change the project results. 
• What indicators could be adopted to measure the program in the areas of coordination 
and capacity building, renovation, technical assistance? 
There are many possible indicators; a list was left with USAID before the team 
departed. A few of those provided are: number of multi-stakeholder joint activities as 
per work plan, number of A&E firms trained, number of RHBs participating in 
assessment and supervision training, number of training institutions that offer 
courses or certificate programs in HC renovation management, assessment & design. 
QUALITY ASSURANCE 
• Has the Project provided appropriate quality standards as part of the TA provided? 
The handbooks for health center assessment and for renovations provided the 
standards for the engineering and for health services use. 
 
• Have the Health Center renovations met minimum international quality standards? If 
not, why not? How can health center renovations be done to meet appropriate 
standards in the most cost effective way? 
As indicated above, the renovations do not meet minimum international standards. 
The rationale given for the inferior specifications was budget limitations. The 
renovations can be done to meet appropriate standards by providing sufficient 
funding and ensuring that the TA contractor requires the use of international 
standards. Equally important to ensure that renovations meet international standards 
is to increase the level of construction supervision. 
• Are the criteria for selecting health centers for renovation appropriate? 
Yes. They are USG’s priority health centers (with respect to HIV/AIDS). 
• Is the selection of health centers to be renovated done according to adequate criteria 
and based on priorities? 
Yes. See above. 
CAPACITY BUILDING AND SUSTAINABILITY 
• How effective is the strategy for building the capacity of the GOE to coordinate and 
technically guide health center renovation? 
The strategy for building the capacity of the GOE has not been effective. Given the 
involvement of the GOE technical staff with other construction in the health sector, 
CAC basically stopped attempting to involve the GOE technical staff in the program. 
 
26 
• What has been the increase in capacity of the GOE to implement and sustain these 
activities? 
There has basically been little or no increase in capacity of the GOE technical staff. 
27 
VII. RECOMMENDATIONS AND LESSONS LEARNED 
The recommendations presented below reflect lessons learned and are focused on 
strengthening future health center renovations projects. 
COORDINATION 
To strengthen the future project for the coordination component, USAID should consider: 
• Investing time and considerable effort to obtain FMOH and RHB buy-in to a HC 
renovation project before it is awarded. 
• For the new project, it is recommended that the contracting for assessments and 
renovations be done by the government. It is suggested that the TA staff be stationed 
at the RHBs. With the assistance of the TA contractor, the staff at the RHBs will 
improve their skills in contracting administration and supervision. In addition, with 
the contracts being led by the RHBs, the contracted A&E firms located near the 
RHBs will have less travel to the sites and the frequency of site inspections should be 
increased. With the overall contract supervision by the RHBs, the site visits by the 
RHB staff will also increase. This will increase the supervision and as a result the 
quality of the work will be improved. 
• The USG agencies (USAID/CDC) should lead a coordination effort among USG 
implementers so that whenever sites are visited and supervision is done, routine HC 
maintenance and safety is addressed: this is part of total quality management. The 
USG, and not one of the (many) USG implementers, should lead this effort (but one 
or more USG implementers may be tasked with assisting and preparing the USG to 
do this). 
TECHNICAL ASSISTANCE 
To strengthen the future project for the technical assistance component, USAID should 
consider: 
• Identifying multiple alternatives to capacity building and TA and assess which of 
them are feasible at the onset of the project. 
• Exploring alternative training opportunities such as supporting pre-service and 
technical training institutions. 
• Budgeting and planning to train several different A&E firms so there is a backup 
when one does not work effectively. 
• It is suggested that a significant training program be a part of the new project. With 
the excellent material developed by CAC for HC assessment, the new project should 
undertake training programs for both the RHB staff and the local A&E firms on the 
proper methods of performing HC assessments. 
• It is recommended that the new TA contract include a comprehensive program that 
will assess the overall maintenance requirements of the Ministry, identify the required 
28 
maintenance staff, and identify the appropriate sources of training for the required 
staff. This program should be all inclusive from routine maintenance to the 
maintenance of the most sophisticated equipment in the national hospitals. 
MANAGEMENT 
To strengthen the management of future programs, USAID could consider the following 
recommendations: 
• USAID should work with the contractor and approve work plans on time, and 
document changes including reallocation of resources. 
 
• When not certain about the extent of renovations needed, set the renovation target low 
for the first year and increase it the second year, based on experience and the 
environment. 
• Give clear guidance to the implementer regarding when and how changes to an 
approved work plan can be made, being careful to document any changes to this 
while it is being implemented and showing, when necessary, how budgeted resources 
are reallocated when activities are added/dropped/replaced. 
• Require, as part of the reporting, a simple inventory of travels (within and outside of 
country) which were financed by the project. For the HCRCP, this would be the CAC 
staff travel and the travel made by the A&E firm as well as any travel and/or per diem 
paid to national counterparts. The inventory includes date, destination, persons 
traveling, and a brief purpose that is identifiable as contributing to one or more 
expected results. 
• Clearly state in guidance that only activities funded from the HCRCP project can be 
reported as services and deliverables for that project. If the implementer has other 
funding via subcontracts – or receives other funds from other projects for related HC 
renovation activities, it cannot report them as achievement deliverables under this 
project unless the cost was shared by more than two sources, in which case this must 
be documented when reported and submitted. 
 
• Provide guidance to implementers as to what is an “expected” service or deliverable 
and what is “unanticipated”. This will largely be developed through a stronger PMP 
and work plan but USAID may wish to add to quarterly reporting requirement a table 
of expected and unanticipated deliverables and services. USAID could then review 
these routinely. 
 
• Consider the level of expertise needed. Engineers and architects are not needed for all 
aspects of health center renovation, especially when they are simple, single story 
buildings. The project could consider training and use of entry and intermediate-level 
personnel for site supervision: this would be more cost-effective. 
RENOVATIONS 
To strengthen the assessment and renovation component of future projects, USAID 
should consider the following recommendations: 
29 
• Establish a minimum renovation package for each site during the project design to 
help in establishing an appropriate budget and implementation plan. Increase the 
threshold for what is considered a “minimum package” renovation - after having 
operationally defined a minimum package (e.g., include in all cases covered waiting 
areas with benches or seats, which will keep the passageways clear, which is not the 
case in all of the sites). 
• Increase the amount of funding available for site renovation (per site) so that a 
minimum package can be offered; factor into the budget small items which 
“complete” a renovation (such as shelves in a newly renovated administrative room or 
seats in the waiting area that was expanded and roofed, rather than have clients sitting 
on the ground). 
 
• USAID, the implementing agency, and the GOE should strategize how to make full 
use of the excellent assessments done and assure better conformity to the (master) 
plan including changes in client/provider flow, infection prevention, improving 
privacy and comfort. 
• Add to the project objectives and budget maintenance of the health center; the 
grounds, the infrastructure and the equipment. 
• Improve the specifications for the renovations to show the correct ways to install 
fixtures. 
• Require that the implementing agency make more detailed specifications for the 
renovations – both for materials to use/purchase and how to install these; use 
commercial grade material. 
• Always include renovation of waiting areas to keep hallways free and to make clients 
more comfortable; this was done at some sites but not at others that need it. 
 
• If PEPFAR permits, take into consideration (to a greater extent) the needs of the HC 
personnel when renovating. If the HC operates around the clock, there should be a 
comfortable place for the providers to rest during the night. Also, showers, separate 
toilets, etc. should be considered. 
• Use commercial grade fixtures, roofing and gutters for renovations instead of cheap 
residential grade materials. 
• Invest some time in training the HC personnel how to do routine maintenance, such 
as tightening loose screws, cleaning the drain traps, knowing where to turn off the 
water supply. 
• When replacing water tanks, always increase tank capacity when renovations 
effectively increase water use and volume. 
MONITORING AND EVALUATION 
To strengthen the monitoring and evaluation of future projects, USAID should consider: 
 
30 
• Adopting some process and outcome level indicators and including these in the PMP. 
Ensure that the PMP has all components as per USAID guidelines. Provide the 
implementer with a standardized work plan and reporting formats and instructions for 
semi-annual reports and ensure that there are operational definitions for all indicators. 
• Providing the implementing agency with standardized tools and procedures and the 
proper guidance for project monitoring and evaluation. 
Scope of Work – Health Center Renovation Evaluation 
VIII. APPENDICES 
APPENDIX 1) SOW FOR THE EVALUATION 
Draft Statement of Work (SOW) for End of Project Evaluation of the 
Health Center Renovation Coordination Project (HCRCP) 
(Draft # 5 - 24 April 2008) 
PROJECT IDENTIFICATION DATA 
1. Project Title: Health Center Renovation Coordination Project (HCRCP) 
2. Project Number: 663-C-00-06-00428 
3. Project Dates: 09/29/06 to 09/27/08 
4. Project Funding: $3,919,941 
5. Implementing Organization: Crown Agents
6. Cognizant Technical Officer (CTO): Jamie Browder
I. Identification of the Task 
The USAID/Ethiopia (USAID/E) Health, AIDS, Population and Nutrition (HAPN) Office 
requests technical assistance from the Mitchell Group (TMG), under the USAID/E Evaluation 
Contract Number 663-C-00-08-00409-00, to design and implement an independent external end￾of-project evaluation of the Health Center Renovation Coordination Project (HCRCP). The 
HCRCP coordinates renovation and construction initiatives to improve HIV/AIDS services in 
Ethiopia, and carries out selected renovations, filling gaps when other funds are not available. 
The HCRCP’s overall goal is to support the Federal Democratic Republic of Ethiopia (FDRE) to 
provide improved HIV/AIDS care and treatment services at health centers through the following 
key objectives: 1) Serve as a coordination and synchronization “hub” with national and bilateral 
stakeholders on health center renovations, 2) Provide technical assistance (TA) on renovations 
to Ministry of Health (MOH) Federal and Regional Health Bureau (RHB) offices, and 3) Provide 
direct renovation support at selected health centers. This external end-of-project evaluation will 
assess the achievements, impact, and challenges for the project in addressing these three 
objectives.
The USAID/E HAPN office requests that the evaluation be completed by May 29, 2008 in order 
that the findings, conclusions and recommendations can be used in the implementation of the 
planned follow-on project for health facility renovation. 
II. Background: 
The number of HIV positive individuals in Ethiopia was estimated at 977,000 in 2007 and is 
projected to increase substantially in the next two years (a 30% increase was projected from 
2006 to 2010). The projected growth in the total size of the HIV-infected population in Ethiopia 
means that efforts to strengthen facility-based services will be critical for a successful national 
HIV/AIDS program. The Government of Ethiopia (GOE) Federal Ministry of Health (FMOH) is 
decentralizing ART and other HIV/AIDS services to the health center level. HIV/AIDS services 
at health centers require adequate infrastructure to accommodate the rapid and sizable increase in 
HIV/AIDS clients at these sites. 
Scope of Work – Health Center Renovation Evaluation 
The quality of Ethiopia’s public sector facilities providing anti-retroviral therapy (ART) is 
among the key limiting constraints to increasing enrollment. ART services require adequate 
infrastructure to support the sizeable increase in clients (up to 200/site) expected at priority 
health centers. The serious infrastructural limitations of Ethiopia’s health facilities are important 
factors in the low utilization of HIV/AIDS care and treatment services, particularly PMTCT. 
An FY06 assessment by Family Health International (FHI) identified infrastructure deficiencies 
as a major obstacle to sustained progress in achieving ART targets. A subsequent FY07 
assessment by Crown Agents of 44 health centers currently providing ART services in the four 
most populous regions of the country (Oromia, Amhara, SNNPR, and Tigray) and the city of 
Addis Ababa found that nearly all health centers suffered from serious physical/structural and/or 
essential function deficiencies (e.g., lack of running water, blocked waste water disposal lines, 
overflowing dry pit latrines, and/or electrical malfunctions). These facilities also faced space 
limitations that compromised patient care and the safety of clients and healthcare providers. The 
report also found that the poor condition of the health centers may also have reduced the shelf￾life and quality of valuable HIV/AIDS commodities supplied by PEPFAR, the Government of 
Ethiopia (GOE) and other sources such as the Global Fund To Fight AIDS, Tuberculosis and 
Malaria (GFATM). It is noted here that the renovation of pharmacies and laboratories was 
specifically excluded from the SOW of the contractor, Crown Agents. 
Ongoing efforts to renovate health facilities: Currently, several bilateral and multilateral 
agencies, as well as non-governmental organizations (many of which are PEPFAR partners or 
USG-supported with non-HIV/AIDS funds) are working independently to renovate health 
centers, hospitals, supply warehouses and other facilities. These include the Japanese 
International Cooperation Agency (JICA), the German Technical Cooperation Agency 
Gesellschaft für Technische Zusammenarbeit (GTZ), the U.S. State Department’s Regional 
Procurement Support Office (RPSO), Pathfinder International, Management Sciences for Health, 
IntraHealth International, and the Packard and Clinton Foundations, all of which have worked 
and in some cases continue to work independently to renovate hospitals, health centers and other 
facilities. 
Health manpower capacity to engineer, plan and monitor the expansion and renovation of 
facilities is limited. Following the Paris Declaration, the GOE adopted a plan to achieve 
universal access to primary health care, including HIV services, by 2010. Under this plan, the 
MOH has a goal to increase the number of health centers in the country from around 630 in 2007 
to 3,153 by the year 2010. This ambitious undertaking to expand the health infrastructure means 
that the engineering, construction planning and monitoring capacity of the federal and regional 
health administrations are stretched to the limit. The application of a systematic and structured 
approach for coordination is urgently needed among the agencies currently supporting 
renovation and construction, the MOH and RHBs, to rationalize infrastructure improvements at 
health centers and other facilities. Improved coordination will heighten the impact that facility 
renovation and construction can have on expanding sustainable HIV/AIDS, chronic diseases and 
related services across the country. The need for coordination and appropriate technical guidance 
will become even more acute as the country moves into a major renovation phase supported by 
GFATM and the Global Alliance for Vaccines and Immunization (GAVI) funding in support of 
the push for universal access to primary health care services. GTZ has been tasked by the MOH 
with carrying out much of this construction, which will convert large numbers of existing lower 
level health stations to health centers. 
Scope of Work – Health Center Renovation Evaluation 
As mentioned above, a number of US Government-funded partners in Ethiopia are currently 
engaged in health center renovation activities. For example, the Rational Pharmaceutical 
Management Plus Project under Management Sciences for Health (MSH/RPM Plus) is funded by 
PEPFAR to renovate pharmacy stores as well as voluntary and counseling testing (VCT) and 
antiretroviral (ARV) dispensing rooms at 23 health centers. Other key PEPFAR partners 
involved in limited renovation and refurbishment include MSH’s HIV/AIDS Care and Support 
Project (HCSP). The State Department’s Regional Procurement Supply Office (RPSO) renovates 
hospitals and laboratories in conjunction with four PEPFAR-funded American university 
partners supported by the Centers for Disease Control and Prevention (CDC). The non-PEPFAR 
USAID-funded DELIVER project is also carrying out renovation of health center stores. Before 
the advent of the HCRC Project, systematic overall coordination of these efforts was limited and 
incomplete. 
III. Overview of the Health Center Renovation Coordination Program 
Begun in September 2006, the USAID/Ethiopia PEPFAR-funded HCRCP has worked to 
standardize, synchronize and facilitate proposed GOE and U.S. Government-funded renovations, 
with a focus on existing health centers. The activity has provided technical assistance to the 
MOH and RHBs for renovation coordination, technical design, engineering, procurement and 
logistics support. The project has also placed short-term technical assistance at the national and 
regional level to support staffing gaps in support of GOE health facility renovation/construction 
efforts irrespective of funding source. A particular emphasis of these health facility 
renovation/construction efforts is on the conversion of health stations to health centers. As part 
of this effort, the HCRC Project has also directly assessed and renovated selected health centers, 
so that the facilities can better support chronic disease management, and HIV/AIDS services, 
especially antiretroviral therapy (ART). 
Geographic coverage: The HCRCP serves the four most populous regions of the country 
(Oromia, Amhara, SNNPR, and Tigray) plus the capital city of Addis Ababa, potentially 
benefiting millions through improved infrastructure for ART services. See Attachment 4 for a 
map of HCRCP sites. 
HCRCP Management and Administration: Management is accomplished via joint oversight 
by the Chief of Party (COP), responsible for technical and programmatic areas, and the Project 
Manager, responsible for administrative and financial matters. Figure 1 depicts the 
organizational structure for implementing and managing the project and diagrams the 
relationship between the HCR Project office in Addis Ababa and Crown Agents Consultancy, 
Inc. headquarters in Washington DC; Crown Agents/UK headquarters in Sutton, England; and 
the Crown Agents Ethiopia country office. 
Scope of Work – Health Center Renovation Evaluation 
CAC/DC Office
Project Manager
Health Centers
USAID/E
Mission
FMOH/
PPD
HCR Project Office
COP I SPO 
I & CE
National 
Coordination 
Committee
CA Country Office
A&E Firm Contractors
Payments
& Transfers
CA/UK Office
Technical Director
(Engineers) 
Senior Consultant 
Engineer
Invoices, Reports 
& Databases
Health Center Renovation (HCR) Project Health Center Renovation (HCR) Project
Sub￾contracts
Program Objectives: The HCRCP’s overall goal is to support FDRE efforts to provide 
HIV/AIDS care and treatment services at health centers. 
The three main expected project results are: 
• Coordination - Serve as a coordination and synchronization “hub” with national and 
bilateral stakeholders on health center renovations to support chronic disease; 
• Technical Assistance - provide assistance on renovations to the FMOH and RHBs, 
including technical design, engineering, procurement and logistics support (100 Health 
Centers (HCs) in total); and 
• Renovation - Provide direct or indirect renovation support at selected health centers to 
support chronic disease management including ART (50 HCs in total). 
According to the work plan for the first year of activities, the HCRCP had three principal 
results: 
• RESULT 1 –Establish a National Health Center Renovation (HCR) Coordination 
Mechanism 
o Identify Key HCR Implementing Agencies 
o Identify HCR Coordination Meeting Participants 
o Develop an HCR Tracking System 
o Develop an HCR Harmonization and Coordination Plan 
o Assist FMOH/Planning and Programming Department (PPD) to conduct a 
National HCR Coordination Meeting. 
Scope of Work – Health Center Renovation Evaluation 
• RESULT 2 - Technical Assistance to FMOH/PPD and RHBs (50 HC assessments per 
year) 
o Identification of TA requirements 
o Field-based Assistance to RHBs 
o Preparation, Testing and Vetting of Standardized Documents and Procedures 
o Contract Documentation 
o Identification of Engineering Assets 
• RESULT 3 - Renovation of 20 HCs 
o Identification of HCs for Renovation with RHBs and Other Stakeholders 
o Preparation of Standardized Guidelines and Procedures 
o Identification and Procurement of Architect and Engineering (A&E) Firm 
o Pre-Qualification of Works Contractors 
o Health Center Renovation Works (20) 
During the first project year (PY1) the HCRCP provided 156 specific technical assistance 
activities related to Result 1; provided TA on HC assessments under Result 2, involving more 
than 65 visits to 59 selected HCs located in the four major regions and Addis Ababa; and, under 
Result 3, provided direct renovation support to 23 selected HCs in the four major regions and 
Addis Ababa. 
HCRCP Strategic Framework: While HCRCP is implemented within the PEPFAR Country 
Operational Plan framework and targets, it is also considered under the USAID Mission’s 
strategic framework elements and sub-elements, the USAID Integrated Strategic Plan FY 
2004-2008 and the 2007 Foreign Assistance Framework. USAID/Ethiopia’s HCRCP was 
initiated under the USAID/E Integrated Strategic Plan (ISP) for the period FY 2004 to FY 
2008 under the strategic objective (SO) SO 14: Human Capacity and Social Resiliency 
Increased and SO 14.2 HIV/AIDS prevalence reduced and mitigation of the impact of 
HIV/AIDS increased. In 2007, the SO14 was incorporated into an alternate Foreign Assistance 
Framework (F-Framework) for the USAID 2007 Operation Plan. The activities under the 
HCRCP project now fit under the F-Framework Objective: Investing in People, Program 
Area: Health, Program Elements: HIV/AIDS; Program Sub Elements: Treatment and Care 
Services. 
 
Recent HCRCP activities: The actual HCRCP renovations of selected Health Centers (HCs) to 
support care and treatment services include: Reception/Registration Areas, Client Waiting Areas, 
Outpatient Delivery, Maternal Child Health, HIV Counseling and Testing, Chronic Care Clinics, 
stores, latrines and toilets. As mentioned above, under the current cooperating agreement scope 
of work, the HCRC Project does not support (or report on) renovations to pharmacy or laboratory 
facilities in health centers. This work is currently covered under separate contracts or 
agreements.
In the first half of Project Year Two, from September 30, 2007 through March 31, 2008, the 
HCRCP completed engineering assessments on 22 HCs located in the four most populous 
regions and Addis Ababa, awarded contracts for the renovation of these 22 HCs, and provided 
TA to the FMOH and RHBs for the expansion of four HCs and the design of a Hospital’s waiting 
and administration areas. In response to an acute shortage of technical experts within the RHBs, 
Scope of Work – Health Center Renovation Evaluation 
the cooperating agreement has recently been modified to permit HCRCP to recruit nine Short 
Term Technical Assistance (STTA) architects and engineers to be seconded to four RHBs and 
the FMOH/PPD. In addition, the HCRCP has drafted a HC renovation and health services 
tracking system, and developed a working draft of Guidelines for Health Center Renovation and 
Expansion and a Health Center Assessment Handbook. 
Monitoring and Evaluation (M&E): The two primary indicators for the HCRCP are shown in 
Table 1 below. 
Major HCRCP Indicators LoP 
Target 
Achieved to date 
(03/2008) 
Number of HCs provided direct renovation support 50 
23 (12/07) 
completed 
22 03/08) under 
contract 
Number of HCs provided with TA for renovation 
planning, assessment and renovation documents 100 81 (as of 03/08) 
Other indicators include: Result 1, the number of technical assistance meetings and visits; Result 
2, the number of times provided TA to RHB and HC/Woreda Staff; and Result 3, the number of 
HC engineering assessments completed, and number of HCs undergoing renovation. For a 
detailed summary table of current HCRCP indicators see Attachment 1. 
Emerging Issues: Over the past year, HAPN has noted the following emerging issues for 
HCRCP program implementation: 
Limited availability of RHB/Zonal/Woreda Engineering staff to assist in conducting engineering 
assessments and other TA activities in four of the five regions and insufficient funding to provide 
substantial short term TA to address this constraint. The former problem is being addressed by 
hiring STTA architects and engineers. 
Lack of interest in the proposed Health Center Renovation (HCR) Coordination Committee. 
Lack of support for leveraging health center renovation resources from non-USG donors. 
Need to upgrade PEPFAR priority HCs to a minimum performance standard, coupled with an 
inadequate funding to implement upgrades. 
Delay in official acceptance of HCR Project Statement of Work by FMOH/Planning and 
Programming Department. 
IV. Purpose of the Assignment 
The purpose of this evaluation assignment is to implement an independent external end-of￾project evaluation of the Health Center Renovation Coordination Project (HCRCP).
The evaluation requires a core team of two experts: 1) a senior expatriate Team Leader trained in 
civil engineering or architecture with extensive health facility renovation program experience (25 
days), and 2) a senior expatriate HIV/AIDS health services expert with prior experience working 
in Sub-Saharan Africa (22 days). The local TMG sub-contractor will provide all needed 
logistical support. Additional technical team members may be provided by local and 
Scope of Work – Health Center Renovation Evaluation 
international stakeholders, potentially including OGAC, CDC/Atlanta, USAID/Washington and 
GOE staff participation. With about six months remaining in the project, this evaluation will 
assess the progress of HCRCP implementation and challenges. It will document lessons learned 
and formulate recommendations for the proposed follow-on program, “Facility improvements to 
strengthen HIV/AIDS services: capacity building at multiple levels.” The evaluation will cover 
the HCRCP performance period of September 2006 through March 2008. The evaluation report 
will assist USAID and Crown Agents to address topics of program management, planning and 
coordination, the provision of TA, quality of services, and sustainability during the time 
remaining in this project.
This is external end-of-project evaluation will assess program process, document lessons learned, 
and inform follow-on activities. 
This evaluation will: 
• Assess the progress toward contract targets. 
• Assess process used to implement the program’s three key areas of intervention to meet 
objectives. 
• Assess the effectiveness of the three key project interventions: the approach and activities 
used by Crown Agents to assist the GOE to improve health center renovation planning, 
design and operations for HIV/AIDS and other chronic diseases among various 
institutions, and other cooperating agencies, other donors, HAPCO, the Ministry of 
Health and other Ministries. 
• Identify additional achievements and unexpected benefits. 
• Identify project components that could be scaled up beyond the project agreement in 
Ethiopia. 
The evaluation will answer the following illustrative questions: 
Program Management: 
• Have the resources, (human, financial and other) been effectively managed to achieve 
program results? 
• Has the management effectively planned, implemented and monitored the program’s 
key interventions? 
• Have the TA resources been identified, managed and supervised adequately? 
• Has management identified problems in implementation and addressed these 
appropriately? 
Implementation: 
• How effective is this approach/model of coordination, TA and direct s renovation for 
supporting the development of GOE capacity to renovate health centers for HIV/AIDS 
and other chronic diseases? 
• How well is this project assisting the GOE to plan and coordinate the activities of 
assessments, planning, technical reviews and renovations done by multiple organizations 
providing health center renovations? (These include PEPFAR partners such as the CDC 
Scope of Work – Health Center Renovation Evaluation 
(which supports the Regional Procurement Support Office or RPSO), the Management 
Sciences for Health (MSH)/Rational Pharmaceutical Management Plus (MSH/RPM 
Plus), non-PEPFAR stakeholders, such as Gesellschaft für Technische Zusammenarbeit
(GTZ) and others as appropriate)
• Are the “engineering assessments’” approach and their implementation adequate to 
address the both the engineering and health service delivery needs of health centers?
• Has the amount and quality of the assistance/resources used for the assessments been 
adequate and appropriate? Have they been directed at priority health centers? 
• Has the direct support to renovate selected health centers been adequate and of sufficient 
quality to meet program targets?
Monitoring and Evaluation (M&E) 
• Has the M&E system adequately tracked the progress using the mandated indicators? 
• Has the information from the M&E system been adequately used to plan and manage the 
program? 
• What indicators of the quality of renovations can/should be incorporated into monitoring 
of the follow on project? 
• What indicators could be adopted to measure the program in the areas of coordination 
and capacity building, renovation, technical assistance? 
Quality Assurance 
• Has the Project provided appropriate quality standards as part of the TA provided? 
• Have the Health Center renovations met minimum international quality standards? If not, 
why not? How can health center renovations be done to meet appropriate standards in the 
most cost effective way? 
• Are the criteria for selecting Health Centers for renovation appropriate? 
• Is the selection of health centers to be renovated done according to adequate criteria and 
based on priorities? 
Capacity Building and Sustainability 
• How effective is the strategy for building the capacity of the GOE to coordinate and 
technically guide health center renovation?
• What has been the increase in capacity of the GOE to implement and sustain these 
activities?
• What are the priority areas of capacity that still need improvement? 
Lessons Learned 
• What lessons have been learned to guide improvement of the program and for the 
new follow-on program development? 
• Identify lessons learned, successful interventions for continuation or replication, best 
or better practices, and significant products and tools from the HCRCP for 
consideration for possible dissemination and replication. 
 V. Evaluation Methods 
The evaluation will be carried out by a core team of two independent, external consultants over a 
three-week period using multiple methods, including review of HCRCP reports, tools, and 
materials, key informant interviews, field observation, facility assessments, and selected health 
Scope of Work – Health Center Renovation Evaluation 
provider and client interviews. The two core team consultants may be joined by up to three 
representatives from the MOH and HAPCO at the national and/or regional level. The evaluation 
team working with USAID/E, the GOE, RHB’s, Crown Agents and others, will identify a 
representative subset of Health Centers to visit. Activities will include: 
• Review of relevant documents. 
• Interviews with key stakeholders, government officials, health services staff, 
representatives of beneficiaries of the health centers. 
• Meetings with Crown Agents staff. 
• Field visits to selected regions and districts (one combined team will be used to assure 
that the issues of engineering and of service delivery are addressed in the facility 
assessments of renovations. The calendar attached shows the major tasks and 
timeframes). 
• Direct observation of assessment process and of renovation (if possible) at selected 
health centers. 
• Assessment of renovated health center facilities. 
• Follow-up on the current status of a sub-sample of the earliest renovated HCRCP 
health centers. 
• Visits to one or more HCs that have not been selected for renovation. 
Specific evaluation techniques may include: interviews, focus group discussions (FGDs), and 
technical health facility assessments from both an engineering and health service delivery 
perspective. 
VI. Information Sources 
Crown Agents will provide consultants with appropriate background documents in preparation of 
the assignment. USAID and Crown Agents will identify priority documents if possible from the 
list. The documents will include but are not limited to: 
• HCRCP Cooperative Contract including all subsequent modifications 
• HCRCP PEPFAR Semi-Annual Report submissions 
• HCRCP 2006 and 2007 Annual Reports 
• HCRCP Quarterly Reports 
• HCRCP Guidelines 
• HCRCP M&E Tools 
• Prior assessments and evaluations carried out by HCRCP 
• USAID trip reports summarizing past field visits to HCRCP 
• GOE Road Map for HIV/AIDS Prevention, Care and Treatment 
• Pertinent GOE documents on related programs such as the Health Services Extension 
Program. 
VII. Tasks to be accomplished: 
Below is a list of the specific tasks to be accomplished by the consultant team, with an estimated 
level of effort for each task. See Attachment 2 for a Planning Calendar for the proposed dates for 
these tasks. 
Scope of Work – Health Center Renovation Evaluation 
Key activities Level of 
Effort in 
Days 
Team reviews background documents, 
initial discussions with TMG 
Coordinator, reviews methodology and 
field visit and interview schedule. 
3 
Team Planning meeting at TMG –DC. 
Review evaluation methodology, 
conference calls to CA HQ, develops 
outline of report, develops draft work 
plan that is sent to HAPN for review prior 
to departure for Ethiopia. 
2 
Travel to Ethiopia 1 
 In country in-brief consultation with 
CTO, Evaluation Coordinator and HAPN. 
Team discusses and submits Draft Work 
Plan to USAID/E. Work plan approved 
based on HAPN comment. 
1 
MOH and other stakeholders meetings, 
key informant interviews 
3 
Full team TPM 2 
One Team conduct field visit, facility 
assessments and interviews 
6 
Full team analysis and report writing 2 
Core team analysis, report writing, and 
debriefings 
4 
Travel from Ethiopia 1 
USAID/ Review of draft report 
Report - Response to USAID/E 
comments Team leader (2) T. member (1) 
2- 
TMG edits final report 
Total LOE – 25 days of LOE for Team Leader and up to 22 days for the Team Member. A six￾day work week is authorized for work in Ethiopia. 
VIII. Team Composition and Participation
The Mitchell Group will provide two key core team members, 1) an expert with civil engineering 
and/or architecture expertise and experience in health center renovation and 2) an expert in 
health services delivery needs. Their qualifications are listed below. In addition, the two 
consultants may be joined by representatives from the MOH and HAPCO at the National and 
Scope of Work – Health Center Renovation Evaluation 
regional level and/or other development partners involved in renovation. Local Evaluation Team 
Logistics will be provided by the local sub-contractor hired by TMG. 
1. The Team Leader will be an international consultant with training in civil engineering or 
architecture as well as extensive health facility renovation program experience. S/he will lead 
the evaluation process in three or more weeks, spending two weeks in-country. The Team Leader 
will lead the team’s efforts, which will include conference calls with team members and 
USAID/E representatives, in-brief USAID/E on arrival, debrief USAID/E and HCRCP on 
evaluation findings, and develop a draft report to be left with USAID/E prior to departure, 
followed by a final report for USAID/E. 
The Team Leader will: 
• Discuss the team Work Plan for the assignment with USAID/E and finalize the work plan 
based on USAID/E comments. 
• Define assignment roles, responsibilities, and tasks for team members. 
• Oversee logistics arrangements in the field. 
• Participate in the TMG Team Planning Meeting (TPM). 
• Lead the preparation of and coordinate team member input, submitting, revising and 
finalizing the report. 
• Lead team meetings. 
• Coordinate and support the team on tasks and ensure that team works effectively. 
 
Team Leader qualifications: 
• An advanced degree (in civil engineering or architecture) pertinent to work with health 
center renovation. 
• Minimum 10 years of progressively responsible experience with recognized 
organization(s) in the design, implementation and evaluation of civil engineering 
programs, especially for renovation projects related to HIV/AIDS, TB, chronic diseases, 
and other PHC services at the health center level. 
• Demonstrated strong analytical, managerial and writing skills. 
• Leadership in coordinating activities; the ability to assign the team the appropriate 
responsibilities, and expert communication and interpersonal skills. 
• Ability to interact effectively with a broad range of internal and external partners, 
including international organizations, host country government officials, and NGO 
counterparts. 
• Fluency in English and proven ability to communicate clearly, concisely and effectively 
both orally and in writing. 
• Must be able to produce a succinct quality draft Final Report that gives direction and 
facilitates improvement for the HCRCP program. 
2. The senior level expatriate team member will be a health services delivery expert with 
(optionally) health services renovation experience with prior experience working in Sub-Saharan 
Africa. The consultant will assess the health services needs for HIV/AIDS and other chronic 
diseases to identify the appropriateness of the design, renovation and use to meet service delivery 
needs. S/he is responsible for writing sections of the report. The consultant will develop 
assessment tools to be used during site visits and analyze data. 
Consultant qualifications: 
Scope of Work – Health Center Renovation Evaluation 
• MD, BSN, MA, MS, or BA in Medicine, Public Health and/or behavioral or social 
sciences related to HIV/AIDS and other chronic disease services, and related MCH and 
PHC service needs related to health center renovations . 
• Five to 10 years of progressively responsible experience in the design, implementation 
and evaluation of service delivery needs in HIV/AIDS and other chronic diseases and 
related MCH and PHC programs related to structural and renovation of health centers. 
• Strong analytical, managerial and writing skills. Ability to interact effectively with a 
broad range of internal and external partners, including international organizations, host 
country government officials, and NGO counterparts. 
• Fluency in English. 
• Proven ability to communicate clearly, concisely and effectively both orally and in 
writing. 
Evaluation Logistics: Evaluation Logistics will be provided by the local sub-contractor hired by 
TMG with support staff who are fluent in Amharic, with a demonstrated: ability to be resourceful 
and to successfully execute complex logistical coordination; ability to multi-task, work well in 
stressful environments and perform tasks independently with minimal supervision; ability to 
work collaboratively with a range of professional counterparts. The local sub-contractor will be 
responsible for logistics, coordination and administrative support, and ensuring the evaluation 
are carried out seamlessly. TMG’s local sub-contractor staff will assist the Team in facilitating 
meetings, coordinating logistics and organizing site visits. As needed, the local sub-contractor 
will collect and disseminate background documentation to the evaluation team. TMG will be 
responsible to manage and direct the efforts of local sub- contractor. 
IX. Schedule and Logistics 
The in-country phase of the evaluation will be conducted in May with an in–country desired start 
date on or about 6 May 2008. See table above for Level of Effort and Attachment 2 Planning 
Calendar for the schedule. The local TMG Evaluation Logistics sub-contractor, in collaboration 
with the Mitchell Group in Washington and USAID/E Evaluation Coordinator and HCRCP, will 
arrange the partner meetings, site visits and debriefings in advance. Associated travel and per 
diem costs for non-USAID staff will be covered by TMG under the technical directive with 
USAID/E. 
X. Period of Performance 
Work is to be carried out over a period of approximately seven to eight weeks, beginning on or 
about (o/a) April 15, 2008 and concluding o/a May 29, 2008 (not including approx four weeks 
time for USAID/E (up to ten days) comment and completion of final editing of the Draft 
Evaluation Report by TMG (3 weeks). 
XI. Financial Plan 
A budget agreement between the USAID/Ethiopia and TMG will be reached and USAID/E will 
approve the evaluation activity by the TMG under the USAID/Ethiopia Evaluation Program. 
XII. Deliverables 
Scope of Work – Health Center Renovation Evaluation 
Prior to arrival (preferably two weeks): Team leader will develop a Draft Work Plan with 
evaluation methodology and field visit and interview schedule in consultation with the USAID/E 
CTO and USAID/E Evaluation Coordinator. The Draft Work Plan will clearly present roles and 
responsibilities, a planned interview schedule, and an analysis plan of who will be responsible 
for writing various sections of the report. 
Three days after Team arrival: Team meeting and in-briefing with USAID/E. USAID/E 
HAPN technical staff to review and comment on evaluation methods. The Draft Work plan will be 
presented and discussed with Mission Staff to ensure the assessment is on track and can be met on time. 
After agreement, the Draft becomes a Final Work Plan.
Prior to departure: Team makes presentation to USG PEPFAR staff, a separate presentation to 
HCRCP partners, and Team Leader submits a draft report in the format specified by the 
USAID/E Evaluation coordinator (See separate MS Word file for TMG Evaluation Report 
Guidelines) to USAID/E CTO - two hard copies and one electronic copy on CD ROM or flash 
drive.
After departure: Team leader submits final draft report content to USAID/E within one week of 
receiving comments from USAID/E. The report (not including attachments) will be no longer 
than 30 pages with an Executive Summary, Introduction, Methodology, Findings, Conclusions, 
and Recommendations in English in a format specified by the USAID/E Evaluation Coordinator 
in consultation with TMG. 
Upon final approval of the content by USAID/E, TMG will have the report edited and formatted 
within three weeks. The final report will be submitted electronically to USAID/E CTO and 
Contract Officer. 
TMG will make the results of its evaluations public on the Development Experience 
Clearinghouse and on its project web site unless there is a compelling reason (such as 
procurement sensitivities) to keep the document internal. Therefore, TMG will request USAID/E 
confirmation that it will be acceptable to make this document publicly available. If there are 
certain restrictions regarding specific parts of the report that should be removed from a public 
version due to procurement-sensitive information, TMG will produce a second version suitable 
for public availability. 
Scope of Work – Health Center Renovation Evaluation 
Attachment 1: Summary of Results 1-3 Technical Assistance/Activities (by quarter) 
(Source: Table 4 on page 7 of the Project Year Two Semi-Annual Progress Report) 
Technical Assistance/Activities Q1 Q2 Q3 Q4 Q1+Q2 
RESULT 1 
1.1. Conduct technical assistance meetings and 
visits with FMOH/PPD and RHBs 11 18 
 
29 
1.2. Conduct follow-on meeting with USAID￾Partners, USG and non-USG organizations or 
other donors 13 16 
 
29 
1.3. Assist FMOH/PPD and/or RHB develop HC 
renovation and health services tracking system 5 6 
 
11 
1.4. Assist FMOH/PPD and/or RHB implement 
HC renovation and health services tracking 
system 
 
1.5. Assist FMOH/PPD plan nation HCR 
Coordination meeting 3 1 
 
4 
1.6. Assist FMOH/PPD conduct national HCR 
Coordination meeting with RHBs and other 
stakeholders 
 
1.7. Assist RHBs leverage resources for HC 
renovation 8 19 
 
27 
1.8. Assist FMOH/PPD and HSD draft HC 
Renovation and Expansion Manual and HC 
Assessment Handbook 5 11 
 
16 
 TOTAL (Result 1) 45 71 116 
RESULT 2 
2.1. Provide TA to (or mentor) RHB and 
HC/Worde staff 22 35 
 
57 
2.2. Orient RHB technical staff (e.g., engineers) 
to updated HC renovations guidelines and other 
materials 5 9 
 
14 
2.3. Mentor RHB technical staff making 
engineering and supervision visits to health 
center(s) under renovation 3 44 
 
 47 
2.4. Provide assistance in using the HCR and 
health services tracking system 
 
2.5 Conduct annual workshop in project priority 
regions with RHB/Worde staff and other 
stakeholders involved in HCR 
 
 TOTAL (Result 2) 30 88 118 
 
RESULT 3 
3.1. HC engineering assessment completed 18 4 22 
3.2. HC construction on-going 17 22 22 
3.3. HC renovation completed and approved 
Scope of Work – Health Center Renovation Evaluation 
Attachment 2: Planning Calendar 
Monday Tuesday Wednesday Thursday Friday Saturday Sunday 
April 1 2 3 4 53 6 
7 8 9 10 11 12 13 
14 15 16 
Document 
review 3 days 
17 18 19 20 
21 22 23 24 25 26 27 
28 29 30 May 1
TPM at 
TMG/DC 
2 
TPM at 
TMG/ 
DC 
4 5 
Travel 
5 
Holiday 
6 
Start in Addis 
Stake holder 
meetings 
Continue tools 
development 
7 
Mission in 
brief 
Stakeholders 
meetings 
Key informant 
interviews 
9 
Full team TPM 
Site visits in or 
near Addis 
10 
Full team 
TPM 
Present 
work plan 
to USAID 
Site visits 
in or near 
Addis 
10 
Refine tools 
11 
Rest 
Travel to 
Asawa 
12 
Field work 
One team 
HC near 
Awasa 
 
13 
Field work 
One team 
HC near 
Shashemene 
14 
Back to Addis 
Travel to 
Bahia 
Regional office 
Bahir Dar – 
HC near Bahir 
Dar 
15 
Field work 
One team 
HC near Bahir 
Dar 
16 
Field work 
One team 
HC near 
Bahir Dar 
17 
Full Team analysis 
and writing 
18 
Rest 
19 
Full Team 
analysis and 
writing 
20 
Core team 
writing 
21 
Core team 
writing 
Debrief USAID 
 
22 
Debrief MOH, 
stakeholders, 
partners 
 Core team 
writing 
23 
Core team 
writing 
24 
Travel 
25 
Scope of Work – Health Center Renovation Evaluation 
Monday Tuesday Wednesday Thursday Friday Saturday Sunday 
26 27 
 
28 
Holiday 
 
29 30 Later
Response to USAID 
comments 
Expat T Member 1 
Team leader 2 
Total LOE 
Team 
leader 26 
 ex pat 
expert 25 
Scope of Work – Health Center Renovation Evaluation 
Attachment 3: Map of HCRCP Activity Locations in Ethiopia 
APPENDIX 2) TMG EVALUATION WORK PLAN F = Finish S = Start 
F = Finish 
S = Start 
APPENDIX 3) TOPICAL OUTLINE FOR THE SMALL GROUP DISCUSSIONS 
HCRCP-Ethiopia Topical Outline for Small Group Discussion at H.C. May 2008 
I. Preparation: 
 A. when first arrive on site, when with HC director, ask if we can arrange for a small 
group session and when this would be most convenient, set it up. 
 B. while doing the site visit and viewing the renovations (engineering and Health 
perspective) 
• Find or draw a picture of the external general layout/design of this Health Center 
(H.C.) 
• Determine which USG funded agencies support this site and for what purpose. 
• Determine if there have been any recent or current construction/renovations. 
 
II. Introduction to the group: 
Hello, I am (we are) here to conduct a needs assessment for the USG (United States 
Government) to help design a new project for Ethiopia 2009-2011. We know that this HC has 
been recently renovated or is now being renovated. In the new project 2009-2011, other HCs – 
similar to this one - will be renovated. 
We want your input – your ideas – regarding these renovations and if the design and function of 
this health clinic is meeting your current (and will meet future) health care needs in here and in 
Ethiopia in general. By design and function, we mean the buildings, rooms in the building of the 
H.C. where they are located, their size, how they are used and equipped. Where and how do you 
work in these buildings and rooms? Where do your clients come to get the H.C. services? 
Ö Show your diagram of this H.C. direct the group to the different buildings/ blocks 
including the toilets, incinerator, storage, admin, waiting areas (this orients the group to 
ALL aspects of the H.C…name only the blocks. Not each room within the block.) 
Ö POINT OUT THE RECENT OR CURRENT RENOVATIONS. 
We want to know if this design/layout of this H.C. has been improved both externally and 
internally. This could be something as simple as fixing or adding a sink, or adding a roof over the 
waiting area. It could be changing the location of the toilets or reorganizing some services. It 
could even be adding a new building. 
Today, we will ask for your ideas as an employee – a service provider – at this H.C. Also, we 
imagine that you, your family, your friends and neighbors come to this H.C. for prevention and 
treatment so we’ll ask you to think and respond as a client. 
Thank you for taking time out of your busy day to talk to us. 
III. May we now start by asking you your names and what you do as an employee at this H.C. 
Ö List names and their title/position here 
1) _____________________________________________________ 
2) _____________________________________________________ 
3) _____________________________________________________ 
4) _____________________________________________________ 
5) _____________________________________________________ 
6) _____________________________________________________ 
7) _____________________________________________________ 
8) _____________________________________________________ 
IV. Please describe to us of any changes in the design and function of this H.C. in the past 6-12 
month – or any changes which are in-progress. (Let group respond.) 
NB: If the group does not spontaneously identify recent or current construction/renovations, 
prompt them by saying “we see what looks like a new addition, building ….or we see there is 
construction going on here (point to diagram). 
V. [for H.C. where has been – or is currently – construction/renovation ask question IV, if not 
skip to V] 
¾ V-A. Were any of you involved in the recent, current changes to the design and function 
of this H.C. By “involved, we mean “did anyone ask for your input/ideas about what is 
needed before these changes were started/made? (Let group respond. If anyone says yes, 
ask them when & how.) 
¾ V-B. What do (will) these changes mean to you as a provider – an employee performing 
your duties. For example, does it make your job easier or harder, safer or less safe, more 
or less efficient, more or less private, etc? (let the group respond) 
NOW THINK OF YOUSELVES AS CLIENTS, think of your family, friends and neighbors as 
they come here for health care. This could be to get vaccinations for your child, get pre-natal 
care, give birth here, visit the TB clinic, get treatment when someone is “sick”, etc. 
¾ IV-C What do (will) these changes mean to you as a client? For example, does it make 
you feel more or less comfortable, your waiting time shorter or longer, your privacy 
better or less, more likely to come here to give birth, etc. (let the group respond) 
VI. Shat - or what other - changes to the design and function of this and similar H.C. would you 
suggest be made and why 
 Vi-A. as an employee (let the group respond) 
 Vi-B. as a client (let the group respond) 
VII. (if time permits and not already covered, if already covered, skip to VIII.). 
Now let’s look a just a couple of the specific situations/services to make sure we didn’t miss 
anything. What changes do you suggest be made. (Go one by one and let group respond.) 
 VIi-A. waiting to be seen (before and after registration) - as provider; as client 
 VIi-B. waste/needle/blood disposal and incinerating - as provider; as client 
 VIi-C. coming for ART services - as provider; as client 
 ViI–D. coming for Tuberculosis (TB) services - as provider; as client 
 VIi-E. coming to give birth - as provider; as client 
 VIi-F. Other – anything you feel is important - as provider; as client 
VIII. are there international organizations working with you to construct/renovate this particular 
H.C. or to help you improve the quality of your services? if yes, which ones and what do they 
do. (let the group respond) 
IX. Pass around a copy of the maintenance checklist and ask the group if they have ever seen this 
or anything like it. Translate and read some of it to them…e.g. “daily” and read the 1st two items, 
then “monthly” and read the 1st 2 items, etc. 
IX-A. If anyone has seen this or something like it, ask “do you use this? Do you do these 
activities on this schedule, etc.? 
IX-B. if no one has seen this or anything like it, ask “would a list like this (translated) be of any 
use to you? To whom specifically? How? 
X. Thank everyone for their time. 
APPENDIX 4) PHOTOS FROM EVALUATION TEAM SITE VISITS 
Schromeda (Addis Ababa). Renovation in progress; 
very good work thus far and coordination with 
MSH/RPM+; HC Admin officer excited about 
renovation, 2nd story being added on which will allow 
admin/finance to move out of areas which should be and 
will soon be for service provision. Would have been 
better to include shelter for clients waiting (none 
planned); could have had workers cleaning mud and 
debris off steps as a safety measure. Administrator who 
signed MOU had not seen CAC maintenance form and 
did not have copy of the renovation blue print nor 
HCRCP’s master plan on site.
View of 2nd story going up over existing 
building; this is the only 2nd story addition 
observed. 
Good construction! Would have benefited from some safety 
measures – cleaning mud and debris from steps 
to 2nd floor 
Client waiting still a problem; blocking 
corridors. 
A view from the addition of a 2nd story 
admin/finance. 
Koteba, (Addis Ababa). Renovation in progress; very 
good work thus far! Service provision building where 
clients were previously waiting will improve/increase 
services to HIV & AIDS clients. Adding new pit latrines 
– very welcome to both clients and providers! Would 
have been better to include shelter for clients waiting; 
could have had workers cleaning mud and debris off 
steps as a safety measure. Administrator who signed 
MOU had not seen CAC maintenance form and did not 
have copy of the renovation blue print nor HCRCP’s 
Renovation in progress; very good work; 
would have benefited from some safety 
measures & covered client waiting area. 
master plan on site.
This major renovation adds exam rooms where 
client waiting used to be; very welcome and 
will increase/improve service provision. 
New Pit Latrines – very welcome! 
Work in progress Improved access will be welcome. 
Bole 17, (Addis Ababa). Small renovation completed 
(waiting room/records and added sewage lines which 
were missing. This complemented MSH/RPM+ and 
Clinton Foundation work. Worked looked very good 
Upon arrival found a new building being built that was 
not on HCRCP master plan that was developed. We 
were asked to leave because CAC had not informed 
them in advance of our visit: we could not complete the 
observations. 
CAC renovated waiting room and records 
room 
Sewage added MSH/RPM+ and Clinton’s contribution 
New building was done but it is not clear by 
whom and for what principal purpose; it is not 
part of HCRCP master plan 
Clean incinerator but would be better if fenced 
in to keep passers-by safe from accidental 
exposure. 
Yergalem. Renovation completed. Done at site 
where MSH/RPM+ had added storage. New 
washing sink added but not functional because no 
drain. Staff did not know of guarantee; did not 
know to contact CAC for problems. Did not 
initiate contact with district. They did not have a 
copy of the renovation blue print nor HCRCP’s 
master plan on site. From SGD information about 
half of clients state they are happier with 
cleaner/painted walls, covered waiting and sink; 
other half stated there was no notable change. 
A cover to the client waiting area was added at this 
site. 
Added a covered passageway New sewage canals; could have been better 
construction. 
A new washing sink (top) was added but 
is not functional so it can’t be used ; The 
old sink is still being used. 
A sky-light of appropriate technology added. 
Ceiling construction with poor quality 
material 
The site staff would have liked that the 
toilet and shower to be renovated and 
functional. 
New water tank 
Still a problem with access to water. Water was leaking into this room for which the 
HCRCP project had installed the guttering 
improperly 

Shashamene. Renovations completed. 
Coordinated with MSH/RPM+ and even added 
ventilation windows in the storehouse. Added 
covered waiting area outside records room; 
unfortunately, this is not where clients wait so 
they still block corridors and are waiting in sun 
or rain. New divider in delivery room in place 
for more privacy. Faucets and sinks incorrectly 
installed with poor fixtures. Water not running 
at sinks near toilet. Placenta pit open (no cover); 
damaged ceiling from leaks. 
Employees quite happy with renovated 
records room, though they wished that new 
shelves had been part of the renovation. 
New cover for waiting area would be welcome 
but clients wait in front of the doctor’s offices. 
Would have been better to extend roof where 
clients wait to protect from the sun and rain 
Ceiling damage from faulty roof/gutter 
installation. 
New Pit latrines also a welcome to the 
employees, but sinks don’t work. 
No running water at faucets installed at latrines 
(and fixtures not installed correctly) so cannot 
wash hands. 
New privacy wall in maternity gives more 
privacy. 
The placenta pit has no cover Water not running in the lab due to 
improperly installed sink and faucets. 
Batu. Renovations completed. A sheltered 
waiting area added; would be more useful if 
some type of seating was included. Some sinks 
installed do work but hazardous exposed 
electrical wires are nearby. Sinks at toilets not 
functional; installed incorrectly. Flooring 
added of poor quality and difficult to clean; 
new toilets but no water supply. Sinks 
installed incorrectly and of poor quality. 
a 
Showers and sink installed in maternity but 
never used because contractor had not turned 
on water: no one knew there were shut off 
valves 
 
A 
A a 
Modjo. Renovation in progress. water, drain lines not being installed correctly 
Improper installation Improper installation and no water 
Comment [U1]: The next series of 
photos and captions is unintelligible due 
to formatting problems. 
Shaded waiting area extended. Site management & maintenance (not HCRCP 
responsibility); needles lying around on 
ground. 
But at the time, the incinerator was not 
repaired and not used (staff were told because 
of USG restriction) 
Sink drain not installed correctly. 
Adet. Renovation completed but with 
problems. Staff knew of guarantee at this site. 
New covered waiting area. 
Outside flooring poorly installed and cracking. No water at sinks 
No water at sinks near new toilets. Ceilings of poor quality material and 
construction. 
Bahir Dar. Completed but with many 
problems. 
Welcome new renovations to maternity area. 
Storage shed built to free up service delivery 
area. 
Did not repair walk after cutting out for water 
lines. 
Placenta pit repair not done well and cracking 
already. 
Sewage system not functional 
Doors knobs not installed well New water tank but not functional/connected 
Large pit left open and incomplete for more 
than six months where septic tank was to go. 
Sink not working. 
Faucets and sinks not installed correctly Damage to ceiling where roof repair and gutter 
not installed correctly. 
Poor quality construction material 
Koladiba. Renovation in progress; early work 
looks good, better than average. MSH/RPM+ 
building on site as well; HC taking 
responsibility for site safety by having staff 
move material out of the way. Extension and 
renovation of maternity ward will be very 
welcome. 
Water and drain installed properly 
Site staff working. 
Addis Zemen. Renovation completed but 
many problems. 
Inside floors are difficult to clean. Doors hung poorly; don’t lock; HC had to bring in 
someone for alternative mechanism to lock 
Water available but fixtures and sinks not 
installed correctly. 
Windows welcome and installed well Renovated walls cracking 
MSH/RPM+ cut a key water line for their 
renovation, leaving site without water in 
places. 
This sink not functioning for some time so back up 
water supply used. 
Sink installation at site. Water not connected to toilet (but it was to shower) 
Injibara. Renovation completed but 
still some problems. 
New toilets are welcome. No water to sinks and installed incorrectly. 
Nice sewage. Nice roof construction but the downspout was put right 
through the center without flashing so water drips 
through. 
Installed new water tank and stand 
which is nice. Project put a smaller 
water tank. 
 
APPENDIX 5) LIST OF PEOPLE MET AND INTERVIEWED 
EVALUATION TEAM MEETINGS
DATE LOCATION ATTENDEE TITLE
USAID Marie‐Claire Sow CO
5/6/2008    Michael Rossman CO
MOH/Addis Sampson D/Chief PPD
USAID Omer Ahmed Omer D/Team Leader/HIV
   Xerses Sidhwa    
   Sam Clark HAPN/BES
   Marie‐Claire Sow CO
   Meri Sinnitt Health Office Chief
   Cynthia Shartzer CO
5/7/2008
   Kassahun Deneke SIA/Health Team
CROWN AGENTS   Paul Wolstenholme ENGINGEER
   Sallehunae Merahi ARCHITECT
   Zelalem Tiwneh ENGINEER
   Efrem Tiruneh ENGINEER
   Noel McIntosh Chief of Party
        
MOH/PPD Rik Nagelkerke PPD
5/8/2008
   Alemayehu Shewarega PPD
USAID Sam Clark HAPN/BES
   Melissa Jones HIV Team Leader
   Kibru Mamusha PRM
   Muluken Chanie PRM
        
5/9/2008
GTZ/IS Log. Office Scott Saarlas GTZ/ARCH
5/10/2008 Hotel Rob HSDPIII/ENG
Awasa RHB Ato Gizachew Kebede   D/Dir. RHB 
   Ato Dereje   PPD Head
        Yirgalem HC  Sr. Abebech Yacob         Health Center chief
   Tadesse (nurse)             EPI Coordinator
   Hibist                    MOH Coordinator
    Endrias Lemma      Security guard
    Beyenech Ayele    Janitor
   Yoseph                   ART
   Fantaye Teshome            Finance and Admin.
5/12/2008
    Mintwab               Sanitation
Shashamene HC Ato Tadesse Jubira Nurse
Batu HC Mame Tura HC Director
5/13/2008
        
EVALUATION TEAM MEETINGS
DATE LOCATION ATTENDEE TITLE
        
Mojo HC  Ato Amare   HC Director
   Sr. Arefa Adem Nurse
    Sr. Senait Tasew Nurse
Bahir Dar RHB Dr. Asrat Genet Dir. RHB 
Bahir Dar HC Ato Mengistu Kebede   HC Director
Adet HC Ato Melkamu   HC Director
   Asmamaw Adamu    Pharmacist
   Dires Finance and Admin 
5/14/2008
   Abiyot Birhane Laboratory
Kola Diba   Ato Habtamu   HC Director
        
Addis Zemen   Ato Ademe Asrat Alemu HC Director
5/15/2008
   Ato Muluye Zeleke   Druggist
Dangela HC Ato Hussein HC Director
   Sr. Yeshi Kassa Wubishet Nurse
        
Durbete  None    
        
Injabara Ato Aragaw  Head Nurse
5/16/2008
   Sr. Selam   Nurse
USAID Omer Ahmed Omer D/Team Leader/HIV
        
   Sam Clark HAPN/BES
   Marie‐Claire Sow CO
   Meri Sinnitt Health Office Chief
   Cynthia Shartzer CO
5/21/2008
   Kassahun Deneke SIA/Health Team
USAID/CAC Sam Clark HAPN/BES
   Marie‐Claire Sow CO
   Cynthia Shartzer CO  
        
   Paul Wolstenholme ENGINEER
   Sallehunae Merahi ARCHITECT
   Zelalem Tiwneh ENGINEER
   Efrem Tiruneh ENGINEER
5/23/2008
   Noel McIntosh Chief of Party
APPENDIX 6) DESCRIPTION OF FIELD GUIDE ASSESSMENT DOCUMENT 
Stage One – Arrive at the Health Center 
Upon arrival, introduce yourself (and the team members) to the HC in-charge and the woreda 
health officer. This should also include meeting available professional staff – nurses, midwife 
(and doctor if it is a large HC). As part of the introductions, explain the purpose of the visit, how 
the HC was selected and expected results of the assessment. 
During the discussion, collect basic HC data (e.g., age of buildings, population served and range 
of healthcare services currently being provided and planned). Record this data on the Health 
Center Assessment and Renovation Works Summary Report form. Also, ask about any 
problems or issues the HC staff consider important and note this on the Health Center Data 
Collection form (A.2). Once this task is completed, request the HC in-charge, or her/his 
designated representative, and the woreda health officer to accompany the assessment team on an 
initial (preliminary) walkthrough of the HC compound. 
Stage Two – Do Initial Walkthrough 
The objectives of this walkthrough are two fold: 
1. To allow the HC in-charge (or designated representative) to show the assessment team 
the major problem areas and to discuss other issues 
2. For the team to learn what each building is used for and the location of all departments or 
units at the HC (e.g., administration, registration and waiting areas, and outpatient department as 
well as other rooms/areas such as labor and delivery or antenatal care, the pharmacy[s] and 
special stores) 
During this initial walkthrough, make a quick sketch detailing the layout of buildings and the 
activity/function of each room (e.g., labor room, sluice room or laundry area). It is important to 
gain access to each and every room, especially those identified as stores, which invariably are 
locked. 
Specific tasks that should be undertaken during this initial walkthrough include: 
• Take photographs of those rooms that have been unlocked during the tour (or may be locked 
immediately after the HC-in charge has left the room) because access may not be available 
later on. (Preferably use a digital camera with a wide-angle lens to document the findings.) 
• Identify those areas/rooms/buildings that require attention/renovation or where there is 
significant deviation from the current FMOH HC standards 
• Observe and record the relationship between HC staff and patients as well as the general 
treatment of patients (e.g., friendly and helpful) offered by the HC staff on the Health 
Center Data Collection form).
Stage Three – Conduct Detailed Walkthrough and Take Measurements 
On completion of the initial walkthrough, a second more detailed walkthrough is required. 
Generally, only the assessment team members, accompanied by the RHB engineer, if available, 
are involved in this walkthrough. The objectives of this second walkthrough are to: 
 
• Upgrade the initial sketch(es) of the HC compound noting the location of all buildings, walk 
ways, access road(s) and out buildings (e.g., dry pit latrine), water storage tank, septic tank 
and placenta pit; 
• take relevant measurements (dimensions) of all buildings; 
• identify physical (structural) problems or damages, including those buildings damaged 
beyond repair (e.g., burned out or old, mud-walled buildings) that should be destroyed; 
• locate water pipelines, the waste water disposal network and sanitary systems (toilets, dry-pit 
latrines, septic tank[s] and placenta pit) and fire box; 
• determine potential areas within the HC compound for future expansion (e.g., site for 
dedicated general stores); and 
• check patient, healthy client and staff flow patterns relative to various HC functional areas 
(i.e., registration, cashier, waiting area[s], preventive and clinical services, pharmacy, 
dispensary and administration). 
During this walkthrough: 
• Take additional photographs of damages, problem areas and areas where alterations are 
needed. (Be sure to include general shots [15-20] of the HC compound, including access to 
the HC and any space available for future expansion.) At all HCs, take specific photographs 
of the labor room, delivery room(s), and sluice and clean rooms (if present). 
• Using the Damages Assessment Checklist, identify the appropriate repair method for the 
observed damages and record the damages on the Damages Assessment Report sheet and 
Damages Assessment Take-Off sheets. Cross-reference the room title on the floor plan 
layout sketch with the damage checklist according to the following sequence: 
a) Ancillary facilities and external works, including manholes and pavement around the 
buildings 
b) External walls, roof and overhanging parts, with particular attention to any structural 
damage to foundations, ground beams, columns, tie beams and load bearing walls 
c) Windows, noting on the back of the checklist their size and type 
d) Doors, noting on the back of the checklist their size and type 
e) Fixtures and built-in cabinets or cupboards, noting on the back of the checklist their size 
and type 
f) Location of central water supply to HC (e.g., ground and elevated water tank) 
g) Location of sanitary fixtures (e.g., sinks and drains, toilets and utility drains, and septic 
tank[s]) 
h) Location and type of electrical fixtures 
i) Walls, floors and ceilings finishings 
• Take additional notes detailing such items as the condition of the access road, any access 
restrictions for construction deliveries or heavy trucks, and any space or site conditions that 
could limit potential expansion. (Attach another sheet if all the space is taken up by notes on 
doors, windows and other items.) 
• Take two water samples from an outlet closest to the point of entry to the HC or from the 
storage tank (reservoir); one sample will be used to identify possible bacterial contamination 
(48 hour test) and the second to measure lead levels (10 minute test). 
Finally, throughout this walkthrough observe patient, healthy client and staff flow patterns to 
identify crowded and high traffic areas. As part of this process, note distances between 
registration and waiting areas, OPD, examination and treatment rooms, cashier and pharmacy. 
Where possible estimate the number of patients waiting at each area in order to identify 
actual/potential bottle-necks. Of particular importance, observe where HIV/AIDS and TB 
patients are examined and treated (e.g., near the entrance to HC or buried within HC compound). 
Then, based on these observations and the data collected produce the following: 
• A detailed set of sketches (“as built” drawings) of the entire the HC compound (external 
buildings and inter-building dimensions) based on the preliminary sketches made during the 
walkthroughs. This drawing should include the perimeter boundary, land levels and any 
natural factors that potentially could affect renovation recommendations. The sketch of each 
building should include the major dimensions and locate fixtures, such as sinks or built-in 
cabinets, desks or other fittings, and electrical outlets on the appropriate schematic sketch 
sections. Label the rooms (including verandas and corridors) and designate all windows and 
doors separately. 
• A floor plan layout sketch with at least one elevation for each building. The drawn floor 
plans, sections and elevations should be sufficiently detailed to define the floor finishing, 
areas of damage and elevations. 
•
Stage Four – Review Data Collected for Completeness 
Prior to departing the HC, it is essential that the assessment team leader: 
• spends time discussing the findings and observations with the other team members; 
• conducts a through review of all data collected on the various forms, 
checklists and work sheets; 
• goes over the sketches and reviews all measurements for completeness; 
• scans the digital photographs to ensure that an accurate and complete representation of the 
HC “as built” has been obtained; 
• is satisfied the major damages and problems have been identified; 
• develops and/or reviews the preliminary set of recommendations with the team members; and 
• has a clear picture of what is needed to improve the function of the HC based on revised 
clinical (patient, healthy client and staff) flow patterns as well as recommended hygiene 
practices and environmental health control measures. 
Stage Five – Debrief and Depart the Health Center 
Before exiting the HC, the assessment team should meet with the HC in-charge, or designated 
representative, and woreda health officer (if available). The purpose of the debriefing is to 
summarize the key findings (problems and deficiencies) and suggest any immediate actions that 
could be implemented to improve patient flow and patient/staff safety (e.g., open windows and 
doors and/or re-arrange furnishings to improve cross ventilation). (See A.6, Environmental 
Health Control Guidelines for TB, for details.) In addition, the team leader should briefly 
discuss those repairs or renovations that will most likely be in the final set of recommendations 
(e.g., leaking roof over OPD rooms or no water to examination or treatment rooms.) In addition, 
it should be explained to, and accepted by, the HC in-charge that re-allocation of space use (e.g., 
changing the location of services within the HC compound to improve patient and staff safety) 
will be a major consideration in the final set of recommendations. 
As part of the exit debriefing, the team leader also should review the content of a sample 
memorandum of understanding (MOU) with the HC in-charge and woreda health officer (if 
available) putting special emphasis on the potential availability of: 
• matching funds from the RHB or woreda health office, and/or 
• in-kind contributions by the HC (e.g., provide new benches, desks or shelving and/or labor 
for moving furniture). 
As a final discussion topic, the Health Center Maintenance Management Checklist should be 
reviewed. Gaining acceptance of the need to implement the simple maintenance activities in this 
checklist is most important. Without benefit of routine maintenance, improvements to the water 
supply, waste water disposal, sanitation and electrical systems will be lost in a short time (sinks 
are again plugged and latrines overflowing). 
Stage Six – Consolidate Data and Write Reports 
As clearly indicated from the above the contractor has prepared required documents to undertake 
the assessments which can be utilized in the future. The reports, forms, checklists, and template 
formats developed to assist the HC assessment team collect and record data in a systematic way, 
simplify the data collection process and provide consistency in data collection are listed 
herewith. 
A.1 Health Center Assessment and Renovation Works Summary Report Form 
A.2 Health Center Data Collection Form 
A.3 Damages Assessment Checklist 
A.4 Damages Assessment Sheets 
A.5 Damages Take-Off Sheets 
A.6 Environmental Health Control Guidelines for TB 
A.7 Sample Health Center Memorandum of Understanding (MOU) 
A.8 Health Center Maintenance Management Checklist 
A.9 Drug Destruction Certificate 
A.10 Sample Health Center Assessment Site Visit Report 
APPENDIX 7) LEVELS OF RENOVATIONS WITH RESPECT TO HEALTH CARE 
For the purposes of this evaluation, one must understand how the HCRCP differs from other 
types of renovation. In addition to the health-care component, it is renovation and therefore 
different than new construction which can be done at fixed cost within several variations. This is 
necessary because the language of the HCRCP drew the link between the HC renovations and 
improving the quality of the health care delivered at the site. Though the HCRCP did not adopt 
performance indicators to measure this, the SOW for evaluation asked to assess this. Thus, we 
have identified three levels of HC renovation - with respect to health care delivery - in which to 
assess the contribution of the HCRCP. The lower levels are subsets of (and therefore included 
within) each subsequent higher level). 
Level 1: From the health care perspective, the most basic renovation would be to restore a given 
site to its’ original design and functionality. That is, there is no change in the design; water and 
sinks where placed should work as originally designed, roofs repairs, walls repaired, toilets 
repaired. 
Level 2: includes all of level 1 but adds to this “reorganization” of the health care delivery within 
the limits of the existing architectural layout of the buildings. Thus, switching the services in 
one location to another location so that patient flow, privacy, infection prevention and 
management is improved. This is particularly important because PEPFAR is one of the primary 
vehicles (with national governments, WB/MAP and GFATM) for adding ARV and TB services 
where they have not previously existed through either mainstreaming or integration. 
Level 3: includes level 2 but adds “modifications” to the equation. Modifications can included 
changing/adding walls, doors, expanding building capacity (building up or out on an existing 
structure), covering waiting areas, increasing water availability (volume and location), etc. 
Although there are no performance indicators for this project other than (a) assess 100 HCs and 
(b) renovate 50 sites, it is clear from the contract that level three assessments and renovations are 
expected. In the contract, the first 2 of 17 bullets under the heading [C.4. Illustrative List of 
Renovations] are: 
• Add or demolish wall partitions whenever necessary to ensure proper functions and 
facilitate patient flow. Install a patient reception counter at the central location of each 
clinic to facilitate patient flow. 
• Covering areas between multiple structures to improve client comfort and client flow to 
create continuous space for patient counseling rooms, waiting areas, administrative 
offices or secure storage 
APPENDIX 8) SUMMARY OF KEY PROJECT ACTIVITIES WITH STATUS NOTES 
Activity description
Expected as per 
1=PMP; 2=contract, 
3= work plan 
Unexpected Status
PROJECT MANAGEMENT ACTIVITIES 
PMP approved 1,2,3 No Completed (and approved 
although it is insufficient) 
Existence of a project strategy 1,2,3 No Completed (and approved) 
Work plan approved 1,2,3 No Completed (and approved for 
PY1; not for PY2) 
Semi-annual progress reports 1,2,3 No Completed 
Quarterly reports with budgetary and 
expenditure information 
1,2,3 No Unknown 
Direct renovation 50 HC (20+30) 1,2,3 No In-progress 
T.A. to 100 HC assessments including 
other PVO and donors 
2,3 No In-progress 
RESULT 1 (COORDINATION MECHANISM) ACTIVITIES
Establish a coordination, planning and 
synchronization “hub” 
1,2,3 No Details provided below 
Set up and preside over a coordination 
committee; a tender committee 
1,2 No Replaced by Coordination 
“Plan” 
Develop a consensus HC renovation 
harmonization and coordination plan 
3 No Unknown – assumed in 
progress 
HC mapping/tracking system (a 
common and simple HC tracking system 
that can be adapted for other uses (e.g., 
tracking progress in the conversion of 
health stations to HCs) 
1,3-PMP No In-progress 
Conduct national coordination meeting 
PY1 
3 No Completed 
Conduct a survey in preparation of the 
donor coordination meeting 
3 No Completed 
Conduct (5 regional) workshops 2,3 No Not started 
RESULT 2 (TECHNCIAL ASSISTANCE) ACTIVITIES
Engage 1 or more A&E firms to assist 
with general design 
1,3 No Completed 
Identify T.A. requirements to 
FMOH/PPD and RHBs and agree on TA 
program 
3 No In-progress 
Mentor RHB technical staff making 
engineering and supervision visits to 
HCs (part of T.A.) 
3 No In-progress 
Other T.A. to FMOH/PPD, other 
agencies, RHB, Woreda level, HC 
personnel 
2,3 No In-progress 
RESULT 3 (RENOVATION OF HEALTH CENTERS) ACTIVITIES
Revised Guideline for FGOE HC 
standards 
3 No In-progress 
Activity description
Expected as per 
1=PMP; 2=contract, 
3= work plan 
Unexpected Status
Inventory of engineering assets 3 No Unknown 
Procurement and contract management 
processes document 
3 No (see below) 
Guidelines2 for HC renovation and 
expansion 
1,3 No In-progress 
Health center assessment handbook 
(field guide) 
1,3 No In-progress 
Road map (to support development of 
standardized approach) 
1,2 No Unknown 
Skills transfer to local sub-contractors 
through mechanisms such as on-the-job 
training 
1 No Not started 
Project safety officer; regular safety 
promotion meetings 
1 No Not started 
A common approach to HC renovation 
being used by the RHBs 
3-PMP No In-progress; 
An A&E firm with at least 10 engineers 
experienced in use of the approved 
engineering assessment guidelines and 
other HC renovation materials; and 
3-PMP No In-progress 
(will not be completed) 
 
2 The guidelines for renovation and expansion includes all of the following elements, each of which was developed 
or adapted for this project: 
Contracts and Contractual Tools 
Pre-qualification documents for works contractor selection (adapted from FGOE standard) 
Tender document for contracting with A&E firm (adapted from FGOE standard) Tender document for construction works (adapted from
FGOE standard) 
Tender document for construction works (adapted from USAID) 
Sample contract agreement (adapted from FGOE standard) 
Master bill of quantities (BoQ) document 
Normal BoQ document 
Memorandum of Understanding for Regional Health Bureaus 
Memorandum of Understanding for Health Centers and Woreda Health Offices 
Checklists and Forms 
 Health Center Maintenance Management Checklist (self-administered) 
 Damage Assessment Checklist 
 Reporting Forms 
• Progress (Supervision) Report 
• Health Center Assessment Site Visit Report 
Construction Works Certificates and Forms 
Health Center Data Collection Form 
Health Center Site Handover Form 
Damage Assessment Checklist 
Damage Take-off Sheet Form 
Initial Payment Certificate 
Final Payment Certificate 
Provisional Acceptance Certificate 
Final Acceptance Certificate 
Activity description
Expected as per 
1=PMP; 2=contract, 
3= work plan 
Unexpected Status
Approximately 10 private-sector 
construction companies located 
throughout the country with experience 
conducting HC renovation works 
consistent with the approved guidelines. 
3-PMP No In-progress 
ADDITIONAL ACTIVITIES REQUESTED
Review and comments on Standards of 
Care for Health Centers 
1,2,3 - at request of Dr. 
Kessetebirhan 
No Completed 
Brief report; Health center space 
requirements for 2008 
Yes, At 
request of 
USAID CTO 
Completed 
Primary HC assessment findings from 3 
regions 
Yes, At 
request of 
USAID CTO 
Completed 
Mapping and Capacity Building Report Not applicable because it is a proposal to USAID for additional funding 
Report on Storage Capacity 
Assessment for Ethiopian MOH 
Facilities 
1,2,3 No Draft 
Brief update report: Seconding STTA 
consultants to FMOH/PPD and RHB 
Yes, At 
request of 
USAID CTO 
Completed 
Reviewed the design being used to 
convert 262 health stations (HSs) to 
“nucleus” HCs in Oromia region,. 
1,2,3 At the request of 
the Oromia RHB/PPD, 
No Completed 
Developed master plans for leveraging 
additional resources to renovate three 
HCs – Shromeda (Gullele) and Arada 
(July 2007). 
1,2,3 Addis Ababa City 
Administration and the 
respective Sub-Cities 
No Completed 
Developed a master plan for 
pharmaceutical stores at HCs. 
1,2,3 (In collaboration 
with MOH/IST and 
RPM+) 
No Completed 
Develop ToR and job descriptions & 
contracts for the (seconded) STTA (8 
total) 
Yes, At 
request of 
FMOH/PPD & 
RHB. 
Completed 
Review resumes and jointly interview 
candidates for STTA (8 total) 
Yes, At 
request of 
FMOH/PPD & 
RHB. 
Completed 
Two-day orientation workshop for STTA 
before posting. (8 total) 
If done, will be part of 
expected TA 
No Completed (as per USAID) 
APPENDIX 9) TABLE OF KEY PROJECT ACTIVITIES WITH STATUS AND NOTES 
Description of 
Key Project 
Activities 
Status Status Explanation and Notes 
PMP approved Completed 
(and 
approved 
though 
insufficient)
Even though there are only 2 performance indicators (# sites 
assessed and # sites renovated), there are no operational definitions 
for these. On this alone, the PMP should not have been approved 
(but it was). Furthermore, the contract states “The Contractor will 
utilize standard PEPFAR indicators and propose additional 
indicators for assessing performance of activities accomplished”. 
CAC did this by reporting “summary of TA activities” as done in their 
PY1 annual progress report. This indicator should have been 
operationally defined either in the PMP and/or report but it was not 
done. The evaluation revealed that no standard definition was 
applied rendering these data (# of TA activities) incomparable and 
not able to be interpreted. 
Existing strategy Completed 
(and 
approved) 
The existing strategy presented with the PY1 work plan does 
present elements of a strategy and it is as good as could be 
expected since it was presented within the first 60 days of the 
contract start date. USAID should have asked for a revised, full 
strategy at start of PY2. 
Work plan 
approved 
Completed 
(and 
approved for 
PY1; not for 
PY2) 
Work plan outline with description was presented to the team though 
it is indicated that an MS project work plan was regularly provided to 
USAID by CAC with all the detail needed to monitor HCRCP 
progress: the evaluation team did not see these. More importantly, 
CAC submitted the PY1 work plan on time but it went through 3 
more revisions with the final being submitted 9 months after HCRCP 
began (June07) and USAID approving this at the end of PY1 Sept 
2007 (Sept07). 
Semi-annual 
progress reports 
Completed CAC went beyond their requirement in reporting on their progress. 
Not only did they include extra analysis, tables and explanations in 
their reporting, but they submitted interim reports all of which are 
highly commendable. However, the reliability of the data in the 
reports is not satisfactory for some of key reporting elements (see 
below); furthermore there is discordance between certain reports 
making it difficult to track true progress. 
Quarterly reports 
with budgetary and 
expenditure 
information 
Unknown Not provided to evaluation team; probably because it contained 
predominantly budget and expenditure information. This was not 
critical to the evaluation, though it does make it difficult for the team 
to attribute weights to the expected outcomes and deliverables 
especially for Result One. For example, if only 3% of the total budget 
for Result One was allocated for the “mapping and tracking” system, 
the product would be assessed differently than if 50% of the budget 
were attributed to it. 
Direct renovation 
of 50 HC (20+30) 
In-progress Recently reduced from 50 to 45 health centers due to inflation 
increasing the cost of material and transport. Yet the report 
(September 29, 2006 to March 31, 2008) states that 45 HCs located 
in the four regions and Addis Ababa have been renovated with the 
remaining 5 to be renovated by/before July 2008 (50 total). We 
visited sites that were not completed and for the same time period, 
the combination progress reports covering Q1-Q4 PY1 + Q1-Q2 PY2 
states that (0+23) only 23 have been completed. There a 
discordance in results reported and observed. 
TA to 100 HC 
assessments 
including other 
PVOs and donors 
In-progress Project Activities Report (September 29, 2006 to March 31, 2008) 
has an annex (table) stating 100 “completed as of March 31st; but 
the executive summary states that 81 were completed. Combining 
tables on page 9+11 and other reports show 81. This makes the 
reporting data unreliable. 
Establish a 
coordination, 
planning and 
synchronization 
“hub” 
See elements 
below 
It is only indicated on the 1st page of #3 as part of introduction, but is 
not found further in the document nor in the outline of the work plan. 
From the documents, the 3 key components of the “hub” were 1) 
steering/coordination committee(s), 2) mapping/tracking system and 
3) meetings and workshops. 
Set up and preside 
over a coordination 
committee; a 
tender committee 
n/a (replaced 
with below) 
Sometimes referred to as a “steering committee”, replaced with 
developing a coordination plan. 
Develop a 
consensus HC 
renovation 
harmonization and 
coordination plan 
Unknown – 
assumed in 
progress 
Described as a “harmonization consensus” document. See above 
consensus HC renovation harmonization and coordination plan 
(below) 
HC mapping/ 
tracking system (a 
common and 
simple HC tracking 
system that can be 
adapted for other 
uses (e.g., tracking 
progress in the 
conversion of 
health stations to 
HCs); 
In-progress Also the Progress Report of October 1, 2008, were to have 
completed in PY1. Sometimes this was called a renovation tracking 
system and/or a leveraging system which was confusing because it 
leads the reader to believe they are different products. 
Conduct national 
coordination 
meeting PY1 
Completed The evaluation team has not seen the report of this meeting 
Conduct a survey 
in preparation of 
the donor 
coordination 
meeting 
Completed Analyze and present results. 
See “Leveraging Donor Resources for HC renovation in PEPFAR 
priority regions” (could have been more appropriately titled ”Results 
of survey; GOE, Donors, Implementers interest in establishing a 
HCR coordination and resource leveraging hub”)
Conduct 5 regional 
workshops 
Not started For Result Two, TA (“at least 1 in the contract”, committed to 5 in 
work plan. Was to orient groups to guidelines and show how to use 
the tracking system. Review work plan and progress for PY2 when 
available. 
Engage 1 or more 
A&E firms to assist 
with general 
design 
Completed There is a need to train the firm(s) in their approach to assessment. 
This was done for one A&E firm. 
Identify TA 
requirements to 
FMOH/PPD and 
RHBs and agree 
on TA program 
In-progress T.A. was finally provided with the extra money available to CAC in 
Dec. 07 due to a revised (reduced) indirect cost-recovery (NICRA) 
rate. With this money, STTAs were hired and placed with the RHB 
and FMOH/PPD. This was later changed and some of these STTAs 
where retained at the FMOH level. 
Mentor RHB 
technical staff 
making 
engineering and 
supervision visits 
to HCs (part of 
T.A.) 
In-progress This is an implied output indicator. The number of RHB staff 
mentored during assessments and supervision visits (distinguish). 
CAC identified in early reports that their visits to the RHB, Zonal and 
Woreda found HR scarce and overloaded with work such that they 
would not be able to be counterparts for mentoring on any regular 
and significant basis) This should have been abandoned and 
replaced (reallocate the budgeted resources). For example, develop 
and promote other types of short term training. 
Other TA to 
FMOH/PPD, other 
agencies, RHB, 
Woreda level, HC 
personnel 
In-progress This is presented in the reports in summary tables as # of 
TA/activities by quarter; because there is no operational definition for 
TA nor of a “T.A. activity”. These tables are unreliable. For example, 
in PY1 year report, there were 74 reported “T.A. activities which 
were follow-up to the survey for coordination purposes – not TA. 
Revised Guideline 
for FGOE HC 
standards 
In-progress In work plan indicated under “preparing, testing and vetting 
“standardized documents and procedures (Result Two)” 
Inventory of 
engineering assets 
Unknown To develop recommendations on engineering assets and present to 
the coordination meeting 
Procurement and 
contract 
management 
processes 
document 
 (see below) In work plan indicated under preparing testing and vetting 
“standardized documents and procedures (Result Two)”. 
Guidelines3 for HC 
renovation and 
expansion 
In-progress Indicated as tools and procedures for standardization in the 
proposal. Tools and procedures for standardization of pre-design, 
design, renovation, and post-renovation stages including service￾delivery concepts and materials utilization by stakeholders and 
maintenance. As of May 20, 2007, needs only final editing and 
submission to FMOH/PPD; great quality. 
 
3 The guidelines for renovation and expansion includes all of the following elements, each of which was developed 
or adapted for this project: 
Contracts and Contractual Tools 
Pre-qualification documents for works contractor selection (adapted from FGOE standard) 
Tender document for contracting with A&E firm (adapted from FGOE standard) 
Tender document for construction works (adapted from FGOE standard) 
Tender document for construction works (adapted from USAID) 
Sample contract agreement (adapted from FGOE standard) 
Health center 
assessment 
handbook (field 
guide) 
 Indicated as tools and procedures for standardization in the 
proposal. Detailed planning of individual projects selected for 
improvement, emphasizing sanitation and infection-control systems, 
including availability and proper function of sinks and toilets, hot 
water in clinical service areas, and function of maternal and child 
health, OPD and chronic-care service areas, and the reception and 
waiting area. As of 20May07, needs only final editing and 
submission to FMOH/PPD; great quality. 
Road map (to 
support 
development of 
standardized 
approach) 
Unknown In contract, this is related to regional TA workshops. The team did 
not see anything called a “road map”. 
Skills transfer to 
local sub￾contractors 
through 
mechanisms such 
as on-the-job 
training 
Not-started Have not found evidence that A&E (or CAC) has worked with the 
local subcontractors through OJT to improve their skills. 
Project safety 
officer; regular 
safety promotion 
meetings 
Not-started Found no evidence of this. 
A common 
approach to HC 
renovation being 
used by the RHBs 
In-progress Not adopted/used by the RHBs although CAC did document very 
early that there is a lack interest in this regard, because GOE now 
focused on massive scale up of new HCs. 
 Master bill of quantities (BoQ) document 
Normal BoQ document 
Memorandum of Understanding for Regional Health Bureaus 
Memorandum of Understanding for Health Centers and Woreda Health Offices 
Checklists and Forms 
 Health Center Maintenance Management Checklist (self-administered) 
 Damage Assessment Checklist 
 Reporting Forms 
• Progress (Supervision) Report 
• Health Center Assessment Site Visit Report 
Construction Works Certificates and Forms 
Health Center Data Collection Form 
Health Center Site Handover Form 
Damage Assessment Checklist 
Damage Take-off sheet form 
Initial Payment Certificate 
Final Payment Certificate 
Provisional Acceptance Certificate 
Final Acceptance Certificate 
An A&E firm with 
at least 10 
engineers 
experienced in use 
of the approved 
engineering 
assessment 
guidelines and 
other HC 
renovation 
materials; and 
In-progress 
(will not be 
completed) 
There are only 3-4 engineers trained and experienced from one A&E 
firm, and they are not permanent employees of the A&E. At this 
stage, there is still a possibility of training additional people but CAC 
has not planned/budgeted for this. 
Approximately 10 
private-sector 
construction 
companies located 
throughout the 
country with 
experience 
conducting HC 
renovation works 
consistent with the 
approved 
guidelines. 
In-progress By end of the project, will be reached. 
Review and 
comments on 
Standards of Care 
for Health Centers 
Completed Listed by CAC as an unexpected activity but clearly this is the type 
of TA that is expected of them as per all documents. 
Brief report; Health 
center space 
requirements for 
2008 
Completed CAC lists this as an “unexpected deliverable” because it states that 
the USAID/CTO requested it. 
Primary HC 
assessment 
findings from 3 
regions 
Completed Interim report; information was also presented in the semi-annual 
reports. It documents that they need an average of $39K per HC for 
minimum renovations using GOE HC Standards of Care as basis for 
this and material cost and renovation data available from CAC’s site 
engineer prior work with Intrahealth. 
Mapping and 
Capacity Building 
Report 
n/a This is a proposal for an additional $1.5M to add GIS mapping to the 
expected HC Renovation Mapping/tracking system. 
Report on Storage 
Capacity 
Assessment for 
Ethiopian MOH 
Facilities 
Draft Not sure if this can be counted as it was done for JSI/Deliver: CAC is 
a sub-contractor to JSI/Deliver. USAID should verify funding source 
as they should do with all the reported “products and services” (in 
light of the fact that the team identified 3 instances when CAC 
reported to the evaluation team a product of HCRCP that we verified 
was paid for out of another project’s money). 
Brief update report: 
Seconding STTA 
consultants to 
FMOH/PPD and 
RHB 
Completed Vital given this was not in the PY1 work plan/contract, although it 
could be included in PY2 work plan. It should have another update 
because the STTA are now primarily supporting the PMU at 
FMOH/PPD. 
Response to 
Services and 
Deliverables 
Issues posed by 
Evaluation Team 
 In section 2&3: Direct & Indirect TA to 100 HC including PVOs and 
donors, there are 12 agencies/projects listed as having received TA 
from CAC under HCRCP. But we found that several of these TA 
assignments were paid for by other projects and cannot be reported; 
others can. 
Reviewed the 
design being used 
to convert 262 
health stations 
(HSs) to “nucleus” 
HCs in Oromia 
region 
Completed This might be an expected deliverable but USAID must verify that 
CAC was not paid from another project for this activity. 
Developed master 
plans for 
leveraging 
additional 
resources to 
renovate three 
HCs – Shromeda, 
Gullele and Arada 
(July 2007) 
Completed This might be an expected deliverable but it USAID should verify 
that CAC was not paid from other project for this activity. USAID 
discussed that the Department of Defense is expected to renovate 
this and two other sites in Addis assessed by Crown. When this 
happens, CAC will have leveraged $660,000. 
Developed a 
master plan for 
pharmaceutical 
stores at HCs 
Completed This design was to be piloted at Mojo HC in the Oromia Regional 
State and Nefasilk No 2 HC in Addis Ababa USAID should verify 
funding source as they should do with all the reported “products and 
services”. 
Develop ToR and 
job descriptions & 
contracts for the 
(seconded) STTA 
(8 total) 
Completed See report Seconding STTA Consultants to FMOH/PPD and RHB 
Report: 
Brief Update January 28, 2008 
Review resumes 
and jointly 
interview 
candidates for 
STTA (8 total) 
Completed See report Seconding STTA Consultants to FMOH/PPD and RHB 
Report: 
Brief Update January 28, 2008 
Two-day 
orientation 
workshop for STTA 
before posting. (8 
total) 
Completed Reported as planned in the Seconding STTA Consultants to 
FMOH/PPD and RHB Report: 
Brief Update January 28, 2008; the report that this was completed 
was provided by USAID; we have no documentation of this. 
 APPENDIX 10) SUMMARY OF THE RESULTS FROM SITE VISITS 
 (n=13) via observations, interviews and, in some cases, small group discussions 
results from site visits to 13 HCs
Construction Safety at site? No: Practical safety precautions were not taken at renovation sites. 
Inside cleanliness of HCs visited Overall was very good. 
Outside Site cleanliness at HCs 
visited 
Overall very good, excluding areas near the incinerator at which some sites 
had needles lying nearby (see pictures). 
Note: this was observed as part of helping USAID identify intervention areas 
for the follow-on project, not as a measure of CAC performance under the 
current project as it is not part of their contract to intervene in this area). 
# sites in-progress & completed 3 of the 13 sites visited by the evaluation team were in progress (under 
construction). 
# times HC director was present 
during site visit 
8 
# time other providers were 
interviewed during site visit 
8 
Number of sites evaluated 13 
# HC which contacted Woreda, RHB, 
about problems with renovations 
when exist 
In north, it was very common for the HC to contact the Woreda (for problems); 
in the south, none contacted the Woreda even if problems with CAC and/or 
contractors occurred. For Addis (pre-test) we didn’t assess this. 
# HC which contacted CAC about 
problems when exist 
Northern sites contacted CAC, Southern sites didn’t know they could contact 
CAC. We don’t know in Addis; didn’t ask during pretest: the sites were under 
construction. 
# HC that contacted the contractor 
about problems when exist 
Most of the sites which had problems said they had tried to contact the 
contractor; there was no difference between the sites in the north and south 
(We don’t know in Addis which were pre-test sites). 
# HCs which knew of the one year 
guarantee on renovations 
Most knew of the one year guarantee but some HCs which had problems and 
did not know of this said that they would have reported the problems had they 
known of it (Addis sites are unknown). 
# HC where person who signed MOU 
present during interview 
About one-half. 
# HC where anyone interviewed was 
present when the HC assessment 
was done 
At all sites, there was someone present during our interview who was also 
present during the time when the HC assessment was done, though in a few 
of these cases, that person was not part of the decision making and could not 
tell us any details about the assessment or renovations. 
The renovation has helped service 
providers deliver better quality 
services 
Because most of the HCs reported as being “completed” had problems, they 
indicated that the renovations had NOT helped them provide better services 
But, If the problems are fixed, they felt it would help improve the quality of their 
services. For 2 HCs visited which had little or no problems, they said it has 
helped deliver better services, especially changes to delivery room and water 
supply. NB: Most indicated more things could have been done, such as roofs 
over waiting area, benches, shelves, showers, living quarters for staff, etc. 
(This latter point is not used to evaluate CAC. It is there to give USAID ideas 
regarding the follow-on project.) 
Renovations help clients; comfort & 
privacy 
Same as above. 
Were problems with the renovations 
identified? 
In all sites reported as completed, there was at least one problem of quality. In 
most sites there were multiple problems, some very serious. At most of the 
completed sites reported as complete, some or all of the water and sanitation 
system was not working (see photos). 
HC service providers have seen the 
CAC maintenance form distributed 
with CAC-HC MOU 
Providers had not seen the form at sites visited. 
The HC was left with a copy of the 
renovation blue print 
HC renovation blueprints were not left at sites visited. 
HC had a copy of assessment? 
 
A copy of the assessment was not left at sites visited. 
Other organizations were working 
with the HC? 
Yes, most frequent is MSH/RPM+ but also, in a minority of sites visited, MSF, 
Intrahealth, etc. 
HC provider attitude about the 
HCRCP renovation. 
For sites visited which were yet to be completed (still in progress), they were 
all very pleased, 
Those sites visited which were reported as completed varied from indifferent to 
very displeased.