FINAL	
Internal	Performance	
Evaluation	of	AWARE	II	
USAID/West	Africa	Regional	Health	Office	
Carmen Tull, Global Health Family Planning Technical Advisor USAID/W                                 
Sheila Mensah, Communications, M& E Advisor USAID/WA Regional Health Office                       
Daniele Nyirandutiye, Deputy Office Director, USAID/WA Regional Health Office                         
7/20/2012
  
Final
Internal Performance Evaluation of AWARE II
1
Table of Contents
Acronyms ..................................................................................................................................................2
Introduction ..............................................................................................................................................3
Summary of Evaluation Design and Methodology ................................................................................... 3
Background ...............................................................................................................................................4
Accomplishments of AWARE II ................................................................................................................. 5
Challenges.................................................................................................................................................8
Lessons Learned for Future Regional Health Programs.......................................................................... 14
Appendix A: Key Informant Questions.................................................................................................... 18
Appendix B: Key Informants ................................................................................................................... 20
Final
Internal Performance Evaluation of AWARE II
2
Acronyms
ACT Artemisinin‐based Combination Therapy
ADESCO Appui au Développement et à la Santé Communautaire
ARI Acute Respiratory Infection
ATBEF Association Togolaise pour le Bien Etrê Familial (IPPF Affiliate in Togo)
CHW Community Health Worker
C‐IMCI   Community‐Integrated Management of Childhood Illnesses
COR Contract Officer Representative
ECOWAS Economic Community of West African States
FP Family Planning
IEC/BCC Information, Education and Communication/Behavior Change Communication
KIP Key intervention Package
MNCH Maternal, Newborn and Child Health
MOST Management and Organizational Sustainability Tool  
MOU Memorandum of Understanding
MSH Management Sciences for Health  
NGO   Non‐Governmental Organization
RAPID Resources for the Awareness of Population Impacts on Development
RFTOP Request for Task Order Proposals
RHO Regional Health Office
SOW Statement of Work
USAID/WA USAID/ West Africa
WAAF West Africa Ambassador’s Fund
WAHO West Africa Health Organization
Final
Internal Performance Evaluation of AWARE II
3
Introduction		
The United States Agency for International Development (USAID)/West Africa’s (WA) Regional Health
Office (RHO) conducted an internal evaluation of its flagship project, Action for West Africa Region II
(USAID AWARE II).  The purpose of this evaluation was to document the overall successes and challenges
of implementing a regional health project.  Findings and lessons learned from this evaluation will be
used to inform management decisions for USAID/WA and the design of similar future programs.  
AWARE II was a three‐year contract (July 2009‐July 2012) with a total ceiling of $20.998 million. The
overall strategic objective was for countries in the region to plan and implement selected high‐quality
health service delivery programs.  To achieve this objective, the project aimed to create a common
vision for health priorities in the sub‐region, improve the overall policy environment, replicate best
practices and strengthen African capacity to deliver services. The project worked in 21 countries in West
and Central Africa and was implemented by Management Sciences for Health (MSH) and its partners,
EngenderHealth, and Futures Group International.
Summary of Evaluation Design and Methodology		
The objectives of the internal evaluation were to: 1) determine the appropriateness of the project’s
original technical and geographic scope and award mechanism; 2) assess program performance,
achievements and challenges; and 3) provide recommendations for future regional health programs.  
To achieve these objectives, the internal evaluation team developed a list of questions for key
informants (See Appendix  A – Key Informant Questions).  The evaluation team interviewed 23 key
informants representing the Ministry of Health in Togo, AWARE II project management, the US Embassy
in Togo, NGO partners and USAID managers (See Appendix  B – Key Informant Questions). The team also
reviewed key project documents such as the contract, work‐plans, and the final project report.   
Information gathered from the key informant interviews and reports was synthesized into this report
which covers three main areas: AWARE II’s achievements, challenges and lessons learned for future
programming.   
Final
Internal Performance Evaluation of AWARE II
4
Background
From 2003 through 2008, USAID funded a Reproductive Health project (AWARE‐RH), which ran parallel
to an HIV & AIDS project (AWARE‐HIV/AIDS). The two projects, together called USAID AWARE I, were
designed to improve the health of West Africans and contribute to economic and political stability.  
Launched in 2009, USAID AWARE II was designed to build on the successes of its predecessors.  
AWARE II’s overall strategic objective was for countries in the region to plan and implement selected
high‐quality health service delivery programs. At the beginning of the project there were five expected
results:  1) Regional common strategic vision and priorities for improving the health status of West
Africans (completed in year 1); 2) Policies developed and implemented to foster effective regional and
national health programs; 3) Selected high‐impact practices adopted and replicated; 4) Selected number
of  West African institutions and networks strengthened; and 5)New funds are mobilized for health
programs, existing donor and national resources effectively used (eliminated after year 1).    
The project’s geographic and technical scope was adjusted three times during the contract amendment
process. As a result, the project’s focus shifted and Expected Results 1 (which was achieved early in the
project) and 5 were removed.  At the same time, the geographic focus became narrower and more in‐
depth and shifted over the course of three years from a focus on eight countries to two districts in Togo,
where delivery of the integrated package by community health workers was piloted.
To achieve these results, AWARE II worked with and through West African partners, NGOs and
stakeholders to:  1) Address constraints hindering the success of health services and programs in the
region; 2) Facilitate the adoption of supportive policies; and  3) Introduce and catalyze the scale up of
quality health services.  
Despite the challenges of working across multiple countries, AWARE II was able to accomplish several
results.  However, as reflected in the fact that its geographical and technical scope was adjusted three
times, one can conclude that there were significant challenges to implementing this project. While
programing at a regional level will always prove challenging, it is hoped that the lessons learned from
AWARE II can help focus the next phase of the regional health program, which in turn could lead to
more efficient use of resources and sustained results for health in the sub‐region.  
Final
Internal Performance Evaluation of AWARE II
5
Accomplishments of AWARE II
Within the first year of the project, AWARE II succeeded in obtaining a common vision for improving
maternal and child health in the sub‐region.  Although it took over a year to operationalize this common
vision, in the final year of the project, AWARE II demonstrated that community health worker (CHWs),
with little training and proper supervision, can correctly provide the first offer of pills and hormonal
injectable, as well as treat childhood malaria, diarrhea and pneumonia.   Other achievements include the
successful administration of the West Africa Ambassador Fund (WAAF) grants, the innovative
monitoring and evaluation systems that were developed and its approach to building the capacity of
several local institutions.  
1. Developed a common vision for FP/MNCH/HIV & AIDS programs in the sub‐region:  The first
expected result of AWARE II was to achieve a “regional common strategic vision and priorities
for improving the health status of West Africans.”   Within the first year of AWARE II, 18
countries agreed to the following common vision for the project: “Harmonized health policies
and standards among committed countries, allocating adequate funds for quality health
services, and achieving health impact.”  In addition to the common vision, the countries with
guidance from AWARE II identified key interventions packages (KIPs) in the areas of family
planning (FP) /Reproductive Health (RH), maternal health, newborn health, child health (MNCH),
HIV&AIDS and health systems.   This agreement on the vision and KIPs provided a common
framework for advancing health programming in the region.  For example, implementation of
the KIPS for FP/MNCH became the basis for the  Memoranda Of Understandings (MOUs) that
were signed between  AWARE II and  four country governments, Togo, Burkina Faso, Niger and
Mauritania.  Additionally, through the WAAF grants, local grassroots organizations were trained
in the KIPs and were able to implement these best practices in their respective program areas.   
2. Implemented the integrated FP/MNCH community‐based demonstration project in Togo:  
AWARE II signed a MOU with the government of Togo to pilot the training of community health
workers (CHWs) to be able to treat and manage acute respiratory infections (ARI), diarrhea and
malaria (community –integrated management of childhood illness(C‐IMCI), as well as provide
the initial offer of pills and administer injectable contraceptives and FP/RH, HIV/AIDS prevention
and nutrition messages.  Although the start‐up of this project was slow, several things were
done particularly well.  AWARE II engaged the government of Togo from the beginning in the
selection of the intervention package and the geographic intervention area.  AWARE II also
Final
Internal Performance Evaluation of AWARE II
6
selected two strong non‐governmental organization (NGO) partners, Association Togolaise pour
le de Bien Etre Familial (ATBEF) and Appui au Développement et à la Santé (ADESCO), which had
over 15 years of experience in community‐based distribution of contraceptives.   AWARE II
expanded the capacity of ATBEF and ADESCO to train CHWs in the initial offer of pills and
administration of injectable contraceptives.  Both NGOs expressed appreciation for the
opportunity to be part of this innovative pilot, which enabled them to expand their capacity to
deliver services.  ATBEF and ADESCO commended  AWARE II for how they worked with the
Togolese Government to prevent stock‐outs of contraceptives at the demonstration sight.   
However, the NGOs did mention that one of the challenges to implementing this pilot was the
repeated stock‐outs of  malaria drugs, artemisinin‐based combination therapy( ACT),  due to
challenges with the Global Fund.
While this pilot needs to be further evaluated to determine its strengths and weaknesses,
overall sustainability and lessons learned, the Government of Togo and the NGOs partners all
claim the initial results as an overwhelming success. By the end of seven months of
implementation, 432 CHWs were trained and deployed and close to 10,000 women had
received contraceptive services in the two districts.  Additionally, it is estimated that close to
6,000 cases of diarrhea, malaria and pneumonia were treated.    The demonstration project in
Togo proved that that properly trained and supervised CHWs can provide the initial offer of pills
and administer hormonal injectables, as well as safely treat children for pneumonia, diarrhea
and malaria.  Moreover, the results demonstrate that there is real demand for family planning
services and that these services are used when they are accessible at the community‐level.  
3. Administered the West Africa Ambassador’s Fund Grants (WAAF):   AWARE II implemented a
small grant mechanism that supported 12 grassroots organizations to replicate and expand
services for HIV & AIDS, family planning and maternal and child health.  This fund, called the
WAAF, was administered through US Embassies in West and Central Africa, where there is no
USAID bilateral mission.   AWARE II and the USAID/WA designed a transparent selection process.  
USAID/WA issued a call for applications to all eligible US Embassies, who in turn advertised the
program to local NGOs.  NGOs submitted their applications directly to the embassies, which
then reviewed the applications and selected two projects each for consideration by USAID/WA
and AWARE II.  USAID/WA and AWARE II reviewed the proposal and provided feedback to the
Final
Internal Performance Evaluation of AWARE II
7
grassroots organizations to address their shortcomings.  The applicants had a chance to revise
their applications.  After they resubmitted their revised proposals, USAID/WA and AWARE II
selected the final recipients. Twelve organizations from 8 countries were selected to implement
grants ranging from $50,000 to $100,000.  At the end of one year, the 12 WAAF grantees had
collectively counseled 63,739 people and tested 24,403 for HIV. Additionally, the NGOs
distributed 245,285 condoms and 47,702 IEC/BCC materials; trained 1,324 peer educators; and
provided modern contraceptives for 7,088 FP users (2,293 new users and 2,912 returning users).
The administration of the WAAF grants is considered successful for several reasons: 1) The
transparent selection process provided the NGOs with an opportunity to receive feedback and
revise their submissions, thus building their capacity to apply for grants; 2)  By involving US
Embassies in the selection process the relationship between the regional health office and the
US Embassies in the non‐presence countries was strengthened; 3) The grants program created
an opportunity to highlight US government development assistance in countries that receive
very small amounts of money for health; 4)  The grassroots organizations’ managerial and
technical capacity was strengthened;  and  5) Select  KIPs were replicated at a smaller scale
across 8 countries.    
4. Signed MOUs with Burkina Faso, Mauritania, Niger and Togo:  AWARE II successfully
negotiated MOUs with four Ministries of Health (Burkina Faso, Mauritania and Togo) to pilot an
integrated package of services at the community level.   These MOUs authorized CHWs to
administer the first offer of pills, hormonal injectables, ACTs and antibiotics.  The fact that four
governments agreed to pilot access to contraception at the community level represents a
significant step forward in efforts to de‐medicalize family planning and child health services in
the sub‐region.   Moreover, strategic advocacy efforts were used to negotiate these MOUs.  For
example, in Togo, the RAPID model, an advocacy tool which demonstrates the impact of rapid
population growth on different sectors such as the environment, education, water and land use
was used to make the case to allow administration of injectables at the community level to
increase access to services.  The RAPID model caught the Minister of Health’s attention in Togo,
to the point where he became supportive of increasing access to family planning through the
community based distribution of injectables.  
Final
Internal Performance Evaluation of AWARE II
8
5. Strengthened technical and managerial capacity of regional Institutions:   With the NGO’s that
were selected to implement the integrated package, AWARE II used the Management and
Organizational Sustainability Tool (MOST) to enhance their capacity in budgeting, staffing,
administration and finance.    By implementing the integrated package in Togo ATBEF and
ADESCO were able to expand their technical skills to include training CHWs on the initial offer of
pills and injectables.  AWARE II worked with all of the WAAF grantees to evaluate their financial
systems and to provide them with technical assistance to improve their overall financial and
management systems.   The WAAF grantees said that this work helped them reach more clients
and raise their visibility with the local US Embassies, as well as the Ministries of health in each
country.  Finally, AWARE II worked with WAHO, especially in the first year of the project to
identify the KIPs and advocate for supportive policies that improved access to health care.  By
working closely with WAHO in year one of the projects, AWARE II strengthened their capacity to
identify and build consensus around key health intervention areas.  
6. Improved monitoring and evaluation and communication efforts across the sub‐region:
AWARE II created a regional online database to facilitate timely, routine data reporting. The
project then trained each NGO to use the database for monthly reporting.  AWARE II also
created a user manual to train future projects and partners in using this web‐based system.  This
tool allowed the project to monitor the grantees progress and print reports for USAID and US
Embassies.  In addition to the innovative online tool, the project developed fact sheets for each
of the project’s 21 countries that included the latest information on the countries’ health
policies, progress towards the Millennium Development Goals, and support by major donors.
English and French versions of each fact sheet were developed and 6,300 copies were
distributed to stakeholders. Ministry of Health officials are using these factsheets as a tool to
inform donors and partners on key health indicators in their countries.
Challenges		
AWARE II faced many challenges during the project’s implementation.  These challenges stemmed
from the project’s broad geographic and technical scope; lack of a shared vision for project
implementation between AWARE II and USAID; changes in management, both in AWARE II and
USAID; implications of several contract modifications; and ineffective communication.  
Final
Internal Performance Evaluation of AWARE II
9
1. A three year contract was not the appropriate mechanism for the project’s geographic and
technical scope:  Originally, AWARE II was a three year, 28 million dollar contract that
covered 21 countries.   Per USAID regulations, a contract is to be used when the Agency is
buying a specific good or service.  Contracts require clear scopes of work with specific, time‐
bound, deliverables.  Working at the regional level, across 21 countries, in the areas of
policy, capacity building and service delivery, clear specific deliverables, within a specific
time frame can be hard to define. Despite all of the best intentions and planning,
approaches to programming are often required to change due to political instability,
government bureaucracies, changing donor priorities and shifting resources. To improve the
overall policy environment, build capacity of organizations to implement the KIPs, a more
flexible mechanism, such as cooperative agreement was needed.  
As a contract, any changes to AWARE II’s technical approach and expected deliverables had
to be amended through the USAID contract’s office, which slowed down the
implementation of the project.   Contracts are also labor intensive for USAID staff to
manage.  The USAID/WA RHO was understaffed for most of AWARE II’s period of
performance and as a result could not devote adequate time to the overall management of
the project.  
Additionally, for the broad geographic and technical scope of the project, three years was
not enough time to see results. To actually start demonstration sites, award sub‐contracts,
train community health workers, sign MOUs with host country governments, and monitor
and evaluate the intervention takes longer than three years.  As one key informant said,
“This project was too ambitious to have everything accomplished in three  years.  As we
know from experience, it takes time to work at the regional level and reach a common
understanding. By the time things were set up, it was time for the project to end… The
project should have been five years for results to be sustained and more engagement with
other donors, MOH, etc.”
2. The project’s SOW was changed three times:   As mentioned above, the regional health
office modified the technical and geographic scope of the project three times. During this
Final
Internal Performance Evaluation of AWARE II
10
process, the countries were narrowed from a focus in eight to four and then finally to one
country.  This re‐scoping  was due to a number of factors:
a. Broad geographic and technical SOW: From the beginning, the Request for Task Order
Proposals (RFTOP) was very broad and the winning proposal was extremely ambitious.  
There was an expectation that AWARE II would produce everything that was promised
in the proposal.  After the fist year it was clear that more focus was necessary for the
project to achieve results and the process of changing the contract began.
b. Evolving USAID management structure: USAID changed CORs three times during the
course of a three year project. There were also three different activity managers.   
Anecdotal evidence points to the fact that the scope of work for the project was
sometimes changed dramatically to reflect personal interests of the COR.  These
interests did not necessarily reflect what was technically best for the project or region.  
There was no common vision on the part of USAID of what was supposed to be achieved
by the project. The USAID/WA front office, health office and contracts office all had a
different vision for the project.
c. Inability to achieve timely results: USAID was forced to amend the contract because
the prime contractor was unable to produce timely results.  At the end of Year 2, it was
evident to the RHO that MSH was not on track to implementing demonstration sites in
four countries with the remaining year that was left.   The RHO felt that by narrowing
the SOW to a geographic focus in one country that there would be greater likelihood
that some concrete results would be achieved.   
3. Changing SOW negatively impacted AWARE II’s image and effectiveness: The three
changes to the project’s technical and geographic scope was detrimental  for USAID and
AWARE II for the following reasons:
a. Damaged relationships with MOH in Burkina Faso, Mauritania and Niger:  MOUs were
signed with these three countries and when USAID abruptly pulled out of their
obligations outlined in the MOUs, USAID’s reputation was damaged in these countries.  
To further complicate matters, USAID did not send a letter explaining why the MOUs
were canceled.  In Niger, for example, the project had already awarded sub‐grants to
NGOs to start implementing the integrated package when the project had to pull back
Final
Internal Performance Evaluation of AWARE II
11
and cancel the sub‐agreement.  Anecdotal evidence indicates that the MOH’s were very
disappointed and frustrated.  
b. Limited application of results from one demonstration site:  A regional project needs to
demonstrate a new approach or model in more than one country in order to validate its
results.  Two districts in Togo are not a representative sample of the entire sub‐region.
Demonstration sites in other countries are needed in order for the results to be
perceived as applicable throughout the sub‐region.
c. Lost valuable time due to the contract modification process: Each change in the
project’s technical and geographic scope required a new workplan that had to be
developed and approved.  Anecdotal evidence indicates that when the contract
modification took place in the second year of the project, up to four months were lost
while the new workplan was developed, reviewed and approved.  In a three year project
with such a large scope, losing such valuable time further challenges the project’s ability
to achieve results.  The time lost also led to delays in the WAAF grants, which did not
start until the last year of the project.  The US Embassy in Togo did express
disappointment that it took so long to get these small grants off the ground and that
they only lasted for one year.  
d. Inefficient use of resources: AWARE II was a $28 million project across three years.  In
the end close to $20 million was spent.  While AWARE II achieved some impressive
results, the overall value for money was questioned by several key informants.  The
amount of staff time and resources spent on changing the contract took away resources
that could have been used to implement project activities.
e. Minimized sustainability of results: By changing the SOW three times, it was difficult to
convincingly demonstrate a sustainable approach to implementation.  As a result, the
lasting impact of the project may be limited.
4. Weak management, leadership  and communication at every level of the project :  
a. Inadequate staff and management structure for a project of this scope: For a project
of this magnitude, AWARE II did not have the appropriate management and staffing
structure.   For example, combined AWARE I RH and HIV&AIDS had over 30 staff
members.  While it was clear that USAID wanted a leaner staffing structure after the
first AWARE projects, AWARE II only had 9 staff members to implement the regional
Final
Internal Performance Evaluation of AWARE II
12
project.  Moreover, the staff had different levels of competencies and it became evident
as the project started that as a group, several key competencies in the areas of
management and leadership were missing.  Towards the end of Year 2, the Deputy
Director was asked to leave because he was not fulfilling his role.
b. Insufficient support from MSH, EngenderHealth and Futures headquarters:   At MSH
headquarters, there was significant turnover among the key backstops of the project.
The MSH Global Technical Director and Regional Support Team Leader, who were both
significantly supporting the project, left their positions.  Their successors, respectively,
had either little involvement with the project during most of her tenure or had not been
replaced at all.  When the Deputy Director / Technical Director left during the second
year of the project, instead of replacing him with a full‐time staff person, MSH replaced
him with a long‐term consultant who spent less than 50% of his time in Ghana.  MSH
leadership was made aware of the problems with this project early, yet did not
intervene early enough with USAID or their own staff to try to get the project on track.
c. Unsatisfactory management of the consortium:  MSH as the prime contractor did not
effectively manage EngenderHealth and Futures Group. It was clear that the Deputy
Director/Technical Director from EngenderHealth was not a good fit early on in the
project, but no effort was made to replace him in a timely manner.  There was no joint
planning with MSH, EngenderHealth and Futures Group.  Additionally, because the
service delivery component was slow to start, the Futures Group at times seemed to
follow their own separate workplan that was not well linked to the work of the other
partners in the consortium.  The other partner, EngenderHealth did not appear to
produce any results.  As prime partner, MSH did not manage its consortium partners so
that they were performing adequately and contributing to results to advance the overall
project, not individual IRs.
d. Poor communication with USAID: During the contract modification process, MSH did
little to articulate what would be best for the project and the region.  MSH accepted all
USAID’s requests for changes.  If there had been more effective dialogue and discussion
by the project’s management, perhaps the contract modifications would not have been
so drastic, nor would they have had such a detrimental effect on the project achieving
desired results.
Final
Internal Performance Evaluation of AWARE II
13
e. Lack of coordination and collaboration with other USAID partners:  The RHO put field
support in the centrally managed RESPOND project to work on training of clinical
providers in long acting and permanent methods in Togo and Burkina Faso.  The idea
was that the work of RESPOND would complement the work of AWARE II in those
countries.  However, very little communication or coordination between the projects
took place.  AWARE II was unaware of what RESPOND was doing and vice‐versa.  This
was a missed opportunity to leverage USAID resources and technical strengths, and to
seamlessly provide comprehensive family planning services to the project’s targeted
populations.
5. AWARE II’s initial mandate to work with WAHO was abruptly changed: The West Africa
Health Organization (WAHO), founded in 1987, is the health branch of the Economic
Community of West Africa States (ECOWAS).  WAHO’s mandate is to “attain the highest
possible health standard and protection of health of the peoples in the sub‐region.”  The
Request for Task Order Proposals (RFTOP) for AWARE II stated that, “WAHO should play a
pivotal role in leading, coordinating and harmonizing West African nation’s efforts to
combat major health problems in the region and as such would be a natural key partner of
AWARE II.  With strengthened capacity WAHO should be front and center in many regional
endeavors in the health sector.” To fulfill the original expectation outlined in the RFTOP,
AWARE II proposed to work closely with WAHO to set the regional agenda for best practices
in FP/MNCH/HIV&AIDS.   
       In the first year of the project, AWARE II signed an MOU with WAHO to outline their
partnership.  WAHO was also involved in developing the first AWARE II workplan, as well as
the regional stakeholders meeting that identified a common vision and key intervention
areas for the sub‐region.  AWARE II and WAHO made joint country visits to help plan how
the countries would implement the KIPs.  However, after Year 1, AWARE II received
guidance from the COR to no longer work with WAHO.  This conflicting guidance on how
AWARE II should interact with WAHO resulted in missed opportunities to influence the
regional policy agenda.  For example, members of the AWARE II team felt that the process
of signing the MOUs would have been much faster if WAHO was part of the process.   
Additionally, the abrupt change in the relationship between AWARE II and WAHO strained
the relationship between USAID and this regional health body.
Final
Internal Performance Evaluation of AWARE II
14
6. Approach for helping countries achieve contraceptive security was not well defined.    
AWARE‐ RH provided technical support to Ministries of Health to effectively forecast and
plan their family planning needs.  Additionally under AWARE I, the RHO helped countries
procure contraceptives. Under AWARE II, contraceptive security was not addressed.
Moreover, the RHO did not have a clear and consistent approach to procuring
contraceptives for the countries in the sub‐region. As a result, countries did not receive the
technical and financial support for commodities. This resulted in confusion at the country
level.  For example, Togo said that they sent in multiple requests to USAID for support for
commodities that went unanswered.  USAID said they responded to their requests, but
needed more information in order from Togo to fill their requests.   This confusion reflects
the need for the RHO to have a clear channel of communication to countries about their
requests for commodities as well as a way to provide technical assistance to countries to
strengthen the actual quality of information contained in their requests. Towards the end of
AWARE II, the regional health office provided support to DELIVER to begin to address some
of the contraceptive security issues in the sub‐region with the establishment of a focused
early warning system for West and Central Africa. However, a consistent approach to
providing technical assistance and support for commodities from the beginning could have
minimized confusion by the countries.   
Lessons Learned for Future Regional Health Programs
1. To effectively build consensus, change policies, implement interventions and achieve results,
the next regional health project should be a cooperative agreement for five years:  Regional
programming is complex and extremely challenging.   To change the policy environment and
implement best practices, multiple relationships with several country governments, NGO
partners, regional bodies, donors and the private sector need to be effectively managed.  
Despite all the planning, these relationships will fluctuate due to political instability, changes in
donor priorities and shifting resources.  To ensure that despite these ever‐changing variables,
desired results are achieved, a mechanism must be in place to allow for some flexibility and
ability to modify technical approaches without going through the process of modifying a
contract.  Moreover a three year time frame does not allow enough time to actually implement
any interventions and achieve desired results. As such, a cooperative agreement for five years
Final
Internal Performance Evaluation of AWARE II
15
would allow for some degree of flexibility, while enabling a partner to have adequate time to get
consensus from partners, implement the desired interventions and achieve results.
2. The RHO should initiate and maintain regular diplomatic/programmatic meetings with MOH
staff in priority countries and WAHO. AWARE II worked hard to establish strong relationships
with the MOH in several countries, most notably Burkina Faso, Mauritania, Niger and Togo.  
These relationships resulted in a signed MOU with each country.  When the MOUs for Burkina,
Niger and Togo were abruptly pulled, the relationships with these countries were strained.  This
was further complicated by the fact that the RHO did not reach out and explain why these
MOUs were discontinued. This communication was left solely to AWARE II.  Similarly with
WAHO, the inconsistent relationship between USAID, AWARE II and WAHO needs to be
addressed. Collaboration with WAHO, if well‐defined with clear outputs and expectations, could
potentially accelerate the adoption of policies that can ultimately help implementation and
replication of the best practices. As the RHO embarks on a new phase of development
assistance, initial visits to the priority countries and to WAHO could be helpful in reestablishing a
strong relationship.  Maintaining regular contact with the Ministries and WAHO with in person‐
visits, phone calls and emails will help to facilitate communication, create and sustain a shared
vision and repair relationships.
3. The next regional family planning project should continue the WAAF grants: The WAAF grants
were well received by the Embassies.  They present a good opportunity to increase the visibility
of US foreign assistance in non‐presence countries.  Additionally, the WAAF grants present an
opportunity to build the capacity of grassroots NGOs to replicate best practices in geographic
areas that are not covered by the RHOs larger health program.   The WAAF selection process
should start in Year 1 of the new project in order to enable the grassroots organizations that are
obtaining results to receive an additional year of   funding.  From the beginning, the project
should establish criteria for how long a WAAF grantee can receive funding and clarify how
grantees are to communicate with the project.  
4. The regional on‐line database to track results and for reporting should be replicated and/or
continued. Managing data across a regional project is difficult.  The data‐base AWARE II
established was a transparent way for all of the partners and USAID to track progress.  Without
Final
Internal Performance Evaluation of AWARE II
16
exception this tool or an adapted version of this tool should be used to monitor achievements in
the future health project.
5. Build on the accomplishments and lessons from AWARE II to ensure rapid start‐up:  As noted
above, AWARE II has built a strong foundation for action in the region.  The consensus achieved
through the negotiated common vision and key interventions is a critical step forward. In the
next phase of the RHO’s family planning project, limited time should be spent on getting
consensus on the key interventions.  Ideally, the project should begin with assessments on what
areas of the countries are most suitable to implement the demonstrations; revisit the MOUs
that were signed; and rapidly deploy resources to implement the demonstrations.  The baseline
reports should be used to document the local context, how the KIPs need to be adapted to fit
the local contexts and the need for family planning services in targeted areas.  Additionally,
USAID/WA should commit to working with the same four countries with which AWARE II signed
the MOUs (Burkina Faso, Mauritania, Niger and Togo).   If at all possible, barring political
instability, the USAID/WA should commit to working in these countries for a five year period.    
6. Mandate coordination with all partners: AWARE II suffered from communication challenges at
all levels.  The next iteration of the RHO health program should involve joint work planning with
the policy, contraceptive security and service delivery components of the project, perhaps
through a joint coordination/leadership team made up of the Project Directors responsible for
the service delivery, the policy and contraceptive security components of the project.  Quarterly
meetings with all partners and the RHO should be scheduled at the project start to identify
areas of coordination, to identify potential partners and to ensure complementary
implementation of activities.  If possible, the service delivery partner, the policy partner and the
DELIVER team should be collocated to increase collaboration.
7. Evaluate the integrated FP/MNCH community‐based demonstration project in Togo: The
Government of Togo is pleased with the results of the Togo pilot in two districts, but they would
like to evaluate this pilot before they replicate the demonstration project. USAID should
consider evaluating this pilot in October/November after the demonstration has been running
for one year.   It is important to note; however, that both ATBEF and ADESCO, feel that they
Final
Internal Performance Evaluation of AWARE II
17
have already conducted an evaluation and based on their results to date that the integrated
package is ready to be taken to scale.  Both NGOs commented that at the end of project event in
Togo, other districts expressed interest in replicating the integrated packaged.  The evaluation
team explained to both ATBEF and ADESCO that an external evaluation has been requested by
the Ministry before the intervention is replicated in other areas.
8. A strategy for providing commodities to these priority countries must be developed and
communicated: The RHOs inconsistent approach to supplying commodities and providing
technical assistance for contraceptive security over the years has led to some confusion over
where and how countries in the sub‐region can expect to procure contraceptives.  As a result,
the RHO needs to work with DELIVER to develop a strategy for supporting contraceptive security
efforts in the sub‐region.  Once the strategy has been developed, the RHO should communicate
with the MOHs in the region about what USAID’s role in the region will be.
9. Maintain a policy component in the service delivery piece to help sign MOUs and increase
political and financial support for family planning: The policy component of AWARE II helped to
facilitate the implementation of the integrated package by negotiating the MOU with the
government of Togo.  To help create an enabling environment for service delivery at the
national and local levels, a policy component needs to be maintained within the new regional
family planning project. This policy piece would work closely with HPP field support activity,
which will work at harmonizing policies and leveraging donor efforts at a regional level.
  
Final
Internal Performance Evaluation of AWARE II
18
Appendix A: Key Informant Questions
USAID AWARE II Internal Evaluation‐MSH
1. What did the project do well?
2. What could the project have done better and how?
3. What do you think is a good balance of service delivery and/or policy work for a regional
project?
4. How did the WAAF grants come about? (Was this an original concept of AWARE II or had RHO
done these types of sub‐grants before)?  
5. Geographic Scope:  What were the challenges and lessons learned with working in 21 countries?
Please discuss:  
‐ AWARE II staffing and expertise?  
‐ Managing country expectations?  
‐ Effective communication with USAID, government officials and NGOs?  
6. The project’s technical and geographic scope was changed three times. How did these changes
affect the project’s ability to influence host country health programs (or achieve expected
outcomes)? How did these changes affect the project’s relationship and perception with
ministry of health and NGO partners in Burkina Faso, Mauritania and Niger?
7. How was  the policy component of the project helpful to the overall project? How could it have
been improved?
8. In what ways did the project work effectively with WAHO to maximize results? What were some
of the challenges of working with WAHO and what were the lessons learned?
9. What were the challenges and lessons learned for working with Technical Leadership
Institutions and other local organizations identified to provide training of trainers and service
delivery?
10. What were the challenges and lessons learned for coordinating work with the RESPOND Project
to provide a wide variety of options for Family Planning in the Haho and Blitta districts?
11. What are the necessary events/steps to obtain support from MOH and other government
officials for the integrated package implementation?  What is the appropriate sequence of
events for getting service delivery on the ground?
12. In the end, how much did the pilot in Togo cost? What were the main drivers of these costs?
How could the Togo demonstration been implemented differently to reduce costs?
13. What are effective ways to coach and monitor WAAF grantees to build capacity, improve service
quality and ensure results?
Challenges and Lessons Learned for Implementation: US Embassy
1. How does the Embassy perceived the AWARE II project?
2. What was the Embassy’s experience with the WAAF Grants implementation?
3. How could AWARE II project have worked more effectively with the Embassy?
4. Should the WAAF grants be continued?
Final
Internal Performance Evaluation of AWARE II
19
Challenges and Lessons Learned for Implementation: Togo: Dr. Kassouta C. N’TAPI, Head of the Family
Health Division  
1. How do you perceive the AWARE II Project?  What did the project accomplish? How could the
project have been more effective?
2. How will your department build on the results achieved by the AWARE II Project? Will the model
be continued in Haho and Blitta districts? Are there plans to replicate the model in other
districts?
Challenges and Lessons Learned for Implementation: ATBEF and ADESCO
1. In what ways has your organization’s ability to provide services improved or expanded as a
result of working with AWARE II?
2. What were the challenges of working with the AWARE II Project?
3. How could the project have worked more effectively with your organization?
4. How did you work with the district health management team and health center personnel to
supervise community health workers?
Challenges and Lessons Learned for Implementation: USAID Staff
1. What did the project do well?
2. What could the project have done better and how?
3. What do are the lessons learned for future programming in the region?
Final
Internal Performance Evaluation of AWARE II
20
Appendix B: Key Informants
Number Name Title Organization
1. Robert Whitehead Ambassador US Embassy, Togo
2. Lanta Spencer   Political/Economic
Officer
US Embassy Togo
3. Chantal A. Afoutou Self‐Help Program
Coordinator
US Embassy Togo
4. Dian Samtu,   HIV/AIDS Program
Coordinator
US Embassy Togo
5. Dr. Kassouta C. N’TAPI,   Chef de Division de
Santé Familiale:
MOH, Togo
6. Yawo‐Measah Damessi DSF MOH, Togo
7. Kassindeyem Odjeke DSF MOH, Togo
8. Akoua Degbeni DSF MOH, Togo
9. Celestine  Amega DSF MOH, Togo
10. Allado Yavi Ahoefavi Akpadza DSF MOH, Togo
11. Akonete Ahessou DSF MOH, Togo
12. Essowazina Akondo   ONG ARECA
13. Daniel Nyirandutiye Deputy Director USAID/West Africa
14. Lauren Kapesa HIV&AIDS Advisor USAID/West Africa
15. Sheila Mensah Communications,
M&E/ Advisor
USAID/West Africa
16. Solange Toussa Ahossu Executive Director ATBEF
17. Kossi Ahadji Sociologist ATBEF
18. Nadzombé Datagni Executive Director ADESCO
19. Alfred Amoatwo Project Management
Assistant/HIV&AIDS
USAID/West Africa
20. Josephine Kitongo Acquisition and
Assistance Specialist  
USAID/West Africa
21. Issaka Diallo Project Director,
AWARE II
MSH
22. John Pollack Acting Vice president:
Center for Health
Services
Senior Fellow, Health
Reform & Finance
MSH
23. Elke Konings Technical Advisor MSH