Monitoring and Evaluation of the 
Emergency Plan Progress (MEEPP): 
End-of-Project Evaluation

Monitoring and Evaluation of the 
Emergency Plan Progress (MEEPP): 
End-of-Project Evaluation 
Samuel Kalibala 
Population Council 
Acknowledgements 
The evaluation of the Monitoring and Evaluation of the Emergency Plan Progress (MEEPP) 
project was made possible with support from the United States Agency for International 
Development (USAID)/Uganda. 
This report was written by Dr. Sam Kalibala of the Population Council. The author is grateful 
to Dr. Paul Bukuluki and Richard Tuyiragize for their contributions in leading the data 
collection process, analyzing the data, and reviewing the report, and to the data collection 
team: Ronald Mukuye, David Mugisa, Esther Nassonko, and Anthony Begumisa. 
The author also thanks several Population Council colleagues for their inputs at various 
stages of the process: Dr. Robert Miller and Deborah Weiss for their review of this report; 
Hena Khan for editing; Sherry Hutchinson for layout; and Norah Omenda for administrative 
and logistics support. 
Finally, special thanks to the participants in the evaluation, including representatives from the 
Government of Uganda, the PEPFAR Country Team, the staff of MEEPP, and the prime 
partners and sub-grantee implementing partners, for sharing their perspectives on MEEPP 
and broader HIV monitoring and evaluation activities in Uganda. 
This report was made possible through support provided by the United States 
Agency for International Development (USAID)/Uganda under the terms of 
Contract No. GHH-I-02-07-00034-00. The contents are the responsibility of 
the authors and do not necessarily reflect the views of USAID or the United 
States Government. 
The Population Council conducts research worldwide to improve policies, programs, and products in three 
areas: HIV and AIDS; poverty, gender, and youth; and reproductive health. www.popcouncil.org 
Published in January 2010. 
Copyright © 2010. United States Agency for International Development. 
Suggested citation: Kalibala, Samuel. 2010. “Monitoring and Evaluation of the Emergency Plan Progress 
(MEEPP): End-of-Project Evaluation,” Final Report. New York: Population Council.
This document may be reproduced in whole or in part without permission of the Population Council provided 
full source citation is given and the reproduction is not for commercial purposes.
Table of Contents 
Acronyms ........................................................................................................................ i 
Executive Summary ...................................................................................................... 1 
1. Introduction ............................................................................................................... 8 
1.1. Background ..................................................................................................................... 8 
1.2. Purpose of Evaluation ................................................................................................... 10 
2. Methodology ............................................................................................................ 12 
2.1. Data Collection Methods .............................................................................................. 12 
2.2. Data Analysis and Management .................................................................................. 15 
2.3. Ethical Considerations .................................................................................................. 15 
2.4. Limitations .................................................................................................................... 15 
3. Results ..................................................................................................................... 17 
3.1. Overall, was the MEEPP design able to address the M&E needs of PEPFAR and 
GOU? ................................................................................................................................... 17 
3.2. Overall, was MEEPP implemented as designed? ......................................................... 18 
3.3. What has been the trend of quality of data reported to OGAC over the last five 
years? ................................................................................................................................... 20 
3.4. What is the status of electronic databases? ................................................................... 22 
3.5. What has been the effects/impact of MEEPP on the reporting capacity of PEPFAR￾funded IPs? .......................................................................................................................... 26 
3.6. What has been the effects/impact of the MEEPP project on the national M&E 
capacity? .............................................................................................................................. 27 
3.7. In summary, what are the lessons that can be learned? ................................................ 28 
4. Conclusions ............................................................................................................. 31
5. Recommendations .................................................................................................. 33 
Appendix 1: Documents Reviewed ............................................................................ 36 
Appendix 2: Questionnaires ....................................................................................... 38 
Master Guide for Key Informant Interviews and In-depth Interviews ................................ 38 
Checklist for Observation of Data Management System ..................................................... 41 
Appendix 3: Prime Partners ....................................................................................... 43 
Appendix 4: Matrix for Selecting Prime Partners and Implementing Partners 
Sample ......................................................................................................................... 45 

MEEPP: End-of-Project Evaluation 
i 
Acronyms
ACP AIDS Control Program 
AIDS Acquired Immunodeficiency Syndrome 
CDC US Centers for Disease Control and Prevention 
COP Country Operational Planning 
COTR Contracting Officer Technical Representative 
DQA Data Quality Assurance 
FY Fiscal Year 
GOU Government of Uganda 
NIH US National Institutes of Health 
HIV Human Immunodeficiency Virus 
HMIS Health Management Information System 
IDI In-Depth Interview 
IP Implementing Partner 
KII Key Informant Interview 
MEEPP Monitoring and Evaluation of Emergency Plan Progress 
M&E Monitoring and Evaluation 
MEMS Monitoring and Evaluation Management Services 
MOH Ministry of Health 
MGLSD Ministry of Gender, Labor, and Social Development 
OGAC Office of the US Global AIDS Coordinator 
OVC Orphans and Vulnerable Children 
PEPFAR US President’s Emergency Plan for AIDS Relief 
PMTCT Prevention of Mother-to-Child Transmission 
SI Strategic Information 
TASO The AIDS Support Organization 
TWG Technical Working Group 
USAID United States Agency for International Development 
UAC Uganda AIDS Commission 
USG United States Government 

MEEPP: End-of-Project Evaluation 
1 
Executive Summary 
Introduction 
The U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) supports a comprehensive 
HIV/AIDS prevention, care, and treatment program in Uganda. PEPFAR funds more than 80 
percent of HIV/AIDS activities in Uganda through implementing partners (IPs), including 70 
prime partners, which receive direct grants from PEPFAR, and which in turn sub-grant about 
1,000 HIV/AIDS organizations. 
The Monitoring and Evaluation of the Emergency Plan Progress (MEEPP) is a United States 
Agency for International Development (USAID)/Uganda-funded project that has been 
implemented by Social & Scientific Systems, Inc. since December 2004. MEEPP was 
designed to achieve the following results: 1) improved availability of PEPFAR data; 2) 
improved quality of PEPFAR data; 3) improved monitoring and evaluation (M&E) capacity 
for PEPFAR-funded partners; and 4) improved coordinated response. 
MEEPP has established and operates a data management system to collate, clean, validate, 
and analyze service output data of PEPFAR-funded partners, and submit them to the 
PEPFAR Country Team in Uganda. The prime partners collate data provided to them by IPs 
located in the periphery. MEEPP receives data that are uploaded online. To ensure data 
quality, the US Government (USG) Technical Working Groups (TWGs) and MEEPP provide 
capacity building on PEPFAR reporting requirements to prime partners and carry out data 
quality assurance (DQA), which involves cleaning and validating the data as they are being 
entered and afterwards. MEEPP also provides feedback to IPs in meetings about the quality 
of their data as well as scheduled and on-demand visits to carry out on-site DQA. 
Purpose of the Evaluation 
In August 2009, the Population Council was contracted by USAID/Uganda to evaluate 
MEEPP. The purpose of the evaluation, as stated in the scope of work, was to determine 
lessons learned, and to critically examine the aspects of the project that may need change in 
order to address gaps, namely, national M&E system’s development and capacity building for 
host-country M&E and other specialists. The evaluation was designed to answer seven 
questions: 1) Was the MEEPP design able to address the M&E needs of PEPFAR and the 
Government of Uganda (GOU)? 2) Was the MEEPP project implemented as designed? 3) 
Did the quality of PEPFAR data improve? 4) What is the status of the electronic database of 
PEPFAR data? 5) What was the impact of MEEPP on the M&E capacity of PEPFAR-funded 
implementing partners? 6) What was the impact of MEEPP on national M&E capacity? 7) 
What were the lessons learned and aspects that may need change? 
2 
Methodology 
The analytical framework for this evaluation was based on two basic principles: a) the 
purpose of the evaluation and the seven evaluation questions as stated scope of work; and b) 
the mandate of MEEPP as stated in the four major results. The evaluation used four data 
collection methods: a literature review, key informant interviews (KIIs), in-depth interviews 
(IDIs), and observation of data management systems. A total of 23 KIIs were conducted with 
respondents from the USG PEPFAR Country Team, USG TWGs, USAID Monitoring and 
Evaluation Management Services (MEMS), the Ministry of Health (MOH), the Uganda 
AIDS Commission (UAC), the Ministry of Education and Sports, and the Ministry of Gender, 
Labor and Social Development (MGLSD). IDIs were conducted with respondents from 18 
PEPFAR-funded prime partners and 14 sub-grantees (see Appendix 4). Six IDIs were 
conducted with respondents from district health offices. Observations of data management 
systems were carried out at the offices of MEEPP, two prime partners, one sub-grantee IP for 
each of the prime partners, UAC, and the MOH Resource Center. 
The literature review was used to generate tentative findings under each evaluation question 
and included: quarterly reports of MEEPP performance, as well as explanatory notes that 
MEEPP produced to accompany its semi-annual and annual reports, and reports of aggregate 
service delivery data compiled according to specific USG funding agencies (Centers for 
Disease Control and Prevention, USAID, Department of State, National Institutes of Health 
(NIH), Peace Corps, and Department of Defense). The evaluation team also reviewed 
PowerPoint presentations summarizing the MEEPP project as well as the DQA process, and a 
report detailing the DQA conducted for a prime partner. Finally, the team reviewed literature 
about PEPFAR-funded activities in Uganda obtained from the PEPFAR website. 
Data from IDIs, KIIs, and observations of data management systems were used to validate the 
tentative findings from the literature review. Data collection and analysis ran concurrently. 
The evaluation team held de-briefing meetings every two days to identify main emerging 
themes, which were shared with USAID staff, the PEPFAR Country Team, and then with 
MEEPP. Through this process a list of recommendations for changes in the MEEPP follow￾on project were developed and discussed. 
Results 
Results are presented in accordance with the seven evaluation questions. 
1. Overall, was the MEEPP design able to address the M&E needs of PEPFAR and 
GOU? 
The MEEPP design was able to meet the reporting requirements of PEPFAR, providing rapid 
reporting of outputs using up-to-date indicators. However, USG respondents underscored that 
MEEPP was never intended to address the M&E needs of GOU. Instead, USG procured other 
means of technical assistance to UAC and MOH/Resource Center, while MEEPP was 
expected to work with USG to provide clean, unduplicated data on PEPFAR-funded activities 
MEEPP: End-of-Project Evaluation 
3 
that contribute to the results of the national HIV/AIDS response. However, the MEEPP 
design and operation does not involve a structured approach to strengthen national M&E 
systems nor to ensure harmonization, aggregation, and coordination with GOU. GOU 
respondents indicated that MEEPP should have been embedded in a GOU agency in order for 
it to work synergistically with GOU M&E efforts. Instead, MEEPP undermined the 
functionality of GOU M&E systems because MEEPP commands the vast majority of HIV 
program data in the country. 
2. Overall, was the MEEPP project implemented as designed? 
MEEPP was implemented as designed, i.e., to provide the PEPFAR Country Team with 
timely, high quality output data from the prime partners which received data from their sub￾grantee IPs. Since early 2005, the USG TWGs and MEEPP have consistently produced the 
required semi-annual and annual program reports. Once the data are analyzed, MEEPP posts 
reports on the website which can be accessed, using passwords, by USG agencies, prime 
partners, and personnel of selected relevant GOU agencies. However, this design is not 
without consequences: tight PEPFAR deadlines require that both MEEPP and prime partners 
suspend other M&E activities during the three-week data entry period. In addition, sub￾grantee IPs have different capacities to deliver their data to prime partners. 
Data generated by MEEPP are intended to be used to strengthen PEPFAR programming in 
Uganda. On a strategic planning level, MEEPP provides data, including ad hoc analyses, to 
assess progress and to facilitate the Country Operational Planning (COP) process. At the 
program implementation level, the data are used to assist individual IPs to improve their 
performance and to encourage partners to seek synergies. Host country policy makers also 
reported using MEEPP data, but in a less structured process, for example, to prepare the 
Global Fund application in 2007 and the National Strategic Plan in 2008. 
3. What has been the trend of quality of data reported to OGAC over the last five 
years? 
USG respondents were satisfied with the quality of data that PEPFAR/Uganda reported to 
OGAC, to the credit of MEEPP. MEEPP has strengthened the capacity of prime partners 
through DQA and feedback on quality of data. However, there are a number of issues that 
continue to adversely affect the quality of data. The absence of national identity cards limits 
the effectiveness of the unique identifier number strategy. Despite efforts to coordinate 
among USG agencies, multiple PEPFAR-funded IPs provide the same type of service (e.g., 
HIV counseling and testing) in one facility, which results in duplication if clients receive the 
same service from more than one IP at that site. Other factors affecting data quality include 
the short deadlines for prime partners to enter data, which do not always allow sufficient time 
to verify and clean data; changes in PEPFAR indicators on short notice, which preclude some 
IPs from reporting on them; and challenges related to prevention interventions that use mass 
media and can only be reported using estimates. It should also be noted that some of the IPs 
are constrained by staff shortages and although this may affect the quality and timeliness of 
data, it is beyond the mandate of MEEPP to address this issue. 
4 
 4. What is the status of the electronic database of PEPFAR data? 
MEEPP has upgraded its database technology to ensure compatibility with various internet 
browsers. While the database is easily accessible to prime partners during the data entry 
periods, slow internet connections require some prime partners to work at night and/or go to 
the MEEPP offices to enter the data. With regard to inter-operability of data-based systems, 
the MEEPP system is adaptable to GOU and other stakeholder needs since many of the 
indicators are similar or relate to national indicators. However, the HMIS is not compatible 
with MEEPP. 
According to staff of USG and of prime partners, the database is user-friendly and is being 
utilized by stakeholders. The prime partners, USG staff, and GOU agencies have access to 
the database, using passwords, at all times and can download reports. However, MEEPP 
respondents indicated that a number of users find excuses, such as loss of passwords, to avoid 
accessing the database directly, and instead request data from MEEPP. In addition, some 
stakeholders reported that the customized reports are insufficiently detailed for decision 
making and planning purposes—a reason for requests for further analyses. The data are also 
only available on a semi-annual and annual basis, yet a number of stakeholders seek monthly 
or at least quarterly data, resulting in additional reporting requirements for IPs. 
After some attention to the development of related skills in database management, DQA, 
analysis, and generation of reports, it is feasible for MEEPP to transfer its database to a prime 
partner or GOU agency. This should entail a phased process starting with merging of data 
collection tools. 
5. What has been the effects/impact of MEEPP on the M&E capacity of PEPFAR￾funded IPs? 
Prime partners, including those with good M&E systems before MEEPP, unanimously 
reported that MEEPP has helped them organize their data systems through strict deadlines, 
feedback, and data quality assurance procedures. While MEEPP’s data validation meetings 
were generally viewed as a useful capacity building tool, some prime partners expressed a 
desire for more focused support. In addition, though it is beyond the mandate of MEEPP to 
support the M&E function of IPs directly, some prime partners maintained that the capacity 
building provided by MEEPP stops at the level of the prime partners and does not sufficiently 
trickle down to the IPs. This gap is a result of insufficient prime partner M&E budgets, or 
because their M&E department was integrated with service delivery departments, thus 
receiving limited attention. Reporting from MOH sites may also be constrained by low 
capacity among staff. 
6. What has been the effects/impact of the MEEPP project on the national M&E 
capacity? 
MEEPP and USG respondents agreed that while capacity building of GOU is not a 
contractual requirement of MEEPP, MEEPP has to some extent supported GOU M&E staff 
MEEPP: End-of-Project Evaluation 
5 
through ad hoc arrangements. Examples include mapping of ART sites, GIS training, and 
technical assistance to UAC’s database assessment exercise. Respondents from MOH, UAC, 
and Ministry of Education and Sports reported that they appreciated MEEPP’s efforts to 
facilitate reporting of quality data and involve staff from GOU. Nevertheless, many USG and 
GOU respondents agreed that the design of MEEPP was not mandated to build the M&E 
capacity of GOU. 
 
7. In summary, what are the lessons that can be learned and what aspects of the 
project may need change in order to address gaps identified in the current project, 
namely inadequate national M&E system’s development and capacity building for 
host-country M&E and other specialists? 
There is no clear strategy for MEEPP to engage public sector stakeholders. A good 
M&E system should be designed to generate data, promote it, and make sure it is accessible 
to decision makers. Yet MEEPP was not designed to ensure host-country use of its data at the 
national or district levels. In particular, MEEPP does not interact with GOU at the district 
level, and there is no system for ensuring that PEPFAR data reaches decision-makers at this 
level. 
Key potential users of MEEPP data are not accessing the data they need. These users 
include GOU staff at the central and district levels, some staff of IPs, and some USG 
Contracting Officer Technical Representatives (COTRs). The COTRs’ concern is that on the 
MEEPP database they only access aggregate data, which does not single out the individual 
IPs that the COTRs are responsible for monitoring. Hence, they request separate reports from 
the IPs, which may cause unnecessary efforts. 
 
Data from non-PEPFAR civil society organizations not working at health facilities tend 
to be left out. PEPFAR data includes data on services delivered by PEPFAR IPs using non￾USG funds as indirect contribution to national results. But, for obvious reasons, data from 
CSOs not funded by the USG are not captured by the PEPFAR database. While these civil 
society organizations are not the responsibility of MEEPP, such data would provide both 
GOU and USG with a more comprehensive context in which PEPFAR recipients are 
operating. 
Conclusions 
In light of these findings, the evaluation team offers the following conclusions: 
1. The MEEPP design was able to address the reporting needs of PEPFAR, but there 
is an emerging concern to address the needs of GOU regarding the HIV/AIDS 
M&E. 
2. In terms of availability and timeliness of output data, MEEPP was implemented as 
designed. However, the customized reports on the MEEPP website are 
insufficiently detailed to meet the needs of some stakeholders. 
6 
3. MEEPP has met the expectations of PEPFAR and the prime partners with regard 
to quality of data; however, several challenges, such as double-reporting, continue 
to affect data quality. 
4. Despite accessibility of the MEEPP website, both in terms of inputting data and 
obtaining reports, some users still make special data requests to MEEPP. 
5. MEEPP has helped prime partners to strengthen their data systems. However 
some primary data sources (i.e., IPs) require support to strengthen their M&E 
capacity. 
6. MEEPP does not have an explicit mandate to build the M&E capacity of GOU 
agencies. However, it is making special efforts to do so on an ad hoc basis. 
7. Aspects of MEEPP that may need change include systematic engagement of 
public sector stakeholders by promoting and supporting utilization of PEPFAR 
data, and increased emphasis on evaluation. 
Recommendations 
The following recommendations are made in the context of existing USG efforts to support 
the national HMIS and UAC’s performance monitoring and management plan (PMMP). It is 
also important to note that, for these efforts to yield tangible outcomes, it is necessary for the 
MOH to demonstrate leadership on the development of HMIS and for districts to coordinate 
collection of data from their health facilities and community-based service facilities. 
1. In designing a follow-on project to MEEPP, USAID should aim to move from an 
“emergency mode” and align it with the national systems of data collection and 
reporting. This would include building capacity of GOU to develop its databases; 
beginning the process of handing over the MEEPP database to a prime partner or 
GOU; continuing to support the establishment of a national database that 
incorporates all HIV and AIDS data in Uganda; harmonizing HMIS and PEPFAR 
data collection tools and promoting their use; and linking with the roll-out of 
UAC’s PMMP. 
2. USAID should consult with MOH, UAC, and other host-country users of 
PEPFAR data to devise ways to address their needs and ensure optimal utilization 
in policy making and program planning. Ideally, all requests for data and ad hoc 
analysis should be sent to UAC, and GOU users should receive quarterly reports 
from UAC. However, in the short-term or transition phase, this should be done by 
MEEPP while simultaneously building capacity of UAC to assume this 
responsibility. 
3. Realizing that MEEPP has so far concentrated on the monitoring aspects of M&E, 
there is the need for USG to work with GOU and other partners to strengthen the 
current outcome evaluation framework, which largely relies on occasional 
national surveys as well as epidemiological studies carried out by specific 
MEEPP: End-of-Project Evaluation 
7 
research groups. Additionally, each USG agency has plans for end-of-project 
evaluations. These plans should be linked with the overall national evaluation 
framework. 
4. The follow-up project to MEEPP should support and monitor prime partners to 
ensure that they provide systematic M&E support to IPs at peripheral sites. This 
element could be included in the DQA process that MEEPP provides to prime 
partners. 
8 
1. Introduction 
1.1. Background 
The U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) was launched in 2003 to 
combat global HIV/AIDS. Uganda is one of PEPFAR’s 15 focus countries, receiving 
approximately $91 million in Fiscal Year (FY) 2004, $148 million in FY 2005, $170 million 
in FY 2006, $237 million in FY 2007, and $284 million in FY 2008 to support a 
comprehensive HIV/AIDS prevention, care, and treatment program. PEPFAR funds more 
than 80 percent of HIV/AIDS activities in Uganda through implementing partners (IPs), 
including 70 prime partners (see Appendix 3), which receive direct grants from PEPFAR, and 
which in turn fund about 1,000 sub-grantee IPs that implement HIV/AIDS services. 
Strategic information (SI) is a key component of PEPFAR and is used “to assist host country 
governments to plan, monitor, and manage a coordinated national response to the HIV/AIDS 
epidemic; to assist PEPFAR country teams to plan, monitor, and manage USG HIV/AIDS 
activities in support of the national plan; to provide information to PEPFAR Headquarters for 
management of PEPFAR; to demonstrate progress of PEPFAR in each annual report to the 
U.S. Congress; to advocate for continued support and resources of HIV/AIDS prevention, 
care, and treatment programs; to coordinate efforts with the international donor community.”1
In Uganda, SI activities are supported by the Monitoring and Evaluation of the Emergency 
Plan Progress (MEEPP) project, which has been implemented by Social & Scientific 
Systems, Inc. since December 2004. MEEPP has the broad aim of implementing a 
comprehensive performance management, monitoring, and reporting program for the 
PEPFAR Country Team in Uganda, which includes the Centers for Disease Control and 
Prevention (CDC), Department of Defense, Department of State, National Institutes of Health 
(NIH), Peace Corps, and the United States Agency for International Development (USAID). 
MEEPP is funded by a $7,536,180 contract with USAID/Uganda. 
Prior to MEEPP, some PEPFAR-funded IPs had not integrated PEPFAR indicators into their 
performance monitoring plans (PMPs), although the data were available. In other words, 
there was no linkage between routine reporting and PEPFAR reporting. This was most 
notable for organizations that were already recipients of other US Government (USG) grants, 
and were routinely reporting on these, after which they received PEPFAR grants. 
The PEPFAR Country Team collects, cleans, and analyze data on PEPFAR-funded activities, 
and conducts special studies. There are four major results that MEEPP set out to achieve: 
1. Improved availability of PEPFAR data 
2. Improved quality of PEPFAR data 
3. Increased monitoring and evaluation (M&E) capacity 
4. Improved coordinated response 
 
1
 The President’s Emergency Plan for AIDS Relief. 2009. Next Generation Indicators: Reference Guide. 
Available at: http://www.pepfar.gov/documents/organization/81097.pdf. 
MEEPP: End-of-Project Evaluation 
9 
To achieve these results, MEEPP engages in the following activities: 
Information collection 
MEEPP collects both qualitative and quantitative information from PEPFAR-funded IPs 
based on a common understanding of PEPFAR indicators and reporting requirements. 
MEEPP has developed an online Partner Performance Management Information System, into 
which IPs enter semi-annual and annual data during two, three-week windows. Quantitative 
data are entered on site-specific data screens and include outputs such as clients receiving 
services or reached by education activities, and number of providers trained. Qualitative data 
is usually presented as a narrative and includes achievements, constraints, and how 
constraints were overcome. Prime partners have M&E departments that collate data from 
their sub-grantee IPs, and then enter the data into databases. 
Ongoing data quality assurance 
Data quality assurance (DQA) is provided in several ways for each semi-annual or annual 
reporting cycle. During data entry, MEEPP supports prime partners using an interactive 
online approach, reviewing data as they are entered. Secondly, the USG Project Officers or 
Contracting Officer Technical Representatives (COTRs) review reported data and provide 
feedback. Thirdly, USG Technical Working Groups (TWGs) and MEEPP use master lists of 
service outlets to assess and eliminate reporting overlaps. MEEPP also cleans and validates 
data during and after each reporting period. Finally, USG TWGs and MEEPP staff schedule 
DQA visits to prime partners and sites, and hold feedback meetings with all prime partners to 
present the data of that reporting period and to discuss data quality issues.
Data analysis and generation of reports 
After the data have been entered and cleaned, MEEPP staff generates and, with the USG 
TWGs, reviews reports to assess overlaps and synergies in service delivery. They also map 
services by type and volume and relate the services to the HIV burden. This analysis helps to 
show the contribution of USG support to the national response and to identify areas for 
special studies. The MEEPP database generates aggregate program outputs in the form of 
semi-annual and annual reports. It also generates reports specific to each USG agency (i.e., 
USAID, CDC, Department of Defense, Department of State, NIH, and Peace Corps), for each 
IP, and for each program area [e.g., prevention of mother-to-child transmission (PMTCT) or 
palliative care]. Other types of reports include district-level aggregate reports by program 
area, coverage maps for each program area, and master lists of program outputs by prime 
partners, district, sub-county, and service outlet. In addition, a number of downloads, 
including graphs, charts, presentations, and comparative trend data, are available for 
stakeholders, such as USG and selected GOU staff who have access rights to the system. 
10 
Collaboration and coordination 
MEEPP coordinates closely with USG TWGs on HIV/AIDS, particularly with the SI TWG, 
which supervises and directs MEEPP, approving MEEPP’s annual work plan, DQA visits, 
and special studies before implementation. Finally, the SI Group ascertains if IPs are in 
compliance with MEEPP procedures and reviews and approves prime partner aggregate 
reports after they are generated by the MEEPP database. 
A variety of groups (clients) use MEEPP’s services as shown in Figure 1. 
Figure 1 User of MEEPP services (clients) 
MEEPP has nine Ugandan staff, including the Project Director, two M&E Specialists, one 
Management Information Specialist, and Finance and Administrative staff. In addition, 
MEEPP has one part-time expatriate Senior Advisor. 
1.2. Purpose of Evaluation 
The current MEEPP project is a five year project implemented from December 2004 to 
December 2009. In August 2009, USAID/Uganda contracted the Population Council to 
conduct an end-of-project evaluation of MEEPP. USG is designing a follow-up project to 
MEEPP, and this evaluation was aimed to inform that process. The purpose of the evaluation, 
as stated in the scope of work, is to critically examine the project to extract the lessons 
learned, including what did and did not work well, and what needs to be changed in the future 
with special reference to capacity building of PEPFAR-funded IPs and the national M&E 
system. To achieve this, the evaluation sought to answer the following questions: 
MEEPP 
MOH-ACP 
MOH-Resource Centre 
USAID MEMS
UAC
Sub-grantee IPs 
(N=1,000) 
MOES 
MOGLSD 
Prime Partners 
(N= 72) 
USG 
MEEPP: End-of-Project Evaluation 
11 
1. Overall, was the MEEPP design able to address the reporting needs of PEPFAR and 
the GOU? 
2. Overall, was the MEEPP project implemented as designed? 
3. What has been the trend of quality of data reported to the PEPFAR Country Team 
over the last five years? What has been the role of MEEPP in this regard? 
4. What is the status of electronic databases of PEPFAR data with regard to 
comprehensiveness, compatibility with national systems, ease of use, actual use, and 
plans or feasibility for handing them over to an appropriate institution once the project 
is ended? 
5. What has been the effects/impact of the MEEPP project on the M&E capacity of 
PEPFAR-funded IPs? 
6. What has been the effects/impact of the MEEPP project on the national M&E 
capacity? (This result was evaluated on the extent to which the project was designed 
to support the national system.) 
7. In summary, what are the lessons that can be learned and what aspects of the project 
may need change in order to address gaps identified in the current project, namely 
national M&E system’s development and capacity building for host-country M&E 
and other specialists? 
It should be noted that MEEPP is only one of many organizations engaged in M&E work to 
ensure a coordinated Ugandan response to HIV/AIDS. Other organizations include the MOH 
AIDS Control Program and Resource Center and UAC. The evaluation team therefore sought 
to learn the extent to which MEEPP interacted with these other groups to ensure a 
coordinated and harmonized M&E system for the country. 
The sections that follow describe the methodology used in this evaluation, the results 
organized by the seven evaluation questions, conclusion, and recommendations for a follow￾on project. 
12 
2. Methodology 
2.1. Data Collection Methods 
The evaluation team began with an extensive literature review to learn about the monitoring 
and reporting needs of OGAC and to examine the reports submitted to the PEPFAR Country 
Team. It then conducted in-depth interviews (IDIs) with key technical staff of IPs, GOU 
agencies, and USG agencies, as well as key informant interviews (KIIs) with executive heads 
of these agencies. Finally, the team conducted a technical review of the database to assess its 
status and usability, and examined collection tools and registers at selected service delivery 
sites. The data were collected between September and November 2009. 
Literature review 
The literature review included quarterly reports of MEEPP performance (first quarter of 2005 
to third quarter of 2009), as well as explanatory notes that MEEPP produced to accompany 
each semi-annual and annual report from 2005 to 2008, and reports of aggregate service 
delivery data compiled according to specific USG funding agencies (CDC, Department of 
State, NIH, USAID, Peace Corps, and Department of Defense). The evaluation team also 
reviewed PowerPoint presentations summarizing the MEEPP project, as well as the DQA 
process and a report detailing the DQA conducted on one of the prime partners (NUMAT). 
In addition, the team reviewed literature about PEPFAR/Uganda obtained from the PEPFAR 
website. (These documents are listed in Appendix 1.)
During the literature review, the evaluation team sought to identify achievements and lessons 
learned from MEEPP’s experience from inception to date. The evaluation team assessed 
MEEPP’s products in terms of readability and presentation of information as well as content. 
Finally, the review considered the quality and utility of special studies and analyses and 
feedback meetings as reported in the quarterly reports. 
Key informant interviews 
The purpose of the KIIs was to evaluate the programmatic aspects of MEEPP. This included: 
client satisfaction; capacity building; facilitation of a coordinated national reporting system 
and harmonization; facilitation of sharing of M&E data, including feedback to districts and 
service outlets; supporting utilization of data for programmatic and policy decisions; 
strengths and weaknesses; and recommendations for strengthening. Further, the evaluation 
team used KIIs and IDIs (described below) to ascertain the views of various users and 
participants on the quality and utility of analyses, feedback meetings, and special studies. 
Finally, KIIs examined the ease with which the data on the MEEPP website are accessed, the 
extent to which these data are put to use, lessons learned, as well as challenges and 
constraints and how these can be overcome. 
MEEPP: End-of-Project Evaluation 
13 
Key informants represented agencies that are beneficiaries of and stakeholders in MEEPP’s 
work. They were identified by the evaluation team with the aim of having at least one person 
representing each agency shown in Figure 1, excluding the prime partners and their sub￾grantee IPs. This list was reviewed by the SI TWG, and modifications were made. A total of 
23 KIIs were conducted—22 in Kampala and one in Entebbe. 
The same interview guidelines were used for the KIIs and the IDIs (Appendix 2). The 
difference was in the depth and structure of the interviews. The interviewers were 
experienced researchers and were able to tailor the interviews so that KIIs were less 
structured and allowed respondents to take their time and focus on a particular aspect of the 
interview that they deemed most relevant, which the interviewers would follow to a logical 
concluding point. In addition, some KIIs were intentionally scheduled later in the evaluation 
process so interviewers could use them to validate or refute emerging themes. 
In-depth interviews
The purpose of the IDIs was to obtain the views of hands-on implementers of M&E on the 
user-friendliness of the system developed by MEEPP, awareness and availability of standard 
operating procedures and data collection tools, supervisory support received, lessons learned, 
challenges, strengths and weaknesses, and recommendations for strengthening the MEEPP 
Project. 
The IDI respondents represented users along the path of information flow: service providers 
who complete the MIS forms at sites; those who make site summaries and forward them to 
prime partners; those who collate summaries at the headquarters of prime partners; those who 
enter data into the MEEPP database online; technical staff at MEEPP; and those who receive 
(or are supposed to receive) the consolidated reports from MEEPP. Since more than one 
person at a site or agency may be involved in these processes (e.g. program staff and M&E 
staff), the IDIs sometimes involved more than one respondent at an agency. 
Both prime partners and sub-grantee IPs were represented in the IDIs. To guide the selection, 
a matrix of prime partners (see Appendix 4) was developed by the evaluation team using the 
following criteria: 
• The USG agency supporting the prime partner: USG agencies funding the largest 
numbers of prime partners (e.g., USAID and CDC, which fund 39 and 23 prime partners, 
respectively) were allocated a larger share of prime partners from which to conduct IDIs. 
However, a deliberate effort was made to ensure at least one prime partner of the other 
USG agencies was visited. 
• The program area covered: Using data provided by MEEPP, the evaluation team 
identified and selected prime partners who contribute a high percentage of output data for 
a given service area such as HIV testing and counseling or orphans and vulnerable children 
(OVC). 
• Geographical regions: The team sought to ensure diversity of distribution across the four 
major regions of the country (Northern, Western, Eastern, and Central). 
14 
This matrix was reviewed, modified, and approved by the SI TWG and sent to MEEPP to 
provide contact information, and where possible, to facilitate appointments. 
Prime partner interviews were conducted first—all in Kampala where their headquarters are 
located, except for one (for which the headquarters are located in Gulu). Following these 
interviews, the evaluation team selected one sub-grantee IP for each prime partner to be 
visited. Because of cost and time limitations, the sample of IPs was conveniently selected to 
suit certain routes of travel for the study team, as follows: 
• Eastern route: Bugiri, Mbale 
• Northern route: Gulu , Masindi 
• Western route: Fort Portal, Mbarara, Masaka, Mubende, Ntungamo, 
• Central: Kampala, Kayunga, Luwero, Mpigi, Wakiso 
The list of IPs was reviewed and agreed on with the SI TWG, and sent to MEEPP to provide 
contacts and assistance with making appointments. For each district visited, the team made an 
effort to visit the District Health Office to obtain its views about MEEPP. 
Interviews were conducted with 25 staff representing 18 prime partners and 17 staff 
representing 14 sub-grantee IPs.2
 In addition, six interviews were conducted with District 
Health Offices. 
Observation of the data management system 
An experienced database consultant conducted the observation of the data management 
system. From the literature review, the evaluation team determined that the MEEPP database 
was well developed and functioning optimally. Thus, the observation of the data management 
system was largely descriptive, and included a review of the types and processes of inputs 
and outputs. A checklist (see Appendix 2) was used to follow the pathway from the sites 
where data are collected by service providers to where information is transferred from the 
source documents and entered electronically. This path included headquarters offices and 
M&E departments of prime partners. In addition to observations, the consultant sought the 
views of staff regarding the user-friendliness of the tools, the clarity of standard operating 
procedures, and the level and quality of support received from MEEPP. 
Observations of the data management system were carried out at MEEPP offices, two GOU 
M&E systems, and two prime partners with corresponding IPs in Eastern and Central regions. 
These sites were purposively selected based on logistical reasons. 
 
2
 For one prime partner, Walter Reed, two IPs were interviewed while for Peace Corps Mubende, a peripheral 
IP, no corresponding prime partner interviewed. For logistical reasons, no IPs were interviewed for AFFORD, 
CSF, Mildmay, and PREFA. 
MEEPP: End-of-Project Evaluation 
15 
2.2. Data Analysis and Management 
The analytical framework for this evaluation was based on two basic principles: a) the 
purpose of the evaluation and the seven evaluation questions as stated scope of work; and b) 
the mandate of MEEPP as stated in the four major results. The evaluation team used the 
literature review to generate tentative findings for each evaluation question. Qualitative data 
from IDIs, KIIs, and observations of the data management system were then used to either 
validate or invalidate these tentative findings. Data collection and analysis ran concurrently. 
The evaluation team held de-briefing meetings every two days to identify main emerging 
themes. Important findings requiring further inquiry were pursued on subsequent days. 
Lines of inquiry that were duplicating existing information were dropped from subsequent 
interviews. Themes that appeared well-established from initial data were written up as part of 
the preliminary evaluation report (but could be changed if they were disputed by later data). 
Emerging themes were shared among the evaluation team, the USAID staff, the Country 
Team, and finally with MEEPP. These sharing sessions enabled the various stakeholders to 
seek clarifications, add more data, add more lines of inquiry, or dispute tentative conclusions. 
In some instances, the evaluation team went back to some respondents after the sessions to 
ask further questions or seek clarifications. 
Through this process, the team developed a list of recommendations for the MEEPP follow￾on project. These recommendations were extensively discussed with the COTRs for MEEPP. 
All interviews were recorded on audio tapes and transcribed into Microsoft Word documents. 
The audio tapes and the printed transcripts were stored under lock and key. Notes from 
sharing sessions and de-briefing sessions were taken by hand and typed into Microsoft Word. 
All Word documents were then read and coded according to themes, which were developed 
by the evaluation team when discussing initial interviews and refined as more data became 
available. The findings were then organized along the seven evaluation questions. 
2.3. Ethical Considerations 
This evaluation methodology was reviewed internally within the Population Council and 
exempted from the full Institutional Review Board review on the grounds that it is not a 
clinical study, does not involve vulnerable populations, confidential medical information, or 
physically invasive procedures, and would not expose participants to health, legal, or 
psychological risks. To ensure that participants’ responses remain confidential, respondents 
were interviewed individually in the privacy of their offices; all transcripts and audio tapes 
were kept under lock and key in the Council’s data room, and personal identifiers have been 
removed when respondents are quoted in this report and in presentations of the data. 
2.4. Limitations 
The scope of this study was limited to MEEPP and thus the results it was contracted to 
achieve. However, as noted above, MEEPP interacts with several other M&E systems, such 
16 
as the GOU M&E systems, which are intended to coordinate the M&E efforts of donors such 
as PEPFAR. A comprehensive assessment of MEEPP’s role in collaboration, coordination, 
and harmonization would have to include these other M&E systems. Thus, the conclusions of 
this evaluation do not reflect the strengths and weaknesses of these systems and how they 
could have affected the success of MEEPP. Another limitation of the evaluation is the 
absence of a baseline assessment of the HIV M&E system in Uganda before the 
implementation of PEPFAR-funded activities. As such, it is difficult to discern the effects of 
PEPFAR’s broader focus on a coordinated USG response from those of MEEPP’s efforts to 
foster collaboration among USG agencies. Further, the selection of prime partners included in 
the evaluation intentionally favored the prime partners reporting the largest absolute numbers 
served in a given program area. This was because they were considered to have the largest 
influence on the data reported. However, it inevitably biased the findings towards the 
experiences of the larger organizations at the expense of the smaller ones. Finally, given cost 
and time limitations, the sample of IPs was conveniently selected to suit travel routes of the 
study team, which could have introduced a bias by excluding more remote IPs. 
MEEPP: End-of-Project Evaluation 
17 
3. Results 
The following section presents the results of the evaluation organized by the seven evaluation 
questions. 
3.1. Overall, was the MEEPP design able to address the M&E needs 
of PEPFAR and GOU? 
If you wanted data about different partners in the country, about what they doing, 
where they are working, the place to go is MEEPP. 
 –Prime partner staff person 
Respondents consistently reported that the MEEPP design was able to meet the reporting 
needs of PEPFAR. As an emergency program, PEPFAR requires timely and accurate 
reporting of its outputs to support ongoing funding. MEEPP has been able to meet this need, 
designing reports based on PEPFAR indicators and meeting Country Team’s deadlines. 
PEPFAR indicators are subject to change, sometimes on short notice, and MEEPP has 
generally been able to reflect these changes in its reports. However, such changes also cause 
considerable stress for some IPs, which struggle to meet deadlines, report on some indicators, 
and enter data with proper cleaning or verification. In some cases IPs miss reporting periods 
altogether. It should be noted that the issue of changing indicators is a Country Team issue 
and not within the control of MEEPP. On the other hand, some respondents reported that 
strict deadlines help the IPs to streamline and organize their work. 
USG respondents acknowledged that it was never the intention of USG to support the M&E 
needs of GOU through MEEPP. At the same time, in commissioning this evaluation, the 
USG team sought to determine ways to address gaps identified in the current project, namely, 
national M&E system’s development and capacity building for host-country M&E and other 
specialists. Thus, the evaluation team specifically investigated the extent to which MEEPP 
was meeting the M&E needs of GOU. 
 
USG respondents underscored that MEEPP was designed by PEPFAR for emergency 
purposes of providing PEPFAR data in order to guide the emergency plan and hence there 
was limited time for local consultations to take into consideration the role of GOU. Instead 
the USG procured other means of providing technical assistance to GOU to strengthen its 
M&E system, including the AIDS Capacity Enhancement project to support database 
development for the UAC, a contract with the Population Council to provide technical 
assistance to UAC to roll out its PMMP, and strengthening the MOH’s HMIS system through 
support to the Resource Centre. 
Nonetheless, GOU respondents widely maintained that MEEPP is not meeting GOU 
expectations and needs. These expectations could be a product of two factors. First, the vast 
majority of HIV/AIDS programming in Uganda is funded by the USG, and MEEPP collects a 
similar proportion of output data from HIV/AIDS services in the country. Secondly, MEEPP 
is considered the most reliable source of service delivery data in the country. Thus, the 
18 
volume and quality of data collected by MEEPP may have fostered the perception that 
MEEPP has overtaken the HIV/AIDS M&E function of GOU. As a result, several 
respondents felt that MEEPP should have been embedded in a key GOU structure such as the 
Resource Center or UAC. 
3.2. Overall, Was MEEPP Implemented as Designed? 
Providing data on PEPFAR funded activities as required by the Country Team 
MEEPP was designed to provide the Country Team with timely, high quality output data 
from all sites funded by PEPFAR in Uganda. It was the unanimous view of respondents that 
MEEPP was well implemented as designed. OGAC requires each PEPFAR-funded Country 
to prepare two reports per year: a semi-annual report, covering the period 1 October to 31 
March, and an annual report, covering 1 October to 30 September. Since its inception, 
MEEPP has consistently produced these reports on time, delivering the draft semi-annual 
reports to the Country Team in April and the annual reports in October. These reports are 
broken down by geographical region, type of service, type of IP, and reporting period. The 
database also provides a variety of standard reports focusing on program-level indicator 
totals. MEEPP submits reports by e-mail and thereafter sends hard copies to the Country 
Team and prime partners. The evaluation team found that the reports were readily available 
and current. 
However, this design is not without consequences. Because the deadlines are set by OGAC 
and are non-negotiable, meeting them places a burden on all that are involved in the process. 
Indeed, MEEPP and IP respondents reported that prime partners suspend their own M&E￾related work during the three weeks of data entry. Prime partners also spend considerable 
time extracting data from IP reports. 
In addition, the level of sophistication varies at the sites reporting data. The primary data 
sources are the service delivery sites of IPs, many of which are based in the districts at MOH 
health facilities or non-governmental organization settings. For example, Mildmay, a prime 
partner, has provided computers to all its sites for data entry. However, the sites do not have 
internet connections to send the data. Instead, M&E staff from headquarters must travel to the 
sites to obtain the data. On the other hand, Joint Clinical Research Center sites send data by 
e-mail, and the AIDS Information Centre enters data in Epi Info, an epidemiological data 
capture program, sometimes hiring consultants to analyze the data locally. 
Quality assurance of data entered into the database is another key component of MEEPP. 
Respondents reported that MEEPP provides sufficient supervisory support during and after 
the data entry period to ensure both the quality and timeliness of the data. The data quality 
assurance is provided either on demand by the prime partners, or as feedback on the data 
entered by the prime partners. 
MEEPP: End-of-Project Evaluation 
19 
Presenting data on PEPFAR-funded activities to inform programming decisions 
Data generated by MEEPP are intended to be used to strengthen PEPFAR programming in 
Uganda. MEEPP cites several ways the data it generates are put to use. On a strategic 
planning level, data are used to assess overall progress achieved by PEPFAR, to compare this 
progress to strategic country targets as well as specific progress registered in each program 
area, and to facilitate the Country Operational Planning (COP) process. Specifically, MEEPP 
provides the data used by the TWGs to set annual targets for the new COP. In addition, TWG 
members request and receive ad hoc analyses on a variety of programmatic issues. These 
requests frequently included additional analyses to be used in program planning, such as data 
by specific age groups, geographical regions, and program areas. 
At the program implementation level, the data are used to assess progress of and provide 
feedback to individual IPs to improve their performance, and to encourage partners to 
network and seek synergies. Review of outputs also helps prime partners to assess the 
strengths and weaknesses of their M&E systems, and to use their own databases for strategic 
planning. MEEPP provides feedback to the prime partners on the data they reported and on 
comparisons with other prime partners. This feedback covers trend analysis, the extent to 
which national targets are being met, and geographical coverage. Prime partners reported that 
this kind of feedback was vital for planning their activities. For example, Mildmay changed 
its HIV testing strategy when the data revealed that it was mainly reaching low risk 
populations. Similarly, the AIDS Information Centre determined that it needed to recruit 
additional staff to increase coverage and meet its targets. On the other hand, some prime 
partners expressed the view that, in general, routine feedback usually focuses on the quality 
of data reported rather than on the program implications of the data. 
Host country policy makers also report using data on PEPFAR-funded activities, although not 
through a structured routine feedback process such as the one MEEPP conducts for the 
TWGs and prime partners. USG invites officials from GOU and the United Nations agencies 
to attend data dissemination meetings. For example, a dissemination meeting on the semi￾annual report of 2008 included staff from UAC, MOH, MOGLSD, the World Health 
Organization, and UNAIDS. During this meeting PEPFAR contributions and trends in 
performance from 2005 to 2008 were reviewed. MEEPP also reported that, with USG, it 
organized and facilitated consultation meetings with UAC, MOH, and MGLSD to validate 
the data from this report. Finally, MEEPP quarterly reports cite several examples of GOU 
utilizing data on PEPFAR-funded activities: for its Global Fund application in 2007, the 
HIV/AIDS National Strategic Plan in 2008, and for compiling a UNAIDS report of the 
national response in 2009. This information was corroborated by some GOU respondents, 
who reported that they receive service delivery data on PEPFAR-funded activities that helps 
them in program planning. 
What I know is that, PEPFAR helped us to improve our capacity in service delivery. 
MEEPP gave us an opportunity to look at data from the source. I also got a report on 
universal access (to antiretrovirals), that report had support from MEEPP. 
 –GOU staff person 
20 
Fostering collaboration, coordination, and harmonization 
USG staff reported that MEEPP has enhanced collaboration between the agencies comprising 
the TWGs, between the TWGs and IPs, and between the Country Team and the GOU. 
Arguably, it may be difficult to tease out the impact of PEPFAR’s efforts at coordination 
from those of MEEPP. However, some USG respondents singled out MEEPP as being 
responsible. 
I give MEEPP all the credit because it has brought CDC and USAID together. Through 
MEEPP’s work and interaction, it has tried to bridge the gap between these two 
organizations. 
–USG staff person 
On harmonization, [MEEPP] has played a big role; they are better resourced, able to 
follow up on information. Sharing information with us means harmonization and also 
data audits—all are towards harmonization. However, the coordination if any has been 
for PEPFAR partners and collaboration is mainly with PEPFAR partners, but also 
with MOH especially on issues of figures. 
 –MOH staff person 
Prime partners also reported that the feedback meetings and the review of coverage data 
demonstrating duplication of service delivery have enhanced coordination and collaboration. 
3.3. What Has Been the Trend of Quality of Data Reported to OGAC 
Over the Last Five Years? What Has Been the Role of MEEPP in 
This Regard? 
USG respondents indicated that they were satisfied with the quality and timeliness of data 
reported to OGAC over the last five years, giving credit to MEEPP. MEEPP respondents, in 
turn, noted that data quality has also improved with the clarity of the indicators. Often when 
OGAC sends new PEPFAR indicators to the country, prime partners have different 
interpretations about the numerators and denominators that feed into that indicator and how 
that data are to be collected at the site level. MEEPP prevents this confusion by holding 
meetings of all prime partners to reach a clear common understanding of the indicators, thus 
resulting in data that have the same meaning across prime partners. 
MEEPP staff also maintain that MEEPP provides the most accurate HIV/AIDS M&E data to 
the USG and other national partners, such as Joint Clinical Research Center, UAC, and 
MOH. This is corroborated by both USG staff and GOU staff: 
MEEPP, collects, cleans and reports PEPFAR results every year during which the semi 
and annual reports are made. It has also done a good job of using the data to improve 
the program. 
 –USG staff person 
MEEPP: End-of-Project Evaluation 
21 
The information most of the times would be accurate and complete as it comes to 
MEEPP than HMIS, e.g., data on HIV counseling. 
–GOU staff person 
MEEPP has had particular success integrating PEPFAR indicators with the other USG 
indicators, which USG-funded partners had previously struggled to reconcile. This has 
resulted in unified reporting on USG grants. 
MEEPP has brought some order regarding reporting systems. 
–USG respondent 
MEEPP has also strengthened the capacity of IPs to conduct DQA by fostering a common 
understanding of the definitions of indicators. As mentioned above, whenever OGAC sends a 
new PEPFAR indicator, USG convenes the prime partners to ensure they have a consensus 
on what the indicator means and how the data will be collected. USG and MEEPP have 
trained IPs to conduct DQA on the web during and after data entry. Further, web-based 
reporting made it easy for the IPs to input data with minimal errors. 
According to most prime partners, a specific area where MEEPP significantly improved data 
quality was the reduction of double reporting of clients receiving services. Double reporting 
can occur if one client receives a service, such as HIV testing and counseling, twice within 
the reporting period from different IPs in the same health facility. The efforts by MEEPP, 
USG agencies and prime partners to promote the use of unique identification numbers for 
each client have been particularly effective. However, considering that there are no national 
photo identity cards in Uganda, the use of a unique identification numbers has its limitations. 
A second MEEPP effort to reduce double-counting is the feedback meetings of all prime 
partners to review outputs reported and data submitted. During these meetings, outputs are 
compared for specific program areas where two IPs are reporting from the same facility. If 
MEEPP identifies overlaps in the data reported by the two IPs, the data are verified by 
bringing the IPs together to reach consensus. A third approach is that MEEPP’s 
documentation of services delivered is broken down by district, county, sub-county and site, 
with a master list showing the primary source in terms of site and sub-county. Where more 
than one IP operates in the same location offering the same service, the IP reporting the 
largest amount of services is taken as the primary reporting IP, and the data of the other IPs 
are included in the data of this index IP. A fourth approach to reduce double- or over￾counting is that outreach-specific data, which are reported at the parish level, are verified 
with the parish populations. In this case if an IP reports reaching a number which exceeds the 
total parish population, MEEPP highlights this error to the IP and it is corrected. Finally, data 
quality assessment visits to selected sites deal with the issue of double-counting at the IP 
level. 
Another way in which MEEPP strives to improve data quality is through comparisons with 
historical trends in performance. USG and MEEPP hold meetings with prime partners to 
clarify issues and address queries. 
However, respondents also expressed concerns about the quality of data produced by 
MEEPP. Double-counting, particularly when multiple service delivery organizations provide 
22 
the same service at the same health facility, remains a concern, despite the measures MEEPP 
has implemented to avoid it. 
You find many programs in the same facility, e.g., a health centre. Yet there are many 
other areas without the same services which are not reached. Actually, an easy solution 
to the problem of double counting would be to reduce double implementation or 
duplication by partners. 
–Prime Partner staff person 
Other respondents indicated that stringent reporting deadlines undermine data quality. This 
can be especially problematic if prime partners enter data on the website before verifying and 
validating them with source IPs. The introduction of new indicators late in the cycle may also 
affect quality, as some sites are unable to revise their data using the new indicators, and their 
data gets left out altogether. It should be noted that the introduction of new indicators and the 
strict deadlines are issues dictated by OGAC and are beyond the control of MEEPP. A final 
issue is the accuracy of service delivery statistics for prevention interventions, especially 
those involving mass communications. While facility-based data sources (e.g., registers) are 
easier to validate, HIV/AIDS prevention data are mainly based on estimates of the numbers 
reached by mass media and may therefore be less accurate than treatment data. 
3.4. What is the Status of Electronic Databases, With Regard to: 
Comprehensiveness, Inter-Operability With National Systems, Ease 
of Use, Actual Use, and Plans or Feasibility For Handing Over to An 
Appropriate Institution Once the MEEPP Project is Ended? 
Comprehensiveness 
The technology for the MEEPP database was initially based on Microsoft Excel. Recently, 
MEEPP changed to the superior SQL Server 2005 and ASP.NET web-based technology. This 
is because web pages developed in ASP.net technology are not affected by the type of 
internet browser the website visitor is using. MEEPP uses Microsoft applications (Word and 
Excel) for reporting. It also uses ArcView for maps, analyses, and generation of reports, 
which is an easy-to-use and flexible program. For example, ArcView can generate reports by 
region, district, county, and sub-county, and summaries can be exported to Word and Excel. 
The MEEPP system complies with the dynamic PEPFAR reporting requirements. Further, 
MEEPP upgrades its system to comply with relevant users such as USG agencies. 
The database and web server are housed at the MEEPP office in Uganda, with an off-site 
backup in the U.S. The system has a broadband connectivity that is guaranteed by two 
internet providers in case one fails. The power system also has a back-up in case of surges. 
The electronic database is accessed on www.meepp.or.ug. Prime partners have passwords to 
log-in, and they access and report directly on the web. Prime partners receive data in various 
MEEPP: End-of-Project Evaluation 
23 
formats from their sub-grantee IPs, though usually in Microsoft Excel. Comprehensive data is 
sent by IPs to prime partners as Excel attachments. The M&E staff of prime partners extracts 
relevant data for input into the MEEPP database. These data are uploaded into the database 
manually, which is time-consuming. A major issue is slow internet connections caused by 
small bandwidth and overloading by many people entering data at the same time. This 
depends on the physical location of the prime partners and the internet service provider. If the 
prime partner is located in an area with many internet users or if the service provider provides 
a small bandwidth, then the internet connections are very slow. Hence, some prime partners 
work at night and/or go to the MEEPP offices to enter the data. 
Inter-operability with national systems and other USG M&E systems 
With regard to inter-operability of data-based systems, the MEEPP system is adaptable to 
GOU and other stakeholder needs since many of the indicators are similar or relate to 
national indicators. However, the HMIS is not compatible with MEEPP. HMIS is Microsoft 
Access-based, while the MEEPP database is web-based. Further, the UAC currently does not 
have a functional database; a web-enabled database was under development through the 
USAID-funded AIDS Capacity Enhancement project, but the project ended before the 
database was completed. 
Respondents were asked about the extent to which they felt the M&E system developed by 
MEEPP was linked with other USG team systems, especially those of the Civil Society Fund 
and USAID/Uganda’s MEMS system. Respondents indicated that while MEEPP links with 
these systems in some way, efforts to draw boundaries or to link MEEPP with these two 
systems are not systematic and that these two systems are still evolving.
Civil Society Fund is one of the largest contributors of data on PEPFAR-funded activities. 
Civil Society Fund is also currently developing its own M&E system. However, many of the 
civil society organizations struggle to meet MEEPP standards, despite Civil Society Fund 
training on M&E. 
There are fragile sub grantees. They send in their reports late. There are also problems 
and inconsistencies in their data. We have to call almost every one of them to sort out 
these inconsistencies and issues related to following guidelines. 
–Prime partner staff person 
There also appears to be confusion among some IPs about the role of MEMS and its 
relationship to MEEPP. 
…there is no connection between MEMS and MEEPP projects. There has been a 
change and MEMS has been given a mandate to give assistance to PEPFAR partners 
in respect to developing PMPs. The Mission has decided that we should support the IPs 
to develop PMPs based on USAID standards. Some PEPFAR partners have PMPs but 
some simply have a list of indicators. The MEEPP database formats focus on the 
facility level. 
–USG staff person 
24 
 
However, some IP respondents identify MEMS with evaluation and analysis of service 
delivery data, with MEEPP focused on monitoring of service delivery. Others view MEMS as 
responsible for the M&E of USAID-funded organizations only, while MEEPP covers all 
organizations funded through PEPFAR. 
Ease of use and actual use 
Prime partners frequently reported that the MEEPP database was easy to use: 
…the web-based reporting system… makes reporting easy, fast, and accurate. 
Their system is user friendly and it cannot be manipulated… We now generate reliable 
data because of their guidance. 
In addition to MEEPP and prime partner staff, a number of other stakeholders have accounts 
with passwords to access to the database online, but with varying rights. The USG COTRs or 
Project Officers and some selected GOU officials have “read only” access. All users have 
access to aggregate data and downloads. 
MEEPP staff, however, complained that some people do not use the accounts. Instead, they 
telephone MEEPP and request data after stating that they have forgotten their password 
and/or username. In addition, some prime partners reported that their COTRs cannot access 
their specific data in the MEEPP database. 
…we can only discuss the narrative report with our focal person in CDC. But that 
person has no access rights to the report. As an IP, we report to CDC with an 
elaborate report every six months. Our report becomes part of the report for CDC on 
country programming. 
 –Prime partner staff person 
This view was also expressed by at least one COTR, who noted that: “The database needs to 
be more user-friendly for COTRs. I need to see whether my IPs are meeting their targets.” 
Thus, it appears that while MEEPP provides passwords to COTRs, their access is limited to 
aggregate data. As a result, prime partners separately report the same data to their COTRs. 
 
It was further expressed that procedures for obtaining deeper analysis of the data are 
sometimes cumbersome. 
There is limited use of MEEPP data because it is given to so few people. In case 
somebody is to help you with the work/delegation, you have to go through a long 
procedure: you need to talk to the Chief of Party, then you inform the MEEPP officials 
MEEPP: End-of-Project Evaluation 
25 
who have to give the delegated person permission to have a password to use the data. 
This process is really long and tiresome. 
 –Prime partner staff person 
MEEPP has also provided passwords to some key GOU personnel in MOH and UAC. 
However, these staff often need data analyzed by different age groups than those provided in 
the MEEPP database. They may also need data by specific districts, which MEEPP does not 
readily provide in its analysis. In these instances, they are encouraged to write to the Project 
Director of MEEPP to obtain that analysis, which some GOU respondents reported as 
prohibitive and time-consuming. 
MEEPP confirmed that there have been numerous requests for data and ad hoc analyses from 
a variety of consumers including GOU, USG, and independent researchers. However both 
MEEPP and senior USG respondents have stated that letting individuals carry out their own 
analyses would jam the system and might put the database at risk of both intentional and 
unintentional contamination. MEEPP staff indicated that when requests are received in an 
orderly fashion, preferably in written form, their response is usually instant, especially if the 
data are readily available. However, requests that require contacting sites for additional data 
may take longer. In addition, if the request came during the three-week period prior to the 
reporting deadline, a delay of up to two weeks is possible to meet such a request. 
Feasibility of technology transfer 
The evaluation team believes that, after some attention to the development of related skills in 
database management, DQA, analysis, and generation of reports, it is feasible for MEEPP to 
transfer its database to a prime partner and GOU agency. This process will be facilitated by 
the fact that MEEPP has good documentation of the database user instructions. The 
evaluation team recommends that MEEPP transfer the database and its related systems to a 
GOU institution once the project is terminated. However, respondents underscored that this 
should entail a phased process starting with merging of data collection tools. USG staff 
concurs, with one individual noting that: “The ideal situation is that the sites should use 
national tools and make copies for MOH and copies for MEEPP.” 
Another prerequisite for technology transfer is the need to harmonize indicators. Some effort 
has already been undertaken to harmonize several indicators between PEPFAR and HMIS, 
and so far seven indicators have been harmonized. USG respondents noted that while 
PEPFAR indicators are revised frequently, HMIS indicators are revised once in every five 
years. They also noted that both PEPFAR and HMIS have indicators that do not overlap and 
hence, in order to continue with harmonization, there is need to identify those indicators that 
are shared. A related issue is that different USG agencies require more data than are collected 
by the PEPFAR indicators. 
The harmonization of indicators should lead to the development of a single data collection 
tool for use by both HMIS and PEPFAR at service delivery sites. Indeed, a number of prime 
partners are already using the MOH data collation tools to extract data from the registers at 
the service delivery sites. These forms are completed and copies are sent to the prime partners 
26 
and to the District Health Office, and a copy is left with the sites. Other prime partners, 
however, have not completely harmonized their data collection tools with those of the MOH. 
Further, HMIS and OGAC reporting operate on different schedules (e.g., quarterly versus 
semi-annually and annually). Hence, service delivery sites use different formats to send 
reports to different stakeholders such as MOH and USG agencies. This results in additional 
work for the M&E staff as well as duplication of effort. 
3.5. What Has Been the Effects/Impact of MEEPP on the Reporting 
Capacity of PEPFAR-Funded IPs? 
Prime partners, including those with good M&E systems before MEEPP, unanimously 
reported that MEEPP has helped them organize their data systems through strict deadlines, 
feedback, and data quality assurance procedures. MEEPP has been of particular value to 
those organizations expanding rapidly under PEPFAR, with concomitant demands on their 
M&E departments. 
Before PEPFAR we were only in six sites. But with PEPFAR support we are now in 52 
sites. We had an M&E system but it was not up to the standards that USAID wanted so 
we had to improve on it. 
–Prime partner staff person 
Web-based reporting has helped to demystify IT to some staff of prime partners who were 
reluctant to fully apply it. MEEPP is also credited more generally with creating positive 
attitudes towards M&E among the staff of some prime partners. 
MEEPP has made M&E enjoyable. Some M&E components, e.g. bio aspects, were 
very difficult for us social scientists. But now we can use the M&E tools very well. 
–Prime partner staff person 
While MEEPP’s data validation meetings were generally viewed as a useful capacity building 
tool, some prime partners expressed a desire for more focused support: 
MEEPP has been inviting all the partners at the same time. You find that the place is 
crowded, and I wonder whether all the partners benefit at the same rate. Therefore, a 
fewer manageable number of partners should meet at a particular time for better 
results. 
 –Prime partner staff person 
It is beyond the mandate of MEEPP to ensure that the IPs have the capacity to report high 
quality and timely data to prime partners. Instead, MEEPP provides support to prime 
partners, which in turn support the IPs. Some prime partners have reported that this “trickle 
down” approach has been successful: 
 
The monitoring officers pass on this knowledge that they have acquired as they do 
support supervision to the implementing sites. We carry out trainings (especially to 
MEEPP: End-of-Project Evaluation 
27 
data officers) when a need is identified. In case there is a feedback from MEEPP, the 
message is passed on to the different region who pass on to the staff concerned. 
–Prime partner staff person 
One prime partner reported that MEEPP even provides training to its IP staff upon request, 
and that whenever it hires new staff, it calls upon MEEPP to train them in M&E. 
However, other prime partners maintained that the capacity strengthening provided by 
MEEPP stops at the level of the prime partners and does not sufficiently trickle down to the 
IPs: 
I have not gotten any training personally. There has been nothing in terms of systems 
development. The M&E officials at the head office have not also helped us much. 
 –IP staff person 
Respondents from MEEPP and USG emphasized that it is the role of prime partners to have a 
strong M&E system within their sub-grantees to ensure high quality and timely data. When 
asked about this issue, most prime partner respondents affirmed that they have budgets and 
work plans for M&E. However, in some cases the budgets were insufficient or the M&E 
department was integrated in service delivery departments, and thus received limited 
attention. 
Further, a respondent from MOH expressed concern about the quality of data on PEPFAR￾funded activities at MOH sites where there are no staff trained in M&E, and the fact that 
some local government sites, that have grants from PEPFAR, do not report regularly. 
3.6. What Has Been the Effects/Impact of the MEEPP Project on the 
National M&E Capacity? (This Result Was Evaluated Based on the 
Extent to Which the Project Was Designed to Support the National 
System) 
The USG requires MEEPP to link with the M&E efforts of GOU, but has not provided for a 
structured relationship between MEEPP and GOU through which this link could be realized. 
In its first quarter of 2005 report, which was also the Inception Report, MEEPP reiterates 
that: “Capacity building is an important focus of the MEEPP project, and it is targeted to 
contribute to strengthening knowledge and systems for a sound, sustainable national 
HIV/AIDS M&E system.” In the same document, MEEPP indicates that its role is: 
“…facilitating harmonization and aggregation of data, coordination with the Government of 
Uganda, and avoiding duplication of effort.”According to MEEPP and USG respondents, 
MEEPP has to some extent provided support to GOU M&E staff through ad hoc arrangement 
in spite of the fact that capacity building of GOU is not a contractual requirement of MEEPP. 
MEEPP does not have a contract with MOH to build capacity of government HMIS. In 
fact they have gone out of their way to help facility reconciliation. MEEPP is a 
contractor and serves employer’s interests. It has done a good job of having a mirror 
28 
system in facility data that MEEPP shows MOH and helps MOH to see problems with 
data. This is not a parallel system but a mirror system. 
–USG staff person 
For example, in its Inception Report, MEEPP recognized that host country institutions 
required capacity building, specifically in the area of mapping of ART sites. Indeed, MEEPP, 
working in collaboration with GOU, did carry out this mapping in early 2005. In addition, 
MEEPP provided GIS training to MOH staff in 2005 and technical assistance to UAC’s 
database assessment exercise, which was part of the National Strategic Plan development 
activities, in 2006. Respondents from MOH, UAC, and Ministry of Education and Sports 
reported that they appreciated MEEPP’s efforts to facilitate reporting of quality data and 
involve staff from GOU. Indeed, based on several data validation visits, one MOH 
respondent reported that “MEEPP produced uncontestable data.” 
Nevertheless, many USG and non-USG respondents agree the MEEPP design paid 
insufficient consideration to how best to build M&E capacity in GOU structures. Moreover, 
GOU had its own expectation that MEEPP would provide capacity building to its M&E 
functions. UAC expressed this view as early as 2005, during a SI meeting convened by USG. 
There is also a view that MEEPP has affected the functionality of the M&E capacity of GOU 
with regard to HIV/AIDS service delivery statistics. GOU respondents, especially those from 
MOH/AIDS Control Program and UAC, have strongly argued that MEEPP should have been 
embedded in a government institution to foster synergy between MEEPP and MOH and to 
ensure good M&E data collection and reporting. This synergy, however, is currently minimal 
and ad hoc. USG staff report that GOU is not carrying out its mandate sufficiently: 
MOH has failed to coordinate the HIV data. Yet MEEPP is functional and seen to have 
taken away the responsibility of the MOH concerning HIV data. As a result, MOH 
blames MEEPP for not working with it somehow. 
3.7. In Summary, What are the Lessons that Can Be Learned, and 
What Aspects of the Project May Need Change in Order to Address 
Gaps Identified in the Current Project, Namely National M&E 
System’s Development and Capacity Building for Host-Country 
M&E and Other Specialists? 
There is no clear strategy for engagement of public sector stakeholders. 
A good M&E system should generate data, promote it, and make sure it is accessible to 
policy makers and program managers to utilize in their decisions. Yet MEEPP was not 
designed with a systematic plan to engage agencies and institutions that supervise service 
delivery—such as the District Health Office, MOH, and UAC—and to ensure that they are 
able to access and use data on PEPFAR-funded activities. In particular, MEEPP does not 
interact with GOU at the district level, and there is no system for ensuring that PEPFAR data 
reaches decision-makers at this level. Indeed, during a debriefing with the USG team it was 
MEEPP: End-of-Project Evaluation 
29 
stated that: “Ideally each IP is required to send reports to the districts.” However, only some 
prime partners share data with the District Health Office. 
Key potential users of data on PEPFAR-funded activities are not accessing the data 
they need from the database. 
These users include GOU staff at the central and district levels, some staff of IPs, and some 
USG COTRs. The COTRs’ concern is that the MEEPP database provides aggregate data, 
which does not single out the individual IPs that the COTRs are responsible for monitoring. 
Hence, they request separate reports from the IPs, which may require unnecessary efforts. 
Data from non-PEPFAR-supported civil society organizations not working at health 
facilities tends to be excluded from PEPFAR reports. 
At present, non-PEPFAR-supported CSOs are reporting to neither MOH nor USG. While 
these civil society organizations are not the responsibility of MEEPP, such data would 
provide both GOU and USG with a more comprehensive context in which PEPFAR 
recipients are operating. 
MEEPP puts little emphasis on evaluation. 
MEEPP focuses on trend analyses of service delivery statistics as well as geographical 
coverage. As a result, PEPFAR performance in Uganda is being judged solely in terms of 
these outputs. Evaluative studies are required to assess quality, efficiency, client satisfaction 
or impact. 
The data that MEEPP reports satisfies PEPFAR because that is what MEEPP was 
contracted to do. But this data is not very useful to us. It is just basic data. [We] need 
more qualitative data than what MEEPP provides. 
 –USG staff person 
We need a formal evaluation for the program focusing mainly on quality, not numbers. 
We can tell the districts that we are doing well. But what about the general 
performance? 
–Prime partner staff person 
During a debriefing meeting with the evaluation team, USAID staff agreed that there is a gap 
with regard to extra data analysis and evaluation, but they were quick to stress that MEEPP 
should not be misunderstood to be a research organization that analyzes and interprets data. 
Prime partners confirm this limited role: 
30 
The evaluation function is so limited and I think it is periodic…But I think it is not their 
mandate to do the evaluation. It is MEMS function, not MEEPP’s function. MEEPP is 
basically to provide information/ data for these evaluation studies. 
MEEPP has, nevertheless, carried out a number of special studies. According to its quarterly 
reports, these include: mapping the service delivery points of ART; validating indirect USG 
support in CT, PMTCT, and OVC; a needs assessment for laboratory data management in 12 
districts; and several studies examining tracking of and accounting for clients. Thus, it 
appears that MEEPP has limited itself to special studies that support its mandate of providing 
accurate monitoring rather than evaluating outcomes. 
MEEPP: End-of-Project Evaluation 
31 
4. Conclusions 
This section highlights key conclusions of this evaluation following the structure of the 
evaluation questions. 
Was the MEEPP Design Able to Address the Reporting Needs of 
PEPFAR and GOU? 
The MEEPP design was able to address the reporting needs of PEPFAR, specifically, to 
collect data for reports to OGAC. However, capacity building of GOU’s key M&E 
institutions was not included in the MEEPP design. In retrospect, GOU would have preferred 
MEEPP to be institutionalized within a GOU agency. In addition, GOU had high but 
unrealized expectations of synergy between MEEPP & HMIS. 
Was MEEPP Implemented as Designed? 
MEEPP was implemented as designed in terms of availability of timely output data. 
Reporting deadlines are met, facilitated by web-based access to the database by prime 
partners. Prime partners can easily enter their data, and MEEPP provides excellent DQA 
during and after the data entry period. Reports are current, readily available, and accessible to 
staff of GOU, USG, and IPs on the MEEPP website. However, these customized reports do 
not provide sufficient details for some stakeholders to use for decision making and planning. 
MEEPP provides further analyses of data upon request, but the procedure for requesting these 
special analyses is considered cumbersome. 
What Has Been the Trend of Quality of Data Reported to OGAC? 
MEEPP has met the expectations of the Country Team and the IPs with regard to quality of 
data. Data on PEPFAR-funded activities are considered a credible, alternative source of 
information for government institutions and other agencies. However, despite MEEPP’s 
efforts, double-reporting remains an issue. While not the responsibility of MEEPP, 
inadequate attention to capacity building of staff at the primary data sources and sudden 
revisions in PEPFAR indicators also compromise data quality. 
What Is the Status of Electronic Databases for PEPFAR Data? 
It is easy to upload data and access reports in the MEEPP database. However, during peak 
periods, data entry can be slowed by poor internet connections and power outages. It is 
feasible to hand over the MEEPP database to an appropriate institution once the project has 
ended. This handover requires transfer of skills through training and a phased approach that 
32 
should start with the harmonization of indicators with HMIS and the use of a single data 
collation tool at service delivery sites. 
What Has Been the Impact on the M&E Capacity of PEPFAR-funded 
IPs? 
MEEPP supports one level of PEPFAR IPs: the prime partners, who in turn are expected to 
support the sub-grantee IPs. The unanimous view of prime partners was that MEEPP has 
helped them to organize their data systems through strict deadlines, DQA, and feedback 
meetings on quality of data and coverage. However, this support does not always reach the 
IPs, largely because the M&E department of some prime partners does not have a clear 
budget and work plan for capacity building of IPs, or because the budget is embedded in the 
service delivery department, where M&E is not always given priority. 
 
What Has Been the Impact on the National M&E Capacity? 
MEEPP does not have an explicit mandate to build the capacity of GOU agencies in M&E. 
Despite USG procurement of other means of supporting GOU HIV and health M&E, there 
still appears to be a gap in this capacity. MEEPP is reported to have provided some capacity 
building in M&E to GOU and has involved MOH staff in DQA activities. 
In Summary, What Are the Lessons that Can Be Learned and What 
Aspects of the Project May Need Change? 
There is no clear strategy for engaging public sector stakeholders. 
This evaluation suggests that MEEPP’s mandate is well understood by stakeholders and that 
it has largely achieved its goal of providing high quality and timely data on PEPFAR-funded 
activities to the Country Team. However, respondents expressed concerns that the MEEPP 
design inadequately considered how MEEPP would promote and ensure access of this data to 
key decision makers in Uganda. Despite ad hoc efforts by MEEPP and other USG activities 
to support GOU M&E capacity, this gap requires further attention in the planning of any 
follow-on projects. In particular, MEEPP’s performance monitoring indicators should include 
the extent to which its data are used for host country decision making. 
Data from non-PEPFAR-supported civil society organizations working outside 
health facilities tend to be excluded. 
While it is not the responsibility of MEEPP to collect data from non-PEPFAR-supported 
service providers, the lack of this strategic information represents a gap in the understanding 
of PEPFAR about current coverage of services in Uganda. 
MEEPP: End-of-Project Evaluation 
33 
5. Recommendations 
This end-of-project evaluation has shown that MEEPP is largely a successful project that has 
met its contractual obligation. Perhaps the greatest challenge MEEPP has experienced is 
weak GOU M&E systems both at national and district levels, as MEEPP was intended to be 
integrated into these M&E systems, which are supposed to coordinate all M&E efforts 
including those of USG agencies. This section of the report proposes recommendations that 
USG could consider in a follow-on project to address these GOU needs. The 
recommendations, which result from extensive discussions with MEEPP, are offered in the 
context of existing USG efforts to support HMIS and UAC. However, it should be noted that, 
for these efforts to yield tangible outcomes, the MOH must provide leadership by 
encouraging district authorities to coordinate the collection of data from their health facilities 
and community-based services. 
In Designing a Follow-on Project, USAID Should Aim to Move From 
An “Emergency Mode” and Align It With the National Systems of 
Data Collection and Reporting. 
Specifically, the follow-on project should: 
1. Contribute to the current USG effort to develop and support GOU databases: In the 
transition phase, the follow-on project should build technical capacity of UAC and 
Resource Center to establish and operate web-based databases. This capacity building 
should be phased in with at least one year of direct support followed by targeted or 
selective support in subsequent years. 
2. Begin to hand over operation of the MEEPP database: After appropriate training, it is 
feasible to transfer the MEEPP database to a prime partner or GOU. It is important, 
however, that this process starts soon with ensuring that only one primary data 
collation tool is used at the sites by MEEPP, HMIS and other agencies. 
3. Contribute to the current USG support for the establishment of a national HIV and 
AIDS database that incorporates all HIV and AIDS data in Uganda: This database 
should be managed by UAC with capacity building from MEEPP during a transition 
phase. UAC is the appropriate location because it has the national mandate to 
coordinate and monitor all HIV and AIDS activities in the country. This process has 
already begun at UAC with USG funding through the just-ended ACE project and 
Population Council. 
4. Support the use of existing national tools and ensure that all PEPFAR-funded partners 
report through both the existing national systems and to USG: There is need for USG 
and other donors to facilitate completion of revision and standardization of all existing 
HIV/AIDS GOU data collection tools in different intervention areas. The recent 
process of revising tools addressed a number of needs and interests of stakeholders; 
however, some tools (especially the clinical management tool) are not yet 
standardized. Using these standardized tools, the data should then be collated in a 
34 
user-friendly format (e.g., Excel). In addition, the follow-on project should ensure that 
data from PEPFAR-supported sites are relayed in a timely manner to the district, 
UAC, and HMIS. Similarly, the project should promote the value of collecting, 
analyzing, and using data for district-level planning and decision making. 
USG has provided technical assistance to UAC to develop a strategy for rolling out its 
PMMP, which covers 58 national-level indicators and 47 district-level HIV/AIDS 
program indicators. According to the draft report of this technical assistance, the roll￾out will include both national- and district-level approaches. At the district, the UAC 
will train a team of key officials, lead by the HIV/AIDS Focal Point Person, to collate 
data from service delivery sites using a district-level tool, and this information will be 
passed on to UAC at the national level. The MEEPP follow-on project should link 
into this proposed approach by supporting and participating in the harmonization of 
indicators and the use of one tool, which the sites can use to report to USG and the 
District AIDS Focal Point Person concurrently. 
In the PMMP roll-out, it is also proposed that, at the national level, UAC plays a more 
effective leadership role in ensuring line ministries report their M&E data to UAC. 
The technical assistance report makes a number of recommendations including: 
increasing engagement with stakeholders, creating reporting schedules, providing 
clear reporting mechanisms, specifying who should provide information, following-up 
with sector persons, and developing a spreadsheet that automatically indicates which 
indicators are out of date. USAID should ensure that the MEEPP follow-on project 
supports UAC to develop a database that allows line ministries to report to UAC. 
5. Streamline and support M&E systems of prime partners to ensure that they can, in 
turn, support their sub-grantee IPs: For example, the follow-up project to MEEPP 
should support and monitor prime partners to ensure that they provide systematic 
M&E support to IPs at peripheral sites. This element could be included in the DQA 
process that MEEPP provides to prime partners. 
USAID should consult with MOH, UAC, and other host-country 
users of data on PEPFAR-funded activities data to devise ways to 
address their needs and ensure optimal utilization in policy making 
and program planning. 
Ideally, all requests for data and ad hoc analysis should be sent to UAC, and GOU users 
should receive quarterly reports from UAC. However, in the short-term or transition phase, 
this should be done by MEEPP while simultaneously providing systematic and targeted 
capacity building for UAC to assume this responsibility. In the medium- and long-term, 
MEEPP should build the capacity of UAC to respond directly to these requests for data and 
specific analysis. 
MEEPP: End-of-Project Evaluation 
35 
Recognizing that MEEPP has so far concentrated on the monitoring 
aspects of M&E, USG should work with GOU and other partners to 
strengthen the current outcome evaluation framework. 
In Uganda, HIV/AIDS evaluation relies on occasional national surveys and specific 
epidemiological studies carried out by various research groups. The report on technical 
assistance to UAC to roll-out the PMMP recommends that UAC, MOH, and partners develop 
a strategy for national level surveys. USG should provide support to GOU to ensure this 
strategy is realized. 
36 
Appendix 1: Documents Reviewed 
Makumbi, Frederick, et al. 2009 (In Print). “Technical assistance to the Uganda AIDS 
Commission for operationalisation of the performance monitoring and management plan.” 
New York: Population Council. 
Monitoring and Evaluation of Emergency Plan Progress (MEEPP) Project. 2010. Description 
of MEEPP on web site of Social & Scientific Systems, Inc. Accessed January 26, 2010. 
www.meepp.or.ug 
———. 2009. “Monitoring and Evaluation of Emergency Plan Progress (MEEPP) – 
Uganda, December 2004–December 2009.” PowerPoint presentation, October 2009. 
———. 2009. “Summary of MEEPP from December 2004 to 2009.” 
———. 2009. MEEPP Quarterly Reports for 2005 to 2009 (second quarter). Unpublished. 
Accessed September 10, 2009. www.meepp.or.ug 
———. 2009. PEPFAR Semi-Annual Reports (SAR) for the years 2006 to 2008 for the 
following USG agencies: Department of Defense, Peace Corps, Department of State, U.S. 
Agency for International Development, and Centers for Disease Control, on the MEEPP web 
site. Unpublished. Accessed September 10, 2009. www.meepp.or.ug 
———. 2009. PEPFAR Annual Program Reports (APR) for the years 2006 to 2008 for the 
following USG agencies: Department of Defense, Peace Corps, Department of State, U.S. 
Agency for International Development, and Centers for Disease Control, on the MEEPP web 
site. Unpublished. Accessed September 10, 2009. www.meepp.or.ug 
———. 2009. Semi-Annual Program Results and Explanatory Notes for the period 2005 to 
2008 on the MEEPP web site. Unpublished. Accessed September 10, 2009. 
www.meepp.or.ug 
———. 2009. Annual Program Results and Explanatory Notes for the period 2005 to 2008 
on the MEEPP website. Unpublished. Accessed September 10, 2009. www.meepp.or.ug 
———. 2008. “Report of NUMAT Data Quality Assessment and Data Validation Report for 
Data Submitted in the EP FY07 Annual Report.” Unpublished. 
U.S. President's Emergency Plan for AIDS Relief (PEPFAR). 2009. PEPFAR Overview on 
web site. http://www.pepfar.gov 
———. 2008. Uganda Fiscal Year 2008 Country PEPFAR Operational Plan (COP). 
http://www.pepfar.gov 
MEEPP: End-of-Project Evaluation 
37 
———. 2007 (Unpublished). The President’s Emergency Plan for AIDS Relief: Indicators, 
Reporting Requirements, and Guidelines Indicators Reference Guide, FY2007 
Reporting/FY2008 Planning. 
Uganda Program for Human and Holistic Development (UPHOLD). 2007. “Quarterly 
Report, January to March 2007.” http://uphold.jsi.com 
38 
Appendix 2: Questionnaires 
Master Guide for Key Informant Interviews and In-depth Interviews 
1. How would you describe MEEPP as an organization? 
What was MEEPP set up to do? 
How does MEEPP work? 
How would you describe the MEEPP design? 
To what extent has the MEEPP design incorporated the use of action-oriented research 
and special studies to supplement routine service delivery statistics in informing 
decision makers? 
Who are MEEPP’s clients? Who does MEEPP serve? Who benefits from the services 
of MEEPP? 
2. How has MEEPP enhanced the availability of HIV/AIDS M&E data? 
How well does MEEPP present results from PEPFAR-supported activities in usable 
formats, e.g., mapping that can inform the PEPFAR Country Team’s programming 
decisions? 
How well does MEEPP meet PEPFAR reporting deadlines for semiannual and annual 
reports? 
What is the extent to which MEEPP reports to PEPFAR are accessible to GOU, IPs, 
and non-USG development partners (e.g., whether sent automatically or only on 
demand)? 
What is the extent to which data from PEPFAR-funded activities has informed 
programming at national at the level of PEPFAR Country Team and Technical 
Working Groups (TWGs)? 
3. What has been the effects/impact of the MEEPP project on the M&E capacity of 
PEPFAR/Uganda and its IPs? 
What is the evidence regarding strengthened monitoring and reporting systems of 
PEPFAR/Uganda and its IPs? 
What role has MEEPP played in enhancing the M&E capacity of PEPFAR/Uganda and 
its IPs? 
To what extent are IPs able to interpret their M&E data and extract implications for 
programs and policy? 
How has MEEPP contributed to enhancing the capacity of IPs to analyze and make use 
of M&E data? 
How has MEEPP interacted with the other M&E systems of the USG such as CDC, 
Infotronics, MEMS, and others? 
4. What were MEEPP’s expectations with regard to strengthening of local M&E 
capacity? 
How has MEEPP interacted with the national M&E system? 
What has been the effects/impact of the MEEPP project on the national /local M&E 
capacity? 
What is the spillover effect of the MEEPP system in enhancing the capacity of the 
MEEPP: End-of-Project Evaluation 
39 
national M&E system to capture data from multiple sources at the national and district 
level on a regular, formal basis while limiting dual reporting? 
How well does the system developed by MEEPP support and align with the national 
reporting requirements, e.g., PMMP of UAC, HMIS of MOH, OVC activities to 
MoGLSD? 
How else has MEEPP contributed to the expansion and enhancement of the local M&E 
capacity? 
What has been learned and what can be improved in regard to MEEPP’s interaction 
with the national M&E system’s development and capacity building for host-country 
M&E and other specialists? 
5. How well is MEEPP meeting the objective of collaboration, coordination, and 
harmonization? 
How has MEEPP contributed to improved coordination and collaboration among USG 
agencies and IPs? 
How has MEEPP contributed to harmonization of data sets across partners and beyond 
USG? 
To what extent has MEEPP improved coherence of USG’s EP portfolio, e.g., training 
of IPs on common data/indicator definitions in order to ensure data comparability, 
collation, and adjustments for double counting? 
 
6. What is the level of satisfaction of MEEPP’s clients with the services provided by 
MEEPP? 
What are the views of the PEPFAR IPs on the benefits of the MEEPP system with 
regard to the programming cycle of their services including planning, implementation, 
and evaluation? 
What are the views of the staff of USG agencies regarding the benefits of the system 
developed by MEEPP in terms of planning, coordination, implementation, monitoring, 
and harmonization of the USG support to the Ugandan response to the epidemic? 
What are the views of GOU agencies on how MEEPP contributed to national planning, 
coordination, implementation, and monitoring of the HIV/AIDS response, as well as 
the extent to which MEEPP assists the UAC to collate data on the national response to 
HIV/AIDS? 
How successful was MEEPP in communicating with constituencies including USG, 
GOU, and civil society organizations both in extracting information and in 
disseminating findings? 
Has the system developed by MEEPP motivated or de-motivated service providers to 
routinely report service delivery statistics? 
What is the client feedback regarding communication style, satisfaction, and capacity 
building? 
To what extent did MEEPP involve the key stakeholders when developing this system? 
Is there more appreciation for data demand and information use or is the focus on 
reporting numbers reached, regardless of the quality of services offered? And what has 
been the role of MEEPP in this aspect? 
7. Overall would you say MEEPP is meeting its objectives? 
Overall, was the MEEPP project implemented as designed? 
40 
Overall, was the MEEPP design able to address the M&E needs of PEPFAR and GOU? 
8. What external factors may have affected MEEPP’s success in attaining the 
results? 
Physical factors such as internet connectivity and power outages 
PEPFAR requirements and pressures on implementing partners 
GoU M&E systems and their ability to co relate with MEEPP’s systems 
9. In general, what lessons have been learned from the MEEPP experience? 
What worked well? 
What didn’t work well? 
What needs to be scaled up in the new project? 
What should be new additions in the new project? 
What challenges need to be addressed by the new project and how? 
What is the added value of an activity similar to MEEPP to HIV programming and 
policy development in Uganda? 
What factors influenced the usefulness of the information from the system developed 
by MEEPP? For example: the nature of information collected; burden of data 
collection; sources of the information; timeliness of reports; packaging and level of 
disaggregation of the information reported? 
How could the value of the information be enhanced? 
Overall, what are the best practices demonstrated by the MEEPP project? 
Overall, what gaps can be indentified in the current project? And what aspects of the 
project may need change in order to address these gaps? 
MEEPP: End-of-Project Evaluation 
41 
Checklist for Observation of Data Management System 
1. How has MEEPP enhanced the availability of HIV/AIDS M&E data? 
What is the status of electronic databases with regard to: comprehensiveness, inter￾operability with national systems, ease of use, actual use, and plans /feasibility for 
handing over to appropriate institution once project is ended? 
How well does MEEPP present results from PEPFAR-supported activities in usable 
formats, e.g., mapping that can inform the PEPFAR Country Team’s programming 
decisions? 
How well does MEEPP meet PEPFAR reporting deadlines for semiannual and annual 
reports? 
What is the extent to which MEEPP reports to PEPFAR are accessible to GOU, IPs, 
and non-USG development partners (e.g., whether sent automatically or only on 
demand)? 
What is the extent to which data from PEPFAR-funded activities has informed 
programming at national and PEPFAR Country Team levels? 
Review of the MEEPP database for: 
• ease of inputting data from and by IPs 
• ease of use for key analyses and generation of reports 
• flexibility to generate reports by geographical regions, type of service, type of 
IP, period of time, etc. 
• accessibility of data sets for use by GOU agencies 
• adaptability of the MEEPP Partner Reporting System by the GoU and other 
stakeholders 
• potential for sustainability of the data base beyond USG funding and the 
existence of MEEPP 
2. Over the last five years, has there been an improvement in the quality of data 
reported by IPs to the Office of Global AIDS Coordinator? 
How does the data meet the set technical standard of PEPFAR reporting? 
What has been the trend of quality of data reported to OGAC over the last five years? 
Over the past five years, to what extent were the reports timely, met the set technical 
standards, and complete per PEPFAR indicators? 
What measures were taken by MEEPP to avoid double-reporting across all program 
areas and how successful were they? 
What is the balance of emphasis between data on treatment and prevention? 
What is the balance of emphasis between facility-based versus community-based data? 
What is the balance in focus between monitoring vs. evaluation? 
3. What has been the effects/impact of the MEEPP project on the M&E capacity of 
PEPFAR/Uganda and its IPs? 
What is the status of electronic databases with regard to: inter-operability with other 
USG systems, ease of use, actual use, and plans for handing over to an appropriate 
institution once project is ended?
42 
4. What were MEEPP’s expectations with regard to strengthening of local M&E 
capacity? 
What is the status of electronic databases with regard to: inter-operability with national 
systems, ease of use, actual use, and feasibility for handing over to an appropriate 
institution once project is ended? 
5. How well is MEEPP meeting the objective of collaboration, coordination, and 
harmonization? 
To what extent is the MEEPP system able to collate data from the different USG team 
M&E systems, e.g., Civil Society Fund and USAID/Uganda’s Monitoring and 
Evaluation Management Services (MEMS)? 
How accessible is the MEEPP database for use by USG agencies? 
MEEPP: End-of-Project Evaluation 
43 
Appendix 3: Prime Partners Reporting for Annual 
Program Report 2007 
USAID-funded Prime Partners 
1. Aid Conservation through Education (ACE) 
2. Access, Quality, and Use in Reproductive Health (ACQUIRE) 
3. AFFORD 
4. AFRICARE 
5. Association of Volunteers in International Service (AVSI) 
6. Basic Education and Policy Support–Presidential Initiative on AIDS Strategy for 
Communicating to Young People (BEPS–PIASCY) 
7. Canadian International Development Agency (CIDA) 
8. CAPACITY Project 
9. Children AIDS Fund 
10. Christian AID 
11. Communities Responding to the HIV/AIDS Epidemic (CORE) 
12. Community Resilience and Dialogue (CRD) 
13. Conflict Districts 
14. Catholic Relief Service (CRS) -TRACK I 
15. DELIVER 
16. Elizabeth Glaser Pediatric AIDS Foundation (EGPAF) 
17. Family Health International (FHI) -SAFE STOP 
18. Hope for African Children Initiative (HACI) 
19. Health Communication Partnership (HCP) 
20. HIV –READERS 
21. Hospice Uganda 
22. International Youth Foundation 
23. Inter-Religious Council of Uganda (IRCU) 
24. Joint Clinical Research Center (JCRC) 
25. John Snow International (JSI) 
26. Legislative Support Activity (LSA) 
27. Monitoring and Evaluation of the Emergency Plan Progress (MEEPP) 
28. Opportunity International–Uganda Agency for Development Limited 
29. ORC MACRO 
30. PART BUSINESS 
31. Program for Appropriate Technology in Health (PATH) 
32. People Living with HIV/AIDS (PHA) Network 
33. PLAN International 
34. POLICY II 
35. Quality Assurance Project- University Research Co (QAP –URC) 
36. Salvation Army 
37. Samaritan’s Purse 
38. Program for Human and Holistic Development (UPHOLD) 
39. World Vision 
44 
CDC-funded Prime Partners 
1. AIDS Information Center 
2. AIDS Relief 
3. African Medical And Research Foundation (AMREF) 
4. Baylor College of Medicine–Pediatric Infectious Diseases Clinic (PIDC) 
5. Integrated Community Based Initiatives (ICOBI) 
6. Jhpiego 
7. Kalangala Local Government 
8. Kumi Local Government 
9. Mildmay International 
10. Ministry of Health 
11. Medical Research Council (MRC) 
12. Makerere University, Kampala School of Public Health 
13. Faculty of Medicine—Mulago Mbarara Joint AIDS Program (MJAP) 
14. PACE 
15. Protecting Families Against HIV/AIDS (PREFA) 
16. Research Triangle Institute (RTI) 
17. Strengthening HIV Counseling (SCOT) 
18. The AIDS Support Organization (TASO) 
19. Uganda Blood Transfusion Services 
20. Uganda Virus Research Institute (UVRI) 
21. Uganda Prisons Services 
22. University of California, San Francisco 
Prime Partners Funded by Department of Defense and Department 
of State 
1. Uganda Peoples Defence Force (UPDF) 
2. Makerere University Walter Reed Program (MUWRP) 
3. International Medical Corps (IMC) 
4. International Rescue Committee (IRC) 
5. Small Grants 
45
Appendix 4: Matrix for Selecting Prime Partners and Implementing Partners 
Sample 
PROGRAM 
AREAS 
AFFORD/ 
JHU/CCP 
(USAID) 
AFRICARE 
(USAID) 
AIDS 
Info 
Centre 
(CDC) 
AIDS Relief 
/CRS 
Consort 
(CDC) 
AMREF 
(CDC) 
Conflict 
Districts/ 
NUMAT 
(USAID) 
EGPAF 
(USAID) 
IRCU (Includes 
IRCU follow on 
targets) -
(USAID) 
Joint Clinical 
Research 
Center/ TREAT 
Follow on/TBD 
(USAID) 
MJAP 
(CDC) 
ART Fort Portal Kampala 
PMTCT Kampala Mulago 
CT Mbale Mbarara 
Lab Jinja Gulu 
Abstinence 
& Being 
Faithful 
Luwero 
Other 
Prevention 
Kampala 
OVC Ntungamo 
Palliative 
Care-TB 
Tororo Kampal
a 
Palliative 
Care-Basic 
46
PROGRAM 
AREAS 
PEACE 
CORPS 
PHA Network/ 
International 
HIV/AIDS 
Alliance -
(USAID) 
PIDC_Baylor 
College 
Uganda- 
(CDC) 
Plan 
International 
/HACI (USAID) 
PREFA / TORORO 
CHILD COHORT - 
(CDC) 
RTI - (CDC) UPDF-DEPARTMENT OF DEFENSE 
ART Gulu 
PMTCT Tororo 
CT 
Lab 
Abstinence & 
Being 
Faithful 
Mubende Bombo 
Other 
Prevention 
OVC Luwero 
Palliative 
Care-TB 
Palliative 
Care-Basic 
 Mukono Mpigi 
47
PROGRAM 
AREAS 
IRC 
(State) 
State 
Small 
Grant 
(State) 
TASO 
(CDC) 
UVRI 
(CDC) 
Walter 
Reed 
ART Mbarara Kayunga 
PMTCT 
CT Kampala 
Lab Gulu 
Abstinence & 
Being Faithful 
Other 
Prevention 
Masindi Kampala 
OVC 
Palliative Care￾TB Masaka 
Palliative Care￾Basic