1 
DELIVER PROJECT: NGO LOGISTICAL 
STRENGTHENING COMPONENT 
FINAL REPORT OF EXTERNAL PERFORMANCE EVALUATION 
September, 2016
Managua, Nicaragua. 
CLARIFICATION: The opinions expressed by the author of this publication do not necessarily 
reflect the opinions of the United States Agency for International Development nor the opinions 
of the government of the United States of America.
2 
Evaluator
Yelba Jarquín, MSc. 
Field and digital support team
Antonio Rivera, MSc. 
Technical support and supervision 
Marianela Corriols, USAID Nicaragua 
Marcela Villagra, USAID Nicaragua
Internal validation 
Oscar Núñez, USAID PrevenSida 
Carolina Arauz, USAID DELIVER 
Nora Quesada, USAID DELIVER 
External validation 
Directors and specialists from eight NGOs 
This document was presented by the outside consultant Yelba Jarquín to the United States Agency for 
International Development under cooperative accord No. AID-524-A-10-00003 (USAID PrevenSida, 
Strategic Information component), implemented by the University Research Corporation (URC). 
3 
ACKNOWLEDGMENTS
We particularly recognize the directors and members of the participating organizations in this process 
of reflection and analysis, the project management teams of USAID and the Nicaragua Mission of USAID, 
who offered the necessary information to sustain this report, and all the personnel who directly or 
indirectly have given their support to this evaluation. 
4 
CONTENTS 
EXECUTIVE SUMMARY............................................................................................... 9
I.INTRODUCTION...................................................................................................... 12
II. METHODOLOGY ................................................................................................... 16
III. RESULTS................................................................................................................. 19
IV. CONCLUSIONS .................................................................................................... 33
V. RECOMMENDATIONS.......................................................................................... 35
VI. APPENDICES ......................................................................................................... 38
APPENDIX # 1: Terms of reference.......................................................................... 38
APPENDIX # 2: Schedule of field visits ..................................................................... 43
APPENDIX # 3: Persons interviewed ........................................................................ 44
APPENDIX # 4: Checklist ........................................................................................... 47
APPENDIX # 5: Polling guide..................................................................................... 49
APPENDIX # 6: Instruments for the focus group .................................................... 53
APPENDIX # 7: Instrument for individual interview............................................... 53
APPENDIX # 8: Evidence........................................................................................... 54
APPENDIX # 9: General output table of the 27 indicators evaluated................... 55
5 
TABLES AND FIGURES
Table 1. Research questions and applied methodology .......................................................... 16
Table 2. Evaluated aspects of the logistical cycle ...................................................................... 17
Table 3. Achievement of training goals by the USAID | DELIVER PROJECT .................... 20
Table 4. Training and enabling NGO supervision ..................................................................... 20
Table 5. Comparison of the 2013 baseline and the 2016 evaluation ................................... 21
Table 6. Fulfillment of recommendations by the organization ............................................... 27
Table 7. Sustainability of actions according to NGO assessment ......................................... 30
Table 8. Persistent gaps in the NGOs in September 2016 ..................................................... 32
6 
ABBREVIATIONS
ABC Activity-based Costing 
ACCS Costal Campaign Association Against AIDS (Asociación Campaña Costeña Contra el 
Sida) 
ADESENI Nicaraguan Association for the Rights of Sexual Diversity (Asociación por los 
Derechos de la Diversidad Sexual Nicaragüense) 
AIDS Acquired Immune Deficiency Syndrome 
ANICP+VIDA Nicaraguan Association of Positive Persons Fighting for Life (Asociación 
Nicaragüense de Personas Positivas Luchando por la Vida) 
ART Antiretroviral Therapy 
ARV Anti-retrovirals 
ASONVIHSIDA Association of Persons with HIV/AIDS (Asociación de Personas con VIH Sida) 
ASSIST Applying Science to Strengthen and Improve Systems 
CANSALUD National Chamber of Health (Cámara Nacional de Salud) 
CEGODEM Center of Studies for Governability and Democracy (Centro de Estudios para la 
Gobernabilidad y Democracia) 
CEPRESI Center for AIDS Prevention (Centro de Prevención del Sida) 
CIPS Health Supply Center (Centro de Insumos Para la Salud) 
CPV Voluntary Counseling and Testing (Consejería y Prueba Voluntaria) 
CURIM Committee for the Rational Use of Medical Supplies (Comité de Uso Racional de 
Insumos Médicos) 
DAIA Assured Availability of Contraceptive Supplies (Disponibilidad Asegurada de 
Insumos Anticonceptivos) 
DAISSR Assured Availability of Reproductive Sexual Health Supplies (Disponibilidad 
Asegurada de Insumos en Salud Sexual Reproductiva) 
DELIVER Project of Technical Assistance in the Logistics of Medical Supplies (Proyecto de 
Asistencia Técnica en Logística de Insumos Médicos) 
EMP Social Security Medical Companies (Empresas médicas Previsionales) 
EMC Quality Improvement Teams (Equipos de Mejora de la Calidad) 
7 
FP Family Planning 
FSW Female Sex Worker 
FY Fiscal Year 
GAO Western Self-help Group (Grupo de Autoayuda de Occidente) 
GBV Gender-based Violence 
GF The Global Fund 
HCI Health Care Improvement Project 
HIV Human Immunodeficiency Virus 
HR Human Rights 
INSS Nicaraguan Social Security Institute (Instituto Nicaragüense de Seguro Social) 
KP Key Population 
LGBTI Lesbian, Gay, Bisexual, Transgender and Intersex 
M&E Monitoring and Evaluation 
MCC Continuous Quality Improvement (Mejoramiento Continuo de la Calidad) 
MINSA Nicaraguan Ministry of Health (Ministerio de Salud de Nicaragua) 
MSM Men who have Sex with Men 
NGO Non-governmental Organization 
OVI Whole Life Organization (Organización Vida Integral) 
PASIGLIM Automated Information Systems Program for Logistical Management of Medical 
Supplies (Programa Automatizado del Sistema de Información para la Gestión 
Logística de Insumos Médicos).
PEN Strategic National Plan (Plan Estratégico Nacional) 
PEPE First to Expire, First to Deliver (Primero en Expirar, Primero en Entregar) 
PEPFAR United States President’s Emergency Plan for AIDS Relief (Plan de Emergencia del 
Presidente de los Estados Unidos para el Alivio del Sida) 
PHIV Persons with HIV 
8 
PREVENSIDA AIDS-prevention Project in Key Populations in Nicaragua (Proyecto de prevención 
del Sida en poblaciones clave en Nicaragua) 
RACCN Autonomic Region of the North Caribbean Coast (Región Autónoma Costa Caribe 
Norte) 
RACCS Autonomic Region of the South Caribbean Coast (Región Autónoma Costa Caribe 
Sur) 
SCMS Supply Chain Management System 
SIGLIM Information System for Logistical Management of Medical Supplies (Sistema de 
Información para la Gestión Logística de Insumos Médicos) 
SILAIS Local Holistic Health Systems (Sistemas Locales de Atención Integral a la Salud) 
STI Sexually Transmitted Infection 
SW Sex Worker 
TDR Terms of Reference (Términos de Referencia) 
TRANS Transgender 
URC University Research Co., LLC 
USA United States of America 
USAID United States Agency for International Development 
WHO World Health Organization 
9 
EXECUTIVE SUMMARY 
The objective of this consultancy was to evaluate the performance of the logistical strengthening 
component of the DELIVER projects in non-governmental organizations (NGOs) that worked with 
key populations with HIV in Nicaragua from 2013 to 2016. 
The external evaluation was conducted by a national consultant with previous experience in 
performance evaluations of the Bilateral HIV Program of USAID Nicaragua (2013) and of the capability 
component of the same program (2015). 
The research questions were as follows: 
 Compared with the baseline, which aspects were strengthened in the field of storage and 
control of inventory in the organizations that serve key populations and that were 
beneficiaries of the project? 
 What were the key actions in the process of transfer abilities in managing the logistical chain, 
applying tools for management strengthening and measuring key indicators? 
 To what extent were the recommendations from the baseline diagnostic fulfilled? 
 How was gender focus applied in the intervention design? 
 What actions were sustainable once the intervention was completed? 
 What gaps remained once the intervention was completed? 
The evaluation was developed over four weeks, from September 1 to September 30, 2016. The 
methodology included coordination meetings with the USAID PrevenSida and Mission teams, 
reviewing relevant documents, adjusting the evaluation instrument used in the baseline, conducting 11 
interviews at the administrative management level of the organizations and developing eight focus 
groups with the studied NGOs in the departments of Managua, León, Nueva Segovia, Río San Juan and 
the Autonomic Region of the South Caribbean Coast. The data were recorded and analyzed in Excel, 
using the normative method for evaluating the performance of a project. 
Personnel from the USAID | DELIVER PROJECT provided technical assistance and training for the 
NGOs funded by USAID PrevenSida, conducting an initial diagnostic of the logistical capacities of the 
NGOs in 2013. The diagnostic included management of HIV supplies (inventory control, estimation of 
necessities, purchase, storage) and identified the baseline of the logistics indicators, priorities for 
technical assistance and recommendations. 
The baseline findings reflected a lack of human resources trained in supply logistics, an absence of 
documents and/or forms to register the flow of supplies, an absence of internal control mechanisms to 
ensure inventory, insufficient space, and inadequate environments for storing supplies, particularly 
condoms, lubricants and HIV tests. 
Based on these findings, the USAID | DELIVER PROJECT developed an intervention to strengthen the 
logistical capacities of the NGOs: the expansion of a pedagogical packet called “Applying Logistics to 
10
Health Care Supplies,” which was used in 2014 for the training of 107 people from 42 NGOs that 
serve key populations. With a methodology for training facilitators, in 2015, 24 facilitators from twelve 
NGOs received grants from USAID PrevenSida and were trained. Both the USAID | DELIVER 
PROJECT and USAID PrevenSida conducted training in capable supervision between 2014 and 2016 to 
accompany the NGO improvement plans, which included providing the tools and equipment necessary 
for adequate storage and inventory control, such as storage guides, posters for humidity and 
temperature control, stowage cards, thermometers, thermohygrometers and electronic Kardex. 
The results of the evaluation indicate that there was a marked improvement in the fulfillment of the 27 
indicators evaluated, improving from 43.4% to 82.9% in all; 22 of the indicators improved, three 
remained the same, and two worsened. The principal advances were observed in having 
thermohygrometers or thermometers (0% to 100%), using thermometers or thermohygrometers, 
having visible expiration dates (0% to 91%), using supply reception forms (0% to 73%), having stock 
data collection forms and stowage cards (6.2% to 91%), having supplies labeled (13% to 91%), having 
sufficient space to store supplies (13% to 82%), applying the PEPE technique (56.2% to 100%), having 
and using consumption data collection forms (50% to 91%), having adequate supply stacking (63% to 
82%) and having clean storage space (69% to 82%). Nevertheless, some aspects remain in need of 
improvement, such as updating stowage cards (0% to 36.4%), using stowage cards and stock data 
collection forms (0% to 46%), and performing physical inventory checks (31% to 55%). 
The key successes were enhancing a plan of action to reduce gaps, focusing on continuous quality 
improvement, developing capacity among the organizations’ work teams, training facilitators, creating 
and distributing tools and the special offering of the USAID PrevenSida in training quality teams, 
enabling supervision visits and supplying equipment. 
The recommendations presented during the baseline evaluation, which focused on improving storage 
and inventory control conditions, were 75% fulfilled. 
Gender focus was integrated into the design, implementation and evaluation of interventions by 
incorporating organizations from LGBTI communities, procuring the development of their personnel 
and institutional capacities and strengthening their leadership. 
Eight of the 11 NGOs believed that technical sustainability related to human resources training and the 
potential to maintain and transfer this knowledge exist within the organization. Only three of the 
NGOs believed that financial sustainability was within reach through alliances and fund management 
with other donors. 
The principal gaps that persisted once the intervention was finished are related to financial limitations, 
the lack of available space for functioning, the lack of an infrastructure to store supplies, the high 
turnover of human resources, systematization of the training of their resources and the limited 
financing by donors that would allow the NGOs to keep their members active. 
The recommendations: 
11
The Mission of USAID Nicaragua should share its own experience and the experience of USAID 
PrevenSida to strengthen the capacities of NGOs on the regional level by direct dissemination of the 
report and the creation of articles to appropriately disseminate their knowledge. 
The USAID | DELIVER PROJECT should disseminate this experience as an example of good practices 
with potential applications in similar contexts. 
USAID PrevenSida should provide continuity to the supervision model with the NGOs, maintaining the 
checklist review as a component of the project’s monitoring routine with the grantee NGOs. 
The NGO should adopt the learning experiences and institutional strengthening reflected in this report, 
continue their process of growth, and seek funding sources to maintain their achievements. 
In general terms, the recommendations focus on evaluating these results and the potential role of 
NGOs in participating in broader logistical efforts under the framework of the WHO country 
recommendations to begin universal treatment for all HIV-positive people; such implementation could 
require more active NGO participation with key populations. 
12
I.INTRODUCTION 
 BACKGROUND
USAID has supported health and development programs in Nicaragua continuously since 1991, 
being the principal funder until 2012 of issues related to population, sexual and reproductive 
health and family planning in Nicaragua. USAID has worked closely with the government of 
Nicaragua, the private sector and non-governmental organizations (NGOs), providing technical 
assistance for logistical management, financing, and training systems for health care providers to 
improve the quality of services.
USAID’s cooperation has significantly contributed to the development of programs related to 
health, nutrition, and maternal and child health, broadening the coverage and consolidation of 
family planning and strengthening the national response to the HIV epidemic. 
In 2010, the Central American region and the government of the United States began the 
Partnership Framework for Cooperation in Response to the HIV/AIDS Epidemic (2010-2014) 
with funding from the United States President’s Emergency Plan for AIDS Relief (PEPFAR). 
PEPFAR’s purpose is to reduce the incidence of HIV/AIDS and its prevalence in high risk 
populations, utilizing an evidence-based focus, an emphasis on government cooperation and a 
technical assistance model focused on increasing the sustainability and appropriation of the 
country. 
In this manner, the Mission of USAID Nicaragua has aligned technical and financial assistance 
from its various projects in the implementation of the PEPFAR framework for cooperation in 
its four areas: prevention, strengthening of health care systems, strategic information and policy 
reform, and prioritizing the needs of key populations (men who have sex with men, sex 
workers and female transgender people). 
With funds from PEPFAR, since 2010, USAID Nicaragua has worked within the strategy of 
strengthening health systems (fortalecimiento de los sistemas de salud – FSS) to develop the key 
population in a more effective manner, coordinating the efforts of implementation allies to 
offer HIV/AIDS services that are sustainable and high quality. These services are centered on
three key areas: offering services, human resources capacity in the area of health, and timely 
and adequate provision of essential medical products.
In this context, through the PrevenSida project, actions have been developed to reduce the 
transmission of HIV/AIDS among key populations, strengthening the institutional capacity of at 
least 50 NGOs to participate in the national response plans to HIV/AIDS. The activities have 
been supported by other projects associated with USAID such as ASSIST, the USAID | 
DELIVER PROJECT, PASMO and PASCA.
13
The USAID | DELIVER PROJECT
The USAID | DELIVER PROJECT, implemented by John Snow, Inc. (JSI), has offered technical and 
financial assistance to Nicaragua since 1998, with the initial purpose of supporting the establishment of 
logistical systems in the Nicaraguan Ministry of Health (Ministerio de Salud de Nicaragua – MINSA) and in 
the Association for the Well-being of the Nicaraguan Family (PROFAMILIA). 
Since 2003, the project has offered MINSA technical assistance to strengthen its actions with regard to 
family planning (FP), providing institutional and inter-agency coordination to guarantee assured 
availability of contraceptive supplies (disponibilidad asegurada de insumos anticonceptivos – DAIA), the 
gradual purchase of contraceptives with fiscal funds, the process of integrating information systems 
regarding contraceptives and medical supplies, the estimation of contraceptive needs, and the 
strengthening of post-obstetric-event contraceptive services. 
Beginning in 2005, the USAID | DELIVER PROJECT broadened its assistance to the Nicaraguan Social 
Security Institute (Instituto Nicaragüense de Seguro Social – INSS) and Social Security Medical Companies 
(Empresas médicas Previsionales – EMPs) through the National Chamber of Health (Cámara Nacional de 
Salud – CANSALUD). In coordination with other associates and donors, the project contributed to
improving the FP indicators and developing abilities among the MINSA personnel to guarantee 
adequate provision of contraceptives in 100% of healthcare units. Similarly, the project contributed to 
the evolution of vertical logistical systems into a singular information system for all medical supplies, 
including contraceptives. 
Between 2010 and 2012, the Automated Information Systems Program for Logistical Management of 
Medical Supplies (Programa Automatizado del Sistema de Información para la Gestión Logística de Insumos 
Médicos – PASIGLIM) was consolidated by national training and equipment; the establishment of the 
Internet in Local Holistic Health Systems (Sistemas Locales de Atención Integral a la Salud – SILAIS) and 
hospitals; the development and implementation of guidelines for acquisition, storage and distribution; 
constant monitoring and completion of logistical information; and quality improvement, processing and 
information-based managerial decision-making. Of the thirty supplies examined for quality control and 
regularly monitored by MINSA through PASIGLIM, four are HIV supplies (two anti-retrovirals, condoms 
and HIV quick tests). 
Beginning with the diagnosis of the country’s storage needs (DAIA/DAISR), the project supported the 
improvement of storage conditions for medical supplies by enhancing 53 plans for repair and 
construction of general medication warehouses. Similarly, the project’s support was a factor in
expanding the diagnostic and development plan for the central storage capacities of MINSA conducted 
by SCMS/USAID; training and offering technical support for best practices in stowage and storage; 
formulating general plans and strategies for the management and development of the medical supply 
system; developing and implementing the strategy for the rational use of supplies (guidelines, manuals 
and evaluation guides, training human resources, organization and function of CURIMs in prioritized 
14
units and municipalities); implementing PASIGLIM and the CURIM strategy in eight EMPs and IPSS; and 
inserting the logistics of medical supplies into the curriculum of training schools in the country’s 
universities. 
The technical assistance of the USAID | DELIVER PROJECT on HIV began in 2011 by MINSA’s request. 
Previously, the project was involved in the decentralization of voluntary tests with consultancy and 
antiretroviral therapy (ART) and the promotion of condom use tied to FP and other vulnerable 
populations. The technical assistance for the design and implementation of SIGLIM required profound 
transformations to unify six inefficient vertical systems in the processes of purchasing, storage, 
distribution, rational use, etc. Although the cooperation began with a focus on FP, its success in this 
area demonstrated the possibility of expanding guidelines and methodologies to the entire set of health 
care supplies, including medications for art, HIV tests and condoms. 
The project also contributed to the strengthening of health care systems, improving the labor force 
capacities by transferring technical content and abilities to university professors and students and 
scientific associations working on HIV logistics. In this project, proposed objectives were tied to 
transferring the logistical and rational use of medication to seven public and private universities by 
implementing the pedagogical packet and developing operative research into supply logistics to
promote evidence-based knowledge management in health care institutions. 
JUSTIFICATION
Between 2013 and 2016, the USAID | DELIVER PROJECT strengthened the logistical management 
capacity of twelve NGOs receiving sub-grants from USAID PrevenSida. The project conducted a 
diagnostic of storage conditions and control of medical supplies and identified gaps in the management 
of condoms and lubricants. These findings and their derived recommendations formed the basis of 
implementing an improvement plan for logistical capacities, including personnel training and the 
formulation of improvement plans for their warehouses, finishing with the training of facilitators in the 
logistics of HIV provisions. In the process of enabling capabilities for NGOs, USAID PrevenSida 
managed to strategically integrate its colleagues, beginning with a recognition of the strengths and 
experience of each. 
In 2014, to train the NGO personnel, the project designed a methodological packet for logistical 
training adapted for NGOs. With this, 107 members from 42 NGOs were trained, including personnel 
from the 16 NGOs included in this initial diagnosis. In 2015, the assistance was strengthened by the 
training of facilitators from twelve NGOs that received grants from USAID PrevenSida and that were 
also included among the initial sixteen. 
This evaluation approaches the problem of the weak logistical capacities for health care supplies in 
NGOs that work with key populations in the last three years, advances following the USAID | 
DELIVER PROJECT intervention, and the principal challenges that persist despite all of the identified 
efforts and interventions. 
15
The objective of this consultancy is to evaluate the performance of the USAID | DELIVER PROJECT 
interventions in strengthening the capacity for health care supply logistics in NGOs that work with key 
populations susceptible to HIV in Nicaragua, conducted during the 2013-2016 period. 
The questions this evaluation answered were the following: 
 Compared with the baseline, which aspects were strengthened in the field of storage and 
control of inventory in the organizations that serve key populations and that were 
beneficiaries of this project? 
 What were the key actions for the training process for transferring abilities in the management 
of the logistical chain, application of tools for management strengthening and measuring key 
indicators? 
 To what extent were the recommendations of the baseline diagnosis fulfilled? 
 How was gender focus applied in the design of the interventions? 
 What actions were sustainable once the intervention was over? 
 What were the gaps that persist following the intervention?
16
II. METHODOLOGY 
The performance evaluation of the interventions in strengthening the NGO logistics implemented by 
the USAID | DELIVER PROJECT was on track to respond to the six guideline questions mentioned in 
the introduction. 
The evaluation has a normative design, beginning with a comparative analysis of the baseline situation 
evaluated in 2013 of the storage and internal medical supply conditions in NGOs that serve key 
populations and the current situation. In Table 1, the methodological aspects are detailed. 
 Table 1. Research questions and applied methodology
Question Type Information Methodology Data 
source
Sample method
Q1 Comparative Quantitative,
Qualitative
Analysis of results Primary The entire process
Q2 Comparative Quantitative,
Qualitative
Analysis of results Primary The entire process
Q3 Descriptive Qualitative Document review Secondary All of the documents
Q4 Normative Quantitative
Qualitative
Analysis of results Secondary All of the results
Q5 Descriptive Qualitative Analysis of results Secondary All of the conclusions
Q6 Comparative Quantitative,
Qualitative
Review of project 
development reports 
(fulfillment of indicators)
Observation and field visits 
to organizations
Focus group discussions
Individual and group 
interviews using guides or 
questionnaires
Primary
The entire process
All the reports
16 NGO participants: 
11 NGO: 65% evaluated
The evaluation developed the following phases: 
Organizational phase:
In the evaluation process, quantitative methods were used beginning with the fulfillment of the 
logistical indicators for health care supplies from each organization and of the set, 
complemented by qualitative information coming from an exhaustive document review and 
from interviews with people involved in the capacitation processes offered by the project and 
with those who have some responsibility in the management and/or storage of supplies. During 
this phase, an initial meeting was held with the USAID PrevenSida and the USAID Mission 
17
teams, with the purpose of reviewing the methodology to use, selecting the organizations to 
visit, incorporating adjustments to evaluation instruments and the fieldwork schedule and that 
of the presentation of the products. 
Field phase
The field phase began with visits to 11 organizations (100% of those planned) that were funded 
by USAID PrevenSida and included in the list of NGOs provided in the TDR. All NGOs that 
had active grants and were therefore managing health supplies were included. Previously, 
coordination was established with each organization, adapting the initial plan to the time 
availability of the organizations. The field work lasted from September 5 -19 of this year. Eleven 
NGO officials who had administrative management roles were interviewed (Annex #3.2). In 
the focus groups, 30 people participated who had technical or promotion roles (Annex #3.3). 
Variables studied
The evaluation focuses on key aspects of the logistical cycle principally related to provisions by 
allocation (generally donations), inventory control, storage and consumption in terms of final 
users. 
The variables studied are presented in Annex #4: Checklist and Annex #5: Polling guide. These 
variables correspond to 27 logistical cycle indicators, which correspond to provision, inventory 
control, product stock data, storage, and distribution or consumption. 
Table 2. Evaluated aspects of the logistical cycle 
Aspect of the logistical 
cycle
Conditions Practices
Provision Has a supply reception form Use a supply reception form 
Estimate needs
Inventory control Has electronic Kardex Uses electronic Kardex
Stock Has form for product stock
Has stowage card
Uses form for product stock
Uses stowage card
Updates stowage card
Distribution or 
consumption
Has forms for delivery or 
consumption
Uses forms for delivery or consumption
Uses PEPE
Absence of expired products
Storage Sufficient space
Adequate lighting
Appropriate humidity
Ventilation
Has a thermohygrometer
Has refrigerators
Clean space
Adequate stacking
Uses a thermohygrometer
Labeled supplies
Visible expiration dates
Absence of toxic products
18
Data collection instruments
The data collection instruments were essentially identical to the instruments applied in 2013 (Annex #4: 
Checklist and Annex #5: Polling guide). The forms were updated to add recent NGO data, the type of 
supplies currently being managed and the type of training and direct technical support received during 
the study period. 
Data collection
The primary data were obtained on two levels: institutional and individual. 
- Institutional level: The Checklist and Polling Guide instruments (Annex #4 and Annex #5) were 
applied in each organization that participated in the initial diagnostic and was active at the time of the 
project evaluation, using the data collection instruments mentioned above. Structural interviews (Annex 
# 7: Instrument for Individual Interview) were also conducted with directors and/or administrative 
personnel from the NGOs, inquiring into the technical assistance received in strengthening logistics, 
transferring abilities, integrating gender focus, examining sustainability, and identifying gaps remaining at 
the end of the project. 
Individual level: focus groups were implemented with operative technical personnel from the NGOs 
to explore new learning in institutional logistics strengthening (type of teaching received, degree of 
appropriation of the issues, aspects strengthened by the training), the characteristics of the subsequent 
supervision and monitoring, lessons learned, sustainability and gaps remaining at the end of the project. 
Data analysis
The data were keyed in by professional personnel, and quality control was conducted on the physical 
surveys to reduce bias. Taking as a base the data obtained from the document review, the structured 
interviews, focus groups, checklists and applied polling guides, the investigation proceeded to qualitative 
synthesis and the construction of quantitative indicators with which descriptive analysis was conducted 
with tables, simple frequencies and percentages. 
Validation of the results
As a component of the methodology, preliminary internal validation of the results was conducted with 
USAID personnel and implementing colleagues, and external validation was conducted with a broad 
number of participants, including colleagues, counterparts and beneficiaries. 
Ethical aspects
An ethical framework was established, respecting the principles of autonomy, confidentiality and the 
volunteer status of the participants both in the spoken interview and the self-administered 
questionnaire. To protect confidentiality, the names of participants were not included on the 
questionnaires. All information collected was duly safeguarded. 
19
III. RESULTS 
3.1 What was the initial situation?
The USAID | DELIVER PROJECT in 2013 began logistics strengthening with twelve NGOs 
receiving sub-grants from USAID PrevenSida to contribute to improving the availability of HIV 
provisions. The baseline diagnostic of storage and control conditions of medical supplies 
identified gaps in the management of condoms and lubricants. 
Of 27 criteria evaluated, only 43% were met. The criteria with the lowest fulfillment 
percentages were as follows: 
 0%: using stock data collection forms, using stowage cards, using supply reception 
forms, updating stowage cards, having visible expiration dates, and having climate 
control 
 6.2%: having stock data collection forms, having stowage cards 
 13%: having sufficient space to store supplies and labeling supplies 
3.2 What was the USAID | DELIVER PROJECT intervention?
The strengthening focus developed jointly by the USAID | DELIVER PROJECT and USAID 
PrevenSida, organized by the Mission of USAID Nicaragua, included the following: 
 An initial baseline diagnostic, 
 A plan for institutional improvement and strengthening based on the findings of the 
baseline diagnostic, 
 A training strategy that included the design of educational materials adapted to the 
needs of the NGOs, 
 Allocating basic tools such as manuals, guides, posters, electronic Kardex, and 
thermohygrometers, 
 A monitoring plan tied to rendering supervision more capable, and 
 A final external evaluation to examine and measure the performance attained. 
In 2014, to train the NGO personnel, the project designed a methodological packet for 
logistical training adapted to the NGOs. The participants selected were responsible for storing 
provisions. In this training, the pedagogical packet “Applying Supply Logistics” was used, 
adapted for NGOs. The course lasted two days. These workshops were organized in 
Managua, Bluefields, León, Bilwi and Río San Juan. 
The workshops trained 107 members from 42 NGOs, including personnel from the sixteen 
NGOs present during the initial diagnosis. In 2015, 24 facilitators were trained in twelve 
NGOs that received grants from USAID PrevenSida and that were also part of the initial 
sixteen (Table 3). 
20
Table 3. Achievement of training goals by the USAID | DELIVER PROJECT
Name Goal/NGO
#
Reached
#
# people made 
capable
Percentage
%
FY13 16 16 Baseline 100
FY14 42 42 107 100
FY15 12 12 24 100
Total 42 42 131 100
During the period evaluated (2013-2015), the 11 organizations received a total of 48 
monitoring and enabling supervision visits, following up on whether the recommendations 
presented during the baseline were enacted. Table 4 summarizes the training, monitoring and 
effectiveness of the supervision training received by each NGO. 
Table 4. Training and rendering NGO supervision capable
NGO
Training by
DELIVER
Rendering supervisors effective by USAID 
PrevenSida and USAID | DELIVER PROJECT 
(# of events)
# of people 2014 2015 2016 Total
1. CEPRESI 3 1 2 2 5
2. ASONVIHSIDA 2 2 2 1 5
3. CEGODEM 2 1 0 1 2
4. IXCHEN 3 1 2 2 5
5. ANICP+VIDA 2 2 2 1 5
6. GAO 5 1 3 2 6
7. ADESENI 3 2 2 1 5
8. CEPS 2 1 2 0 3
9. FSL 2 2 2 1 5
10. ACCCS 3 1 2 1 4
11. OVI 2 1 2 1 4
Total 29 15 21 12 48
3.3 What were the results?
The results of the evaluation indicate that there was a marked improvement in the fulfillment of 
the 27 indicators evaluated, increasing from 43% to 83% of global fulfillment; 22 improved, three 
remained the same, and two worsened (Table 5). 
The principal improvements were observed in having thermohygrometers or thermometers 
(0% to 100%), using thermometers or thermohygrometers, having visible expiration dates (0% 
to 91%), using a form for supply reception (0% to 73%), having stock data collection forms and 
stowage cards (6.2% to 91%), having labeled supplies (13% to 91%), having sufficient space to 
store supplies (13% to 82%), applying the PEPE technique (56.2% to 100%), having and using 
consumption data collection forms (50% to 91%), having adequate supply stacking (63% to 82%) 
and having clean storage space (69% to 82%). Nevertheless, aspects remain that require 
improvement, such as updating the stowage cards (0% to 36.4%), using the stowage cards and 
the stock data collection forms (0% to 46%), and conducting physical inventory checks (31% to
55%). 
21
Table 5. Comparison of the 2013 baseline with the 2016 evaluation 
Logistical cycle indicators 2013 2016 Performanc
e
Stock Data (provision, stock, consumption)
Conditions
1. NGO has a form to collect consumption or delivery data. 50 91 Improved
2. NGO has a form to collect stock data. 6.2 91 Improved
3. NGO has a stowage card. 6.2 91 Improved
4. NGO has an electronic Kardex. 100 100 Stayed the 
same
Practices
5. NGO uses supply reception forms. 0 73 Improved
6. NGO uses a form for stock data collection. 0 46 Improved
7. NGO uses a form for consumption or delivery data 
collection.
50 91 Improved
8. NGO uses a stowage card. 0 46 Improved
9. NGO conducts a physical inventory of supplies. 31 55 Improved
10. NGO uses an electronic Kardex. 100 91 Worsened
11. NGO estimates supply needs (applies to NGO). 100 100 Stayed the 
same
Storage
Conditions
12. NGO has sufficient space to store the supplies. 13 82 Improved
13. The storage place has sufficient lighting. 88 91 Improved
14. The storage place is free of humidity. 88 91 Improved
15. The storage place has adequate ventilation. 63 100 Improved
16. The supplies are kept under lock and key. 56.2 64 Improved
17. The NGO has refrigeration equipment. 19 73 Improved
18. The NGO has thermometers or thermohygrometers. 0 100 Improved
Practices Improved
19. The storage space is clean. 69 82 Improved
20. The stacking of supplies is adequate. 63 82 Improved
21. The NGO uses thermometers or thermohygrometers. 0 91 Improved
22. Absence of harmful toxic substances. 100 91 Worsened
23. Expiration dates are visible. 0 91 Improved
24. Supplies are labeled. 13 91 Improved
Distribution or consumption
Practices
25. The PEPE technique is applied. 56.2 100 Improved
26. Absence of expired supplies. 100 100 Stayed the 
same
27. The stowage cards are updated. 0 36.4 Improved
Total 43 83 Improved
22
Based on the general results, we responded to the evaluation questions: 
Q1 Compared with the baseline, which aspects were strengthened in the field of 
storage and control of inventory in the organizations that serve key populations 
and that were beneficiaries of this project? 
Of the 27 indicators evaluated, 22 improved, three remained the same, and two worsened by 
9% (absence of harmful toxic substances and the use of an electronic Kardex). Of the 22 
indicators that improved, the majority reached a fulfillment level greater than 80% (Table 5). 
The indicators with the greatest performance are as follows: 
Between 91% and 100%: NGO has a form to collect consumption or delivery data. 
 NGO has form to collect stock data. 
 NGO has a stowage card. 
 NGO uses a form for consumption or delivery data collection. 
 NGO has an electronic Kardex. 
 NGO uses an electronic Kardex. 
 The storage area has adequate ventilation. 
 The storage area has sufficient lighting. 
 The storage area is free of humidity. 
 NGO has thermometers or thermohygrometers. 
 NGO uses thermometers or thermohygrometers. 
 The PEPE technique is applied. 
 Absence of expired supplies 
 Supplies are labeled. 
 Expiration dates are visible. 
 Absence of harmful toxic substances 
 Supply needs are estimated. 
Between 81% and 90%: 
 Sufficient space to store supplies 
 Storage space is clean. 
 Stacking of supplies is adequate. 
Between 61% and 80%: 
 NGO uses supply reception forms. 
 NGO has refrigeration equipment. 
 The supplies are kept under lock and key. 
Less than 60%: 
 NGO uses a form for stock data collection. 
 NGO uses a stowage card. 
 The stowage cards are updated. 
23
 NGO runs a physical inventory of supplies. 
Below, the results are analyzed according to the logistical cycle: 
Insofar as the purpose of the logistics is to guarantee that the health supplies required by the 
organizations reach their destination in the correct quantities at the correct time in the correct 
condition and at the correct price, the current appropriation and use of forms that allow NGOs 
to rely on the necessary information to avoid shortages and to conduct the opportune 
distribution of supplies have been key. In measuring the indicators, it is important to differentiate 
between practices and conditions of storage. Practices always depend on people; conditions 
depend on the infrastructure, the environment, etc. 
Provisions: The principal source of provisions for ten of the 11 NGOs is donations; only one 
NGO estimates its needs and acquires products. The use of supply reception forms in the 
warehouse area increased from 0% to 73% of the people currently using that area. The NGOs 
financed by the Global Fund have requisition forms from the Health Supply Center (Centro de 
Insumos Para la Salud – CIPS). 
Inventory control: At the beginning of the logistical cycle, information is collected regarding 
the consumption of products and the stock levels to ensure the availability of supplies to attend 
to the population served. In this sense, the logistical information was strengthened in eight of 
the 11 evaluated aspects. 
- Stock data: Even when there are limits to stock data, it is evident that this area was 
significantly strengthened, increasing the number of organizations that had a stowage card. 
Nevertheless, its use and updating remain limited. Four of the ten NGOs presented evidence of 
its use in 2016; the other six NGOs have records until 2015. Having stock data collection forms 
and a stowage card has significantly improved (6.2% to 91%). Use of a stock data collection 
form and a stowage card (0% to 46%) and conducting physical inventory checks (31% to 55%)
are improved but remain low. Only one aspect worsened, the use of the electronic Kardex 
(100% to 91%). One of the organizations has an Activity-Based Costing (ABC) system, which is 
an automated accounting system that includes a module for managing inventory. 
- Consumption or input of supply data: Ten of the 11 NGOs have their own forms for 
input or distribution of supplies and use those forms for internal control. The forms differ 
among the organizations because the forms depend on the activities developed. Among the 
most commonly observed forms are sheets for egress of material, records of ingress, records 
of delivery of supplies to home bases and records of delivery during visits to persons with HIV. 
Having and using a consumption or delivery data collection form has improved (50% to 91%). 
Storage: Storage is a basic condition for the quality and integrity of health supplies (condoms, 
water-based lubricants, HIV quick tests, medications, periodic replacement and clinical 
laboratory material). It is important to differentiate between storage practices and conditions. 
The practices always depend on the people; the conditions depend on the infrastructure, the 
24
environment, etc. In this component of the logistical cycle, ten of the 11 evaluated indicators 
improved. 
- Improved in terms of conditions: Although the majority of the NGOs improved their 
storage practices, some NGOs also improved their conditions through management, for 
example, obtaining donated poles or poles constructed with their own means. Efforts were 
made to obtain temperature and humidity measurements. Having thermometers or 
thermohygrometers (0% to 100%), having sufficient space to store supplies (13% to 82%), 
having refrigeration equipment (19% to 73%) and having adequately ventilated storage space 
(63% to 100%) have improved: At baseline, there was no information regarding the storage 
temperature during the visit; however, the current evaluation observed that in ten of the 11 
NGOs, the temperature measured by thermometer or thermohygrometer was in an 
appropriate range for the stored supplies. 
- Improved in terms of practices: The personnel notably improved their practices: 
personnel reorganized their physical space to create storage space, modified procedures to 
store the products on poles, kept supplies under lock and key, maintained adequate cleanliness, 
and designated a person to be in charge of filling out the inventory and temperature forms. The 
most prominent indicators are the use of thermometers or thermohygrometers (0% to 91%), 
adequate stacking of supplies (63% to 82%), and clean storage space (69% to 82%). Only one 
aspect worsened, the absence of toxic substances. 
Distribution: The practices related to the final distribution of supplies to the beneficiary 
populations were evaluated. These practices improved in 100% of the aspects evaluated: the 
expiration dates were visible (0% to 91%), the supplies were labeled (13% to 91%) and the PEPE 
technique was applied (56.2% to 100%). Updating of the stowage cards (0% to 36.4%) improved 
but remains low. 
These achievements were possible as long as the trained personnel under the management of 
the organization developed management alliances and other connections with other institutions 
to enable and improve conditions for the management and storage of supplies. 
It is worth noting that the majority of organizations do not currently need to estimate their 
needs. These organizations do not make purchases, with one exception, because the supplies 
are only condoms, lubricants and HIV tests that are assigned in accordance with the quantity 
established according to the number of activities and the beneficiary population reported in the 
POA and in the framework of project performance. 
Q2. What were the key actions for the training process for transferring abilities in 
the management of the logistical chain, applying tools for management 
strengthening and measuring key indicators? 
25
One of the most relevant characteristics of USAID projects that institutional and health-system 
strengthening has enabled is the singular property of permanently articulating various 
training methods. This articulation may be conducted by improving supervision, formal 
workshops or accompanying activities; coaching or technical assistance effectively drives their 
strategies and simultaneously manages to incorporate monitoring and evaluation systems that 
effect the expected results and continuous quality improvement. 
In this sense, the project elaborated a plan of action, proposing goals and objectives to 
encourage the organizations to enter into and adopt good practices for supply storage. This 
mixed plan of action included activities under the responsibility of both projects. 
The key actions were divided into training human resources, monitoring and improving 
supervision, allocating materials and equipment, and making organizational changes. Among the 
most important actions are the following: 
Training or enabling human resources 
- The development of capacities among the work teams of the organizations and their 
management structure is a result of training in basic issues of the logistical cycle in six 
workshops using the pedagogical packet as a tool designed for maintaining the processes of 
Continuous Quality Improvement (Mejoramiento Continuo de la Calidad – MCC) and focused on 
training competencies in the organizations. 
- Training facilitators in 2015 occurred through a workshop on logistics, with the objective 
of strengthening work competencies among NGO personnel for the correct use of 
methodological designs, production of workshop schedules and the development of content 
during the training events. In this event, 24 people from 12 NGOs that serve key populations 
and are grantees through USAID PrevenSida, were trained. 
Monitoring and improving supervision 
- The monitoring visits of the USAID | DELIVER PROJECT were made to all trained 
organizations. A total of 27 visits were conducted, during which the employment of good 
storage practices and inventory control were verified. These visits enabled the determination of 
which practices were best implemented by the use of poles and adequate ventilation compared 
with previous visits. Similarly, those responsible for managing the supplies were supported in 
updating the stowage cards, jointly conducting physical inventory checks of the supplies stored 
in the organizations. In this period, a mean implementation of 80% was reported, which was 
considered highly satisfactory. 
- The focus on developing competencies began with the training of five members of the USAID 
PrevenSida group on issues chosen from the pedagogical packet “Applying Logistics to Health 
Provisions” to develop abilities to follow up on the improvement plans conducted by the NGOs. 
26
- The coaching visits of the USAID PrevenSida team resulted in 21 visits to the 11 NGOs being 
evaluated as a component of the regular monitoring of the project. These visits 
were to offer support and monitor goals as well as budget execution, monitor the 
strengthening of the administrative area, human resources, financial administrative procedure 
manuals, advances in the Single Record System, the acquisition and management of inventory, 
and verify the achievement of recommendations from previous visits. To this methodology was 
added a checklist of twelve items for monitoring the adequate storage of supplies in all of the 
organizations since 2014, achieving complementarity and completion of the processes. When 
gaps were identified, improvement plans were developed, whose implementation was monitored 
by the technical personnel of the project and whose goals were achieved in the majority of 
cases. In addition, there is a record of agreements to accomplish certain goals that were verified 
by the project consultants in subsequent visits. 
Allocating materials and equipment
- Production and delivery of tools serve as a guide to adequate storage conditions for 
NGOs that serve key populations and include a poster of adequate conditions for the storage 
and humidity of supplies managed by the NGOs. Similarly, a poster was created regarding the 
optimal temperature and humidity for supplies stored in the key NGOs: condoms, lubricants 
and quick tests. 
- Useful and practical equipment such as thermometers and thermohygrometers
were allocated to measure the temperature and relative humidity of the air and the 
environment to maintain the storage of supplies in optimal conditions. 
- Provision of an electronic Kardex by USAID PrevenSida for quick tests was adopted for 
the control of other supplies, allowing greater inventory control. 
- Assigning financial resources to some organizations to ensure adequate spaces or 
infrastructure includes purchasing fans, extinguishers, poles and other items to improve storage 
conditions. 
Organizational changes
- The configuration of quality teams promoted by the USAID PrevenSida and ASSIST 
projects was another key element that resulted from follow-up regarding the adequate 
management of supplies. Tasks and responsibilities were assigned to each human resource; one 
important element was the production of a quality management manual. 
27
Additionally, the USAID | DELIVER PROJECT developed a Virtual course in logistics and the 
rational use of medical supplies facilitated by UNAN León, which, in addition to being dynamic 
and interactive, offered a range of knowledge that required a great deal of dedication and time 
to complete successfully. The course lasted six months, 24 weeks (176 hours), beginning at 
the end of 2015, and included individual and group activities with orientation and 
accompaniment. The training was offered to all of the NGOs; however, only 30% of the 
organizations took advantage of this offer because of the difficulty in meeting the 
requirements, which included low educational level, lack of experience in independent study, 
lack of available time during work hours and limitations in reading documents and completing 
the homework assigned in the course. 
The experience of the project in transferring abilities and competencies in managing the 
storage and control of inventory has been unique in the region. In general, the organizations 
perceive USAID and its allies as a source of continuous learning and enrichment processes. 
Q3. To what extent were the recommendations of the baseline diagnosis fulfilled?
The diagnostic conducted in 2013 resulted in a set of recommendations that abetted direct 
actions and helped the organizations themselves conduct an internal reflection process that 
resulted in a plan directed at reducing the gaps regarding the storage and control of their 
inventory. In total, the 11 NGOs received 48 specific recommendations, ranging from three to 
nine each (Table 6). The specific evaluation of these recommendations indicated that 75% 
were fulfilled. Four NGOs fulfilled 100%, another four between 60 and 80%, and two achieved 
less than 50%. 
Table 6. Fulfillment of recommendations by organization
NGO
Type of 
NGO
# of 
recommendatio
ns presented in 
2013
# of 
recommendations 
fulfilled in 2016
% fulfillment
IXCHEN FSW/women 4 4 100
CEGODEM MSM 3 3 100
FSL GBV/KP 7 7 100
ANIC+VIDA Persons + 4 4 100
CEPRESI MSM 5 4 80
ADESENI Transgender 4 3 75
OVI MSM 3 2 67
ASONVIHSIDA Persons + 5 3 60
GAO Persons + 4 2 50
ACCS KP/Persons + 9 4 44
Total 48 36 75
28
The principal recommendations involved the search for resources to implement the 
recommendations, including the production and use of stowage cards for each of the supplies 
within their responsibility, the production of forms for egress of the supplies used in the 
development activities of the promoters, the electronic Kardex to control HIV quick tests, 
producing reception reports for the supplies they received and placed in the safekeeping area, 
producing internal request forms for CIPS and broadening or acquiring shelving for the supplies. 
The training process in issues of health care supply logistics, offered by the Project, created 
expectations in the organizations to improve their operations and motivated the organizations 
to seek the means to create the necessary conditions to ensure that the recommendations 
would not simply remain in the report. 
Not all NGOs have identical capacities and development; some NGOs did not have great 
difficulty in acquiring the necessary means to achieve their goals, having the ability to rely on 
certain sources of income. Other NGOs took advantage of two leaderships developed in their 
territories and utilized alliances to acquire the tools to improve storage conditions. 
In this context, just as the recommendations were presented in the diagnostic, the fulfillment of 
each recommendation was analyzed by the organization and a point was assigned for each 
recommendation fulfilled; the points were totaled and divided by the total number of 
recommendations to derive a quantitative percentage of the level of fulfillment. 
Beyond the numbers and percentages reached, the level of fulfillment can be evaluated as a joint 
achievement because the majority of organizations began at zero on the baseline. In addition, 
most organizations suffer from constant changes in personnel because of the identical financial 
limitations that oblige the organizations to operate with minimal human resources and 
constantly search for financing for economic and institutional sustainability, developing along the 
way simultaneous actions to respond to the need to fill gaps or correct difficulties. 
Q4. How was the gender focus applied in the design of the interventions?
In its programmatic design and as an axis of its management and services, the USAID/HIV 
program fully assumes a reduction in gender inequality. The projects and programs developed 
by USAID have followed the principles that compose the base of the framework of USAID 
gender policies. The following principles have been applied in a satisfactory manner: 
- Integration of gender equality: Organizations that work with LGBTI populations, 
particularly MSM and female transgender people, were incorporated. The focus on gender 
29
equality is evident in the opportunity the NGOs have to enhance sexual diversity by 
empowerment and equality of opportunity. Of the 11 NGOs with institutional strengthening, 
three (27%) were MSM NGOs, one (9%) was a female transgender person NGO, three (27%) 
NGOs were for people with HIV, and four (36%) NGOs worked with sex workers and the 
LGBTI population. Of a total of 29 people trained, 13 (44.8%) were women and 16 were 
(55.2%) men; by sexual orientation, seven (24%) were MSM, three (10%) were female 
transgender people, and 17 (58.6%) were heterosexuals. 
- Empowering women within the programs: Three organizations that work with women 
in situations of gender violence were also incorporated, and organizations of people with HIV 
promote the strengthening of capacities and leadership in these groups to allow these people to 
exercise their sexual rights and live their sexuality more safely. 
- The construction of alliances to integrate sectors that benefit women and key 
populations. Organizations rendered possible the exchange of experiences with social actors 
and alliances with the MINSA and other state institutions that serve key populations. The 
organizations have improved relationships with the community and health service providers, 
facilitating access to services, which in turn contributes to reducing the stigma and 
discrimination against key populations. 
- Adopting an integrating focus to foment equality. The cross-cutting nature of the 
gender focus is specified on two levels of the daily dynamic of the projects that compose the 
program, such as the contents and methodologies of training implementation and the 
institutional strengthening of the organizations in which key populations participate. 
The strengthening of leadership is present in all training opportunities and generates the 
recognition and respect of persons in their communities, strengthening organizations 
comprising sexual diversity and women. 
 
The other dimension in which the gender focus is apparent is the development of the capacity 
to implement organizations and focus on the people who compose these organizations to
participate in a more informed and protagonistic manner in the dynamic of the program itself, 
both perceived in their internal relations and from the perspective of key populations. 
Q5. What actions are sustainable once the intervention is over? 
This issue was broached by each of the NGOs, and the results are synthesized in Table 7. The 
majority of the replies mentioned the technical sustainability related to the development of the 
NGOs’ human resources. Only three of the 11 NGOs considered it possible to have attainable 
financial sustainability by alliances and managing funds from donors. 
30
Table 7. Sustainability of actions according to NGO assessment 
NGO Technical sustainability Financial sustainability Global evaluation
CEGODEM New alliances with other 
projects. It is important for the 
grants to be systematic
Proposals to the European 
Union for other funds on 
the HR agenda to stabilize 
institutional achievements
Possible financial 
sustainability
ADESENI New alliances maintaining 
continuity with previously 
established alliances with MINSA, 
the Office of the Mayor, Duty 
Free Zones
Apply for financing Possible financial 
sustainability
IXCHEN Human resources’ logistical 
training contributes to 
institutional sustainability.
Continuity in seeking 
financing
Human Resources 
sustainability and 
possible financial 
sustainability 
CEPRESI Continuity in training the 
resources of the organization
Human resources 
sustainability
OVI Training and knowledge transfer 
to new personnel
Human resources 
sustainability
GAO With the development of 
institutional capacities and by 
inter-organizational alliances
Possible institutional 
sustainability
FSL Maintaining the volunteer force Human resources 
sustainability
CEPS Education as facilitators enables
training new resources and 
continuing to develop good 
storage practices
Human resources 
sustainability
ANICP+VIDA Storage and inventory control 
conditions imply costs (energy 
bills to maintain ventilation and 
purchase of papers)
Dependent on financing Uncertain
ASONVIHSIDA Many technical limitations Low financing Uncertain
ACCCS Dealing with private companies 
to acquire poles through 
alliances
Dependent on their own 
income
Uncertain
31
Recognizing the importance of logistical quality management
Recognizing the importance of logistical health care supply management by the management of 
the organization contributes to the development of the organization and enables other donors 
to apply for new grants that contribute to sustainability. Incorporating the indicators of the 
logistical chain in the operative plans and quality manuals of the organizations offers the 
opportunity for permanent monitoring of all actions, guaranteeing adequate management and 
storage of supplies. The improvement of logistical management has contributed to financial 
stability and, therefore, to greater institutional sustainability. 
Permanent human resources training
Sustainability resides in actions that develop the competencies of the human capital of each 
organization and of the capacity to accumulate knowledge and experience to be shared 
internally and with other organizations. Facilitator training has been used by twelve NGOs, to 
whom responsibilities in supply management have been assigned, including training new human 
resources entering the organizations. This includes personnel attitudes toward the work to 
always obtain the best results. 
Experience exchange within the network
The dynamic of the organizational functioning as a network contributes to the sustainability of 
the logistical capacity for strengthening actions, both as exchanging experiences and good 
practices for growth. For some organizations, sustainability is associated with constant alliances 
with other organizations. 
Q6. What are the gaps that persist after the intervention is over?
The majority of organizations identified the financial limitations when grants or projects end. 
Similarly, not having their own location becomes a threat to being able to maintain supplies 
under optimal conditions because organizations can lose the conditions and progress achieved 
when changing locations. 
Some additional gaps identified during the field visits of this evaluation are as follows: 
- Lack of infrastructure to store supplies: Some organizations do not have adequate 
locations for the quantity of supplies the organizations receive as donations. For example, the 
amount of condoms assigned to some organizations has increased considerably, affecting 
storage conditions. 
- Turnover of NGO human resources: The turnover of human resources in organizations 
affects the application of guidelines and standards in the logistical cycle. 
- Lack of systematic training: Even when there are facilitators from the NGOs to continue 
to conduct training, the organizations lack financing to facilitate training sessions. 
32
- Periods without external funding: The waiting time for new grants is a critical period 
during which the NGOs do not have the resources to pay the salaries of their network of 
promoters and the services the NGO provides to the populations that benefit. 
Table 8. Persistent gaps in the NGOs in September 2016
NGO Gaps
CEGODEM Lack of financing continuity
Loss of systematization of interventions
CEPRESI Continued human resources training
Lack of their own climate control equipment
ASONVIHSIDA Lack of adequate physical space
IXCHEN Improve inventory control for educational materials
ANICP+VIDA Lack their own location
ADESENI General institutional strengthening, financial limitations
GAO Financial
OVI Maintaining equipment in working order
CEPS Financing to replicate learning
FSL No capacity to maintain supply stock
ACCCS Lack of institutional budget
The lessons learned reported by the NGOs are as follows: 
 Keeping track of documents and supplies is of the utmost importance 
 The learning was useful in improving the organization of work 
 The ability to control the input and output of supplies prevents being left with zero 
supplies. 
 Strengthening allows for a better positioning of organizations. 
 Maintaining the practice of adequate storage. Everything should follow guidelines and 
protocol. 
 All practices and all processes should be documented. 
 Currently, the expiration dates of supplies are reviewed before receiving the supplies 
and passing the supplies on to the warehouse 
33
IV. CONCLUSIONS 
The USAID | DELIVER PROJECT successfully strengthened the logistical capacity of health 
supplies in NGOs that worked with key populations with HIV in Nicaragua from 2013-2016. 
The results of the evaluation indicate that there was a marked improvement in fulfillment of the 
27 criteria evaluated, increasing from 43.4% to 82.9% of global fulfillment. Twenty-five aspects 
were strengthened, three remained the same, and two worsened. 
The principal advancements were observed in having thermohygrometers or thermometers (0% 
to 100%), using thermometers or thermohygrometers, having visible expiration dates (0% to 
91%), having collection forms for stock data and a stowage card (6.2% to 91%), having labeled 
supplies (13% to 91%), using a supply reception form (0% to 73%), having sufficient space to 
store supplies (13% to 82%), applying the PEPE technique (56.2% to 100%), having and using 
forms for consumption data collection (50% to 91%), having adequate supply stacking (63% to 
82%) and having clean storage space (69% to 82%). Nevertheless, some aspects remain in need 
of improvement: using the stowage card and stock data collection (0% to 46%), updating the 
stowage card (0% to 36.4%) and performing physical inventory checks (31% to 55%). 
The indicators that achieved less than 60% fulfillment were using a stock data collection form, 
conducting physical inventory checks of supplies, and using and updating a stowage card. 
The key actions were elaborating on a plan of action to reduce gaps, focusing on continuous 
quality improvement, developing capacities in the organizations’ work teams, training 
facilitators, creating and using tools, receiving special input of USAID PrevenSida in the training 
of quality teams, enabling supervision visits, receiving endowments for equipment, and 
facilitating electronic Kardex. 
The level of fulfillment of the 48 recommendations from the baseline diagnostic was 75%. These 
recommendations focused on improving storage conditions and practices, improving inventory 
control, creating and using supply input and output forms, and using Kardex to control HIV 
quick tests. 
Gender focus was integrated into the design of the intervention by incorporating organizations 
from LGBTI communities, procuring the development of personal and institutional capacities 
34
that contribute to improving HIV prevention services for key populations, strengthening 
leadership and empowering these communities. 
The sustainability of these actions is possible by incorporating organizational, procedural and 
behavioral changes in the management of the logistical cycle. Eight of the 11 evaluated NGOs 
considered technical sustainability related to human resources development to be possible, and 
only three believed that financial sustainability was possible by alliances and management of 
funds from other donors. 
The gaps that persist are completing the technical assistance to correspond to financial 
limitations (both from internal and external sources), the lack of availability of their own spaces 
for operation, and rotation and loss of human resources, which are directly related to 
limitations in financing permanent work positions. 
35
V. RECOMMENDATIONS 
FOR THE MISSION
To share the experience of the USAID | DELIVER PROJECT and USAID PrevenSida in 
strengthening the logistical capacities of NGOs on the regional level by directly disseminating 
the report and by writing articles to disseminate the knowledge in appropriate formats. 
To incorporate the strengthening of logistical capacities in future programs for HIV care in key 
populations. 
FOR USAID | DELIVER PROJECT
To document and disseminate this experience as an example of good practices with the 
potential of applying these practices in similar contexts. 
FOR USAID PREVENSIDA
To follow up and ensure continuity to the enabling supervision model with NGOs, maintaining 
the checklist review as a component of the project monitoring routine with grantee NGOs and 
emphasizing the use of tools that ensure adequate storage and control of inventory. 
To facilitate the use of the pedagogical packet “Applying Logistics” for trained NGO facilitators 
to transfer abilities in storage and inventory control when onboarding new personnel and when 
updating trained personnel. 
To maintain logistics issues through workshops at the beginning of grant periods to facilitate 
coaching, experience exchange and internships among NGOs. 
To accompany the implementation of pending recommendations. 
FOR NGOS
To take as their own the learning and institutional strengthening experiences reflected in this 
report and to continue with their growth process and search for funding sources to maintain 
their achievements. 
To analyze the effects of personnel rotation in the various stages of the logistics process and to 
evaluate the measures that could be taken to reduce that rotation, including expanding training 
to more team members and reflecting on the utility of having greater control over supplies. 
36
To reinforce the use of standardized requisition forms that fulfill all technical requirements. 
In general terms, recommendations include the evaluation of these results and the potential role 
of NGOs in participating in broader logistics efforts in the framework of the World Health 
Organization’s country recommendations and beginning universal treatment for all HIV-positive 
people. Such implementation could require more active participation from NGOs focusing on 
key populations. 
37
V. BIBLIOGRAPHY 
1. Aráuz, C., & Núñez, J. (2013). Diagnóstico: condiciones de almacenamiento y control 
interno de insumos médicos en organizaciones no gubernamentales que atienden a 
población de alto riesgo en VIH-Sida 2013. Managua, Nicaragua.
2. MINSA. (2007). SIGLIM. Manual de Procedimientos. Managua, Nicaragua. Disponible en: 
www.minsa.gob.ni/index.php/.../SIGLIM-Sistema-de-Información-para-la-Gestión
3. USAID DELIVER. (2007). Nicaragua, Informe Final de País. Nicaragua.
4. USAID DELIVER (2010) Una Alianza Modelo entre El Instituto Nicaragüense de 
Seguridad Social y el Ministerio de Salud. Nicaragua.
5. USAID DELIVER, U. (2011). Manual de Logística. Guía práctica para la gerencia de cadenas 
de suministros de productos de salud . Managua. Disponible en: 
http://DELIVER.jsi.com/dlvr_content/resources/allpubs/guidelines/LogiHand_ES.pdf 
6. USAID DELIVER. (2012). Informe trimestral abril -junio. Managua. Nicaragua.
7. USAID DELIVER (2013). Informe anual 2013 Nicaragua. 
8. USAID DELIVER (2014) Informe anual 2014. Nicaragua. 
9. USAID Nicaragua (2015) HIV Bilateral Program Mid Term Performance. Period 2007-
2013. Nicaragua: USAID Nicaragua; November 2015. Disponible en
http://pdf.usaid.gov/pdf_docs/pa00k865.pdf
10. USAID Nicaragua (2015). Jarquín, Y., Larios, M., Hernández, C., & Ochoa, J. Evaluación 
del Componente de Capacitación del Programa Bilateral de VIH USAID Nicaragua. Managua. 
Disponible en: http://www.prevensida.org.ni/index.php/documentos
11. USAID PrevenSida (2014) Informe anual. Año 2014. Nicaragua.
12. USAID DELIVER (2015) Informe anual 2015. Nicaragua. 
13. USAID PrevenSida. (2015) Proyectos de subvención de ONG. Nicaragua.
14. USAID PrevenSida. (2016) Informe anual 2016. Nicaragua. 
15. USAID PrevenSida. Reporte de coaching a ONG. Años 2014, 2015, 2016. Nicaragua. 
16. USAID PrevenSida, U. (2015). Paquete pedagógico para desarrollar competencias en 
gerencia administrativa y técnica del programa de prevención de VIH. Managua, Nicaragua. 
Disponible en http://www.prevensida.org.ni/index.php/documentos
17. USAID PrevenSida, U. (2016). Intercambio de experiencias: “Fortaleciendo la cadena de 
suministros para VIH en las ONG que atienden población clave”. Managua, Nicaragua. 
Disponible en: http://www.prevensida.org.ni/index.php/noticias/217-intercambio-de￾experiencias-fortaleciendo-la-cadena-de-suministros-para-vih-en-las-ong-que-atienden￾poblacion-clave
38
VI. APPENDICES 
APPENDIX # 1: Terms of reference 
PERFORMANCE EVALUATION OF THE COMPONENT OF 
LOGISTICAL STRENGTHENING FOR THE NGO BY THE DELIVER 
PROJECT (EXTERNAL EVALUATION) 
1. Type of contract: Individual 
consultant 
2. Place of contract: Nicaragua 
Background: 
USAID has supported health and development programs in Nicaragua continuously since 
1991, being the principal funder until 2012 of issues of population, sexual and reproductive 
health and family planning in Nicaragua, working closely with the government of Nicaragua, the 
private sector and non-governmental organizations (NGOs). USAID has provided technical 
assistance for logistical management, financing and training systems for health care providers to 
improve the quality of services.
With funds from the United States President’s Emergency Plan for AIDS Relief (PEPFAR), since 
2010, USAID Nicaragua has worked within the strategy of strengthening health systems 
(fortalecimiento de los sistemas de salud – FSS) to develop the key population in a more effective 
manner by coordinating the efforts of implementation allies to offer HIV/AIDS services that are 
sustainable, high quality, and centered around three key areas: offering services, human 
resources capacities in the area of health and timely and adequate provision of essential 
medical products. 
As a component of this effort by the PrevenSida project, actions have been developed to 
reduce the transmission of HIV/AIDS among key populations, strengthening the institutional 
capacity of at least 50 NGOs to participate in the national response plans to HIV/AIDS. The 
activities have been supported by other projects associated with USAID such as ASSIST, 
DELIVER, PASMO and PASCA.
In 2013, strengthening of the logistics of the DELIVER Project began, starting with twelve 
NGOs who received sub-grants from PrevenSida (See Annex I) to contribute to improving the 
availability of HIV supplies. A diagnostic of storage and control conditions of medical supplies 
identified gaps in the management of condoms and lubricants; this diagnostic became the basis 
for implementing the strategy of personnel training and the formulation of plans to improve 
their warehouses, ending with the training of logistics facilitators for HIV supplies (See 
Annexes). 
To train personnel, a methodological packet for logistical training adapted to NGOs was 
designed. In 2014, 107 members from 42 NGOs were trained, including personnel from the 16 
NGOs included in the diagnostic. In 2015, facilitators from 12 NGOs that received grants from 
PrevenSida and that were also a portion of the 16 initial NGOs were trained. In the Annexes 
section, a list of potential NGOs to evaluate is presented.
39
The questions this evaluation should answer are as follows:
 Compared with the baseline, which aspects were strengthened in the field of storage 
and control of inventory in the organizations that serve key populations and that were 
beneficiaries of this project?
 What were the key actions for the training process for transferring abilities in the 
management of the logistical chain, applying tools for management strengthening and 
measuring key indicators?
 To what extent were the recommendations of the baseline diagnosis fulfilled?
 How was gender focus applied in the design of the interventions?
 What actions are sustainable once the intervention is over?
 What gaps persisted after the intervention was over?
The evaluation will begin with coordination meetings between USAID, the PrevenSida project 
and the evaluator. The evaluation will include an exhaustive review of the bibliographic and 
document material (baseline diagnosis, educational materials, reports from the DELIVER and 
PrevenSida projects). The evaluation should include quantitative and qualitative methods by 
applying individual surveys and interviews using the methodology of focus groups to describe 
the degree of appropriation of the issues. It is expected that at least 11 NGOs from the list in 
the annex will be visited (65% of 17). To evaluate the sustainability of the strategies supported, 
advances and gaps in institutional and individual capacity for logistics in HIV supplies will be 
identified, which will allow the NGOs to face their needs and the challenges to sustainable 
development, offering a clear idea of their strengths, weaknesses and opportunities for 
improvement. For the external evaluation meeting with members of NGOs and personnel 
from the project and from USAID, the consultant is expected to organize and host a midday 
event. For the translation of the document from Spanish to English, an internationally certified 
site is required. For example, the estimated cost of a document between 8,000 and 10,000 
words is between $1,200 and $1,350 (approximately 20 pages of central document and two or 
three weeks for the translation process). All of the costs of land and air transport, travel, 
printing, communication, translation, and event organization should be included in the 
proposed budget.
General objective of the consultancy: 
To evaluate the performance of the DELIVER project in strengthening the capacity of health 
supply logistics in non-governmental organizations (NGOs) that worked with key populations 
for HIV in Nicaragua from 2013 to 2016.
Required structure of the evaluation report
The report will have a maximum length of 20 pages, not including annexes. The report should 
be presented in the format required by Branding & Marking of USAID.
I. Executive Summary
II. Introduction: background, justification, objective
40
III. Methodology: qualitative and quantitative
IV. Results: description of the interventions conducted, comparison with baseline
V. Discussion and Conclusions: evaluation of the advances and gaps, sustainability, 
application of gender focus, sustainability potential
VI. Recommendations
VII. Annexes: instruments used, tables, figures, pictures, etc.
VIII. Bibliography and documents consulted
3. Duration and schedule 
Estimated duration of the consultancy: 7 weeks
 Organization, coordination and instrument design phase: one week
 Field phase: two weeks
 Writing the report in Spanish: one week; translating the report into English: three 
weeks
The principal tasks to fulfill are listed below and should be presented according to their level of 
effort, expressed in number of days.
Activities
1. Coordination meeting between PrevenSida and USAID
2. Elaboration on and presentation of a detailed work plan, including field visits
3. Bibliographical and document review
4. Interviews and visits to NGOs in Managua (at least 7)
5. Interviews and visits to NGOs in León, Ocotal, San Carlos and Bluefields (4)
6. Writing report draft #1
7. Validation meeting between PrevenSida and USAID
8. Workshop to return the results of the evaluation
9. Writing report draft #2
10. Meeting to present the final report in Spanish
11. Writing the final report in Spanish
12. Translation and review of final report in English
4. Expected products 
 First report with detailed work plan
 Report of field phase 
 Final evaluation report in English and Spanish. Of the final document, 30 copies are 
required in Spanish and 5 copies in English (full color cover and black and white 
interior, spiral- or perfect-bound). In addition, a CD with the versions in Word and 
PDF will be delivered.
5. Required qualifications/professional experience:
 Education: health sciences professional with educational courses in health supply 
logistics
 Experience: at least five years of experience in monitoring and evaluation in the field of 
health, preferably in the field of health supply logistics
 Preferable: experience in writing documents according to USAID publication standards
41
 Required: present prior evaluation work experience
6. Coordinator: Oscar Nuñez, Coordinator USAID|PrevenSida. 
To contract the person proposed will require the VoBo of the Mission.
7. Deadline to receive technical financial offers: August 26, 2016 at 3:00 pm.
8. Requirements:
Present letter of intent confirming immediate availability
Updated CV focused on the topic of the evaluation, maximum 3 pages
Technical offer and financial offer in US dollars separately
9. Correspondence address: 
Direct technical and economic proposals to participate in this consultancy to Yudy Wong.
The technical-financial proposal can be submitted physically or by email to cwong@urc￾chs.com
Address; De donde fue la Vicky Altamira. 1cuadra abajo, 1 cuadra al sur.
Plaza San Ramón. Segundo piso. Módulo 9. Managua Nicaragua
Direct Line: (+505)2270 6747, 2278 0392 ext 102
10.Prepared by: Oscar Nuñez 11. Date: August 12, 2016
12. Approved by: 
Dr. Marianela Corriols, USAID Nicaragua
Lic. Marcela Villagra, USAID Nicaragua 
13. Date: August 12, 2016
BIBLIOGRAPHY
Aráuz Carolina, Jairo Núñez 2013. Diagnóstico: condiciones de almacenamiento y control 
interno de insumos médicos en organizaciones no gubernamentales que atienden a población 
de alto riesgo en VIH-Sida 2013. Arlington, Va.: USAID | PROYECTO DELIVER, Orden de 
Trabajo 4. 
Documentos clave de los proyectos DELIVER y PrevenSida accesibles en el DEC 
USAID. https://dec.usaid.gov/dec/home/Default.aspx
42
LIST OF NGO PARTICIPANTS
Department Name of the NGO
Managua, Managua 1. ANICP+VIDA
2. ASONVIHSIDA
3. IXCHEN
4. CEGODEM
5. CEPRESI
6. RED TRANS
7. OVI
8. ACAJ-PELG
9. ICAS
León 10. GAO
Managua, Mateare 11. ADESENI
Nueva Segovia, Ocotal 12. CEPS
Rio San Juan, San 
Carlos
13. Fundación San Lucas
RACCS, Bluefields 14. ACCS
15. RDS-RAAS
RACCN 16. FADCANIC
43
APPENDIX # 2: Schedule of field visits 
Source: Work plan and methodology. Evaluation of the performance of strengthening the 
capacity of NGOs in Nicaragua, 2016
Territory NGO Goal Population Date
Managua CEPRESI MSM 09/05/2016
ASONVIHSIDA PHIV 09/06/2016
CEGODEM MSM 09/07/2016
ANICP+VIDA PHIV 09/07/2016
IXCHEN SW 09/07/2016
OVI MSM 09/19/2016
ADESENI TRANS 09/08/2016
León GAO PHIV 09/08/2016
Ocotal CEPS MSM and Mobile 
populations
09/09/2016
Rio San Juan FSL SW, PHIV, MSM and 
Mobile populations
09/10/2016
RACCS ACCS SW, PHIV, MSM and 
Mobile populations
09/13/2016
44
APPENDIX # 3: Persons interviewed 
3.1 Persons individually interviewed at USAID and USAID projects 
No Person interviewed Institution
1 Dr. Marianela Corriols, HIV Advisor Misión/USAID Nicaragua
2 Dr. Carolina Aráuz, Resident Advisor USAID/ DELIVER 2013-2016
3 Dr. Oscar Nunez, Chief of Party USAID/ PrevenSida 2010-2017
3.2 Persons individually interviewed at NGOs 
No Name Type of 
Population served Location Person interviewed Email
1 CEPRESI PEMAR Managua Norman Gutiérrez direccion@cepresi.org.ni
2
ASONVIHSIDA PVS Managua Bruce Menzies Offer asonvihsida@asonvihsida.org.ni
3 ANICP + VIDA PVS Managua Julio Mena juliocesarmena@hotmail.com
4
ANFAM / 
IXCHEN Women, SW, GBV Managua María Lourdes 
Rodríguez Bolaños ixchennacional1988@gmail.com
5
ADESENI PEMAR Mateares Marlene Vivas adeseni-mateare@hotmail.com
6
CEPS
Victims of GBV, 
Treatment of people 
PEMAR
Ocotal Rommel Ponce
ceps@ibw.com.ni
7
CEPS
Victims of GBV, 
Treatment of people 
PEMAR
Ocotal Ingrid Reyes
ceps@ibw.com.ni
8
Fundación San 
Lucas
Victims of GBV, 
Treatment of people 
PEMAR
RSJ Sonia Herrera
sonyliz03@yahoo.es
9
ACCCS PEMAR, Uniformed Bluefields Adolfo Rosales 
Ordoñez acccsida@yahoo.com
10 ACCCS PEMAR, Uniformed Bluefields Keitha Cooper acccsida@yahoo.com
11 OVI MSM, Trans, SW Managua Nelson González 
Moreno <oviorg2011@hotmail.com>
45
3.3 Persons interviewed in focus groups 
Name
Type of 
Population 
served
Location Focus Groups Email
CEPRESI PEMAR Managua María Antonia Sánchez, direccion@cepresi.org.ni
Oscar Guillen
Salvador Reyes
ASONVIHSIDA PVS Managua Bismark Cortez Castro asonvihsida@asonvihsida.org.ni
Oxana Delgadillo
Martha Berrios
Gary Mena
CEGODEM PEMAR Managua Manuel Calderón fidelmoreira@gmail.com
Jorge Luis Rivera 
Fidel Moreira
ANICP VIDA PVS Managua Luis Espino Delgado
Sandra Largaespada
Julio Mena juliocesarmena@hotmail.com
IXCHEN Women, SW, 
GBV Managua Patricia Marenco Ríos ixchennacional1988@gmail.com
Ana Marina López
Flavia Garay
GAO PVS León Salvador Medal grupoautoayudaoccidente@gmail.co
m
Martha Torrez
Gerald Zelaya
Pablo Salazar
Norma Rubí
Yader Romero
ADESENI PEMAR Mateares Tracy Boniche adeseni-mateare@hotmail.com
Cristabella Berríos
Amanda Moncada
Natasha Hernández
Natsumi Hernández
FSL
Victims of 
GBV, 
Treatment of 
people PEMAR
RSJ Lidieth Arana
lidietharare@yahoo.com
Darving Reyes
Mirjana Mondragón
46
3.4 Participants in internal validation 
No Name Institution
1 Nora Quezada Directora Regional JSI
2 Oscar Núñez USAID PrevenSida
3 Carolina Aráuz DELIVER
4 Marianela Corriols USAID Nicaragua
5 Marcela Villagra USAID Nicaragua
3.5 Participants in external NGO validation 
No Name NGO From Telephone no.
1 Bruce Menzis ASONVIHSIDA Managua 89996344
2 Gary Mena Guevara ASONVIHSIDA Managua 88215942
3
María Lourdes 
Rodríguez IXCHEN Managua 89969212
4 Patricia Marenco Ríos IXCHEN Managua 58420725
5 Darvin Reyes H FSL Río San Juan 58708073
6 Sonia Herrera FSL Río San Juan 89188852
7 María Antonia Sánchez CEPRESI Managua 22707988
8 Rommel Ponce CEPS Ocotal 82583413
9 Marlene Vivas ADESENI Managua 87146652
10 Cristabella Berríos ADESENI Managua 89791805
11 Julio Mena ANICP VIDA Managua 86826573
12 Sandra Largaespada ANICP VIDA Managua 86541739
13 Yudy Wong USAID PREVENSIDA Managua 86719688
14 Oscar Núñez USAID PREVENSIDA Managua 88504631
15 Yelba Jarquín CONSULTORA Managua 84352427
47
APPENDIX # 4: Checklist
CHECKLIST FOR THE FOLLOW-UP ON LOGISTICAL ASPECTS IN THE NGOs 
THAT ATTEND KEY POPULATIONS FOR HIV
Objective: To verify the implementation of the improvements in internal control and storage 
conditions of health supplies in the organizations trained with the pedagogical packet “Applying 
Logistics to Health Supplies” 
Date: ___/____/____ Name of the NGO:________________________ 
Name of the person interviewed _____________________________________ 
Was he/she trained in health supply logistics? YES_____ NO______ 
Name of the person supervising ____________________ 
ITEMS TO VERIFY YES NO
1. The NGO has a form to collect consumption or delivery data.
2. The NGO uses a form to collect consumption or delivery data.
3. The NGO has a form to collect stock data.
4. The NGO uses a form to collect stock data.
5. The NGO estimates supply needs.
6. The NGO has a stowage card to control supply.
7. The NGO uses a stowage card to control supply.
8. The NGO runs physical inventory on supply.
9. The NGO has sufficient space to store supplies.
10. The NGO’s storage space is clean.
11. The storage space has sufficient lighting.
12. The storage space is free of humidity.
13. The storage place has adequate ventilation.
14. The supplies are kept under lock and key.
15. The stacking of supplies is adequate.
48
ITEMS TO VERIFY YES NO
16. The PEPE technique is applied.
17. Absence of expired supplies
18. The supplies are labeled.
19. The stowage cards are updated.
20. The expiration dates are visible.
Total items fulfilled
Percentage of fulfillment: Total items fulfilled/20*100
Observations:____________________________________________________________
_______________________________________________________________________
______________________________________________________________________ 
49
APPENDIX # 5: Polling guide 
POLLING GUIDE ON THE LOGISTICAL, INFORMATION AND 
STORAGE SYSTEMS FOR MEDICAL SUPPLIES
NGOs FUNDED BY 
USAID PREVENSIDA
I. ESTABLISHMENT DATA
(This section collects information related to the location of the establishment, date of visit and 
participants in the monitoring and/or supervision process. Be precise in each item considered).
DATE: _/ _/ 
(Specify month, day and year) 
Name of the participating NGO: 
(Include the name and role of each person involved in the monitoring and/or supervision process: 
Supervising team and participants from the health establishment visited).
Name of the Participant Role
1.___________________________________________________________________ 
2.___________________________________________________________________ 
3.___________________________________________________________________ 
4.___________________________________________________________________ 
5.___________________________________________________________________ 
II.TYPE OF PRODUCTS (Mark all)
Condoms Yes No 
Lubricants Yes No 
Quick tests Yes No 
Educational materials Yes No 
Basic medication stock Yes No 
Other Yes No 
Specify:_________________________________________________
 ________________ 
50
III. ESSENTIAL LOGISTICAL DATA
What are the sources of information used to collect CONSUMPTION data?
Confirm the information offered by the interviewee by randomly reviewing a SIGLIM report, and
contrast the consumption data reported by the prescriptions sent with one of the other sources of 
information used to obtain consumption data. If using another instrument, specify in Observations.
Pre-designed forms Yes No
Other Source:
Observations:
Yes No
What are the sources of information used to collected the STOCK data? Confirm the information 
offered by the interviewee by randomly reviewing a SIGLIM report, and contrast the stock data included 
in the report with one of the sources of information used to obtain the stock data. If using another 
instrument, specify in Observations.
Stowage Cards Yes No
Kardex Cards Yes No
Physical Inventory Yes No
Other Source: Yes No
 Observations: 
51
V. PROVISION SYSTEM
1. Does the NGO have an established period of re-stocking 
for their level of resolution? 
Yes No Observations:
2. Does the NGO have established registry forms for the input of supplies they manage? 
Yes No Observations:
3. Does the NGO have established registries for the output of supplies they manage? 
Yes No Observations:
4. Has the NGO made emergency requests in the last 2-3 months? 
Corroborate this information with the reports of supply movement and the stowage 
cards. Clarify the cause in the observations.
Yes No Observations:
V. INVENTORY CONTROL SYSTEM
1. Does the NGO have an inventory control system for the supplies? If possible, show the 
system. 
Yes No Observations:
2. Does the NGO run inventory on supplies, by period, by whom? If possible, specify details. 
Yes No Observations:
3. Does the NGO perform selection of supplies by accounting, by frequency, by whom? 
Yes No Observations:
52
VI. STORAGE SYSTEM
(This section allows the evaluation of the fulfillment of adequate storage 
conditions and their timely availability).
Verify the basic storage conditions in the warehouse or office area.
Perform a visual inspection and corroborate each of the aspects enumerated in each 
area.
Sufficient space Yes No 
Clean space Yes No 
Sufficient lighting Yes No 
Dry environment Yes No 
Adequate ventilation Yes No 
Security of the supplies Yes No 
Adequate stacking Yes No 
PEPE stacking Yes No 
Absence of harmful toxic substances Yes No 
Absence of expired supplies Yes No 
Labeled supplies Yes No 
Updated Stowage Cards Yes No 
Visible expiration date Yes No 
Climate control Yes No
2. Has there been damage to supplies? The stock control cards and the record 
of losses will be reviewed to verify damage to supplies, and damage will be noted 
in the corresponding box. If there has been damage, the cause should be 
explained in the observations box.
Yes No Observations:
Other Findings:
Agreements:
VII. CHARACTERIZATION OF TRAINING
VIII. BACKGROUND OF COACHING AND SUPERVISION
IX. SUSTAINABILITY
53
APPENDIX # 6: Instruments for the focus group 
OPEN QUESTIONS FOR THE FOCUS GROUP 
1. Do you consider that the DELIVER project generated new learning for institutional strengthening? If 
so, what aspects were strengthened with the capacities developed by DELIVER? (If not, explain) 
2. Was there some type of follow-up or monitoring for the praxis or implementation of the learning? 
Someone from the organization? ________________ How often? _________________ 
From the DELIVER project? ___________________At what intervals? ______________ 
3. What lessons were learned from the entire process of strengthening the health supply chain? 
4. What actions developed by DELIVER could be sustainable in the NGO? How? 
5. What are the gaps? 
APPENDIX # 7: Instrument for individual interview 
INSTRUMENT FOR INTERVIEWS WITH NGO DIRECTORS AND/OR 
ADMINISTRATIVE PERSONNEL 
Name of the NGO_____________________________________ Years of Trajectory___________ 
Name_____________________________________________________________Age________ 
1. What key actions will the DELIVER project implement to strengthen the capacities in the 
logistics of health supplies in the organization? 
2. What interventions developed by the DELIVER project allowed for the transfer of abilities in 
managing the logistics chain and strengthening management in this organization? 
3. In what manner was the focus on gender integrated into the interventions developed by 
DELIVER? 
4. What aspects do you consider were not developed by DELIVER with regard to the 
strengthening of capacities to manage the logistics chain? 
5. What actions have been developed for the sustainability of the interventions developed by the 
projects, and what are the gaps? 
 
54
APPENDIX # 8: Evidence 
 
 
 
 
 
CEPRESI GA0
CEPRESI CEPS
CEPS
FUNDACION SAN LUCAS
55
APPENDIX # 9: General output table of the 27 indicators evaluated 
ITEMS TO VERIFY CEP
RESI
ASONVI
HSIDA
CEGO
DEM
ANICP+
VIDA
ANFAM/I
XCHEM
GA0 ADES
ENI
CEP
S
Fund. Sn 
LUCAS
ACCCSi
da
OVI % FULFILLMENT BY
ITEM
YES NO
1 The NGO has a form to collect 
consumption or delivery data. Yes Yes Yes Yes Yes Yes Yes Yes Yes NO Yes 90.9 9.1
2 The NGO uses a form to collect 
consumption or delivery data. Yes Yes Yes Yes Yes Yes Yes Yes Yes NO Yes 90.9 9.1
3 The NGO has a form to collect stock 
data. Yes Yes Yes Yes Yes Yes Yes Yes Yes NO Yes 90.9 9.1
4 The NGO uses a form to collect 
stock data. NO NO Yes Yes Yes NO NO Yes Yes NO NO 45.5 54.5
5 The NGO estimates supply needs.*
DOES NOT APPLY Yes DOES NOT APPLY 100 -
6 The NGO has a stowage card to 
control supply. Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes NO 90.9 9.1
7 The NGO uses a stowage card to 
control supply. NO NO Yes Yes Yes NO NO Yes Yes NO NO 45.5 54.5
8 The NGO runs physical inventory on 
supply. NO NO Yes NO Yes Yes Yes Yes Yes NO NO 54.5 45.5
9 The NGO has sufficient space to 
store supplies. Yes Yes NO Yes Yes Yes Yes Yes Yes Yes NO 81.8 18.2
10 The NGO’s storage space is clean. Yes NO Yes Yes Yes Yes Yes Yes Yes NO Yes 81.8 18.2
11 The storage space has sufficient 
lighting. Yes Yes Yes Yes Yes Yes Yes Yes Yes NO Yes 90.9 9.1
12 The storage space is free of humidity. Yes Yes Yes Yes Yes Yes Yes Yes NO Yes Yes 90.9 9.1
13 The storage place has adequate 
ventilation. Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 100.0 -
14 The supplies are kept under lock and 
key. Yes NO NO Yes Yes NO NO Yes Yes Yes Yes 63.6 36.4
15 The stacking of supplies is adequate. Yes NO Yes Yes Yes Yes NO Yes Yes Yes Yes 81.8 18.2
16 The PEPE technique is applied. Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 100.0 -
17 Absence of harmful toxic substances. Yes Yes Yes Yes Yes Yes Yes Yes Yes NO Yes 90.9 9.1
18 Absence of expired supplies. Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 100.0 -
19 Supplies are labeled. Yes Yes Yes Yes Yes Yes Yes NO Yes Yes Yes 90.9 9.1
20 The stowage cards are updated. NO NO NO Yes Yes NO NO Yes Yes NO NO 36.4 63.6
21 Expiration dates are visible. Yes Yes Yes Yes Yes NO Yes Yes Yes Yes Yes 90.9 9.1
22 The NGO has refrigeration 
equipment Yes NO Yes NO Yes Yes NO Yes Yes Yes Yes 72.7 27.3
56
ITEMS TO VERIFY CEP
RESI
ASONVI
HSIDA
CEGO
DEM
ANICP+
VIDA
ANFAM/I
XCHEM
GA0 ADES
ENI
CEP
S
Fund. Sn 
LUCAS
ACCCSi
da
OVI % FULFILLMENT BY
ITEM
YES NO
23 The NGO has thermometers or 
thermohygrometers Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 100.0 -
24 The NGO uses thermometers or 
thermohygrometers. Yes Yes Yes Yes Yes Yes NO Yes Yes Yes Yes 90.9 9.1
25 NGO has an electronic Kardex. Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 100.0 -
26 NGO uses an electronic Kardex. Yes Yes Yes Yes Yes Yes NO Yes Yes Yes Yes 90.9 9.1
27 NGO uses supply reception forms Yes NO Yes Yes Yes NO Yes Yes Yes NO Yes 72.7 27.3
Average of global fulfillment based to 27 ítems 
82.7 
 
17.2