Midline Evaluation of the
Tanzania Public Sector
System Strengthening Program
Final Report
January 2020

Midline Evaluation of the 
Tanzania Public Sector 
System Strengthening Program
Final Report
Gustavo Angeles, PhD (Team Leader)
Jessica A. Fehringer, PhD, MHS
Lisa Calhoun, MPH
Brittany Iskarpatyoti, MPH
Willis Odek, PhD, MSc
Gideon Kwesigabo, MD, MSc, MEd, PhD
Joe L.P. Lugalla, PhD
Aimee M. Benson, MA
Hannah Silverstein, MPH
Courtney McGuire, MPH
January 2020
MEASURE Evaluation 
University of North Carolina at Chapel Hill
123 West Franklin Street, Suite 330
Chapel Hill, NC 27516 USA
Phone: +1 919-445-9350
measure@unc.edu
www.measureevaluation.org
This publication was produced with the support of the United States Agency 
for International Development (USAID) under the terms of MEASURE Evaluation 
cooperative agreement AID-OAA-L-14-00004. MEASURE Evaluation is 
implemented by the Carolina Population Center, University of North Carolina at 
Chapel Hill in partnership with ICF International; John Snow, Inc.; Management 
Sciences for Health; Palladium; and Tulane University. Views expressed are not 
necessarily those of USAID or the United States government. This report was 
prepared by Gustavo Angeles (team leader), Jessica A. Fehringer, 
Gideon Kwesigabo, Willis Odek, Lisa Calhoun, Hannah Silverstein, 
Brittany Iskarpatyoti, Courtney McGuire, and Joe L.P. Lugalla. TRE-19-26 
ISBN: 978-1-64232-214-9
2 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
ABSTRACT 
Th e Public Sector System Strengthening (PS3) project, funded by the United States Agency for International 
Development (USAID), supports the Government of Tanzania (GOT) in strengthening the public-sector 
system to promote the delivery, quality and use of public services. Th is report presents the fi ndings of a 
midline performance evaluation of PS3 conducted by MEASURE Evaluation which examined time trends 
and pre- and post-program inception changes in the uptake of health services as well as in fi nancial and 
human resources indicators in PS3 regions and in other regions of the country. Th e evaluation also used 
qualitative methods to examine the perceptions from program implementers, community members, and 
other stakeholders about the adoption and performance of the PS3 intervention, its strengths and remaining 
challenges, and the stakeholders’ recommendations on ways to address those challenges. 
PS3 works across national, regional, and local government levels. Th e implementation strategy seeks to 
strengthen each of the fi ve component areas, while working across all sectors and levels of public governance. 
Th e expectation is that strengthening systems at all levels will result in improved service delivery at the lowest 
level—the service provider—which will lead to higher public service utilization by the population and better 
health outcomes. At the national level, emphasis is placed on providing support for key policies and strategies 
as well as strengthening the national system to foster improvements in service delivery at lower government 
levels. At the regional and LGA levels, the PS3 program facilitates the understanding and implementation 
of national policies and guidelines as well as providing targeted assistance to improve the management and 
strengthening of systems and resources.
Th is midline evaluation undertook an outcome evaluation to understand how system-level indicators related 
to human resources, fi nancial resources, and health service utilization changed over time in PS3 and in 
other (non-PS3) regions of the country. Data from national routine information systems formed the basis for 
quantitative measurements. Adapted diff erence-in-diff erences models were used to examine time trends before 
PS3 and during the time PS3 has been active, changes in those trends, and to examine diff erences between PS3 
regions and other regions of the country. Potential diff erences between Phase 1 and Phase 2 PS3 LGAs were 
also examined. In addition, qualitative methods provided a more in-depth understanding of the performance 
of the PS3 intervention in human resources, fi nance, and system strengthening, captured unintended, indirect 
eff ects of the intervention, illuminated the process of change, and examined the quality and character of 
the intervention implementation. Th e qualitative component also captured the stakeholders’ perceptions of 
pending challenges in system functioning and community engagement and their recommendations on how to 
address them.
Th e fi ndings of this evaluation will contribute to USAID’s portfolio of projects focused on public-sector 
strengthening while informing PS3’s continued program implementation. Th is evaluation was performed at 
about the mid-point of PS3, three years after the project’s inception.
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 3
CONTENTS
ABSTRACT ............................................................................................................................................................ 2
ACKNOWLEDGMENTS ............................................................................................................................................ 8
ABBREVIATIONS ........................................................................................................................................................ 9
EXECUTIVE SUMMARY .........................................................................................................................................11
 Evaluation Purpose ..................................................................................................................................................11
 Project Background .................................................................................................................................................11
 Evaluation Questions ..............................................................................................................................................11
 Methods .......................................................................................................................................................... 12
 Findings .......................................................................................................................................................... 12
 Discussion .......................................................................................................................................................... 13
 Limitations ...........................................................................................................................................................18
1. INTRODUCTION ..................................................................................................................................................19
 1.1. Country Context ..............................................................................................................................................19
 1.2. Th e Development Problem: Health Outcomes ...............................................................................................19
 1.2.1. Infl uence of Devolution on the Tanzanian Health System .................................................................. 20
 1.3. USAID’s Response to the Need to Strengthen Health Systems in Tanzania ................................................ 20
 1.3.1. PS3 Project Overview ............................................................................................................................ 20
 1.3.2. Target Population and Geographic Areas ............................................................................................. 22
 1.3.3. PS3 Results Framework ........................................................................................................................ 23
 1.4. Evaluation of the PS3 Project .......................................................................................................................... 24
 1.4.1. Evaluation Objectives ........................................................................................................................... 24
 1.4.2. Evaluation Questions ........................................................................................................................... 24
2. EVALUATION METHODS .................................................................................................................................. 25
 2.1. Quantitative Evaluation Design ...................................................................................................................... 25
 2.1.1. Quantitative Analytical Approach ........................................................................................................ 25
 2.1.1.1. Examining Diff erences in Time Trends between PS3 Phase 1 and Phase 2 LGAs .................. 26
 2.1.2. Outcome Measures ............................................................................................................................... 27
 2.1.3. Description of Quantitative Data Sources and Data Collection .......................................................... 27
 2.2. Qualitative Study Design ................................................................................................................................ 29
 2.3. Ethical Considerations .................................................................................................................................... 30
3. FINDINGS ...........................................................................................................................................................31
 3.1. Quantitative Evaluation: Evaluation Questions 1 and 3 .................................................................................31
 3.2. Quantitative Evaluation: Evaluation Question 2 ........................................................................................... 49
 3.2.1. Financial Resources Indicators .............................................................................................................. 50
 3.2.2. Human Resources Indicators ............................................................................................................... 54
 3.3. Qualitative Results .......................................................................................................................................... 63
 3.3.1. Evaluation Question 4 ...........................................................................................................................64
 3.3.1.1. Community Engagement and Governance ................................................................................64
 3.3.1.2. Finance ....................................................................................................................................... 66
4 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
 3.3.1.3. Human Resources ....................................................................................................................... 68
 3.3.1.4. Information Systems ................................................................................................................... 69
 3.3.2. Evaluation Question 5 .......................................................................................................................... 70
 3.3.2.1. Community Engagement and Governance ............................................................................... 70
 3.3.2.2. Finance ...................................................................................................................................... 70
 3.3.2.3. Human Resources ...................................................................................................................... 71
 3.3.2.4. Information Systems ................................................................................................................... 72
 3.3.3. Evaluation Question 6 ......................................................................................................................... 72
 3.3.3.1. Community Engagement and Governance ............................................................................... 72
 3.3.3.2. Finance ....................................................................................................................................... 73
 3.3.3.3. Human Resources .......................................................................................................................74
 3.3.3.4. Information Systems ....................................................................................................................74
4. LIMITATIONS ........................................................................................................................................................ 76
 4.1. Limitations of the Quantitative Data .............................................................................................................. 76
 4.1.1. DHIS2 Data Strengths and Limitations ............................................................................................... 76
 4.2. Limitations of the Qualitative Data ................................................................................................................ 77
5. DISCUSSION AND PROGRAMMATIC RECOMMENDATIONS .............................................................. 78
 5.1. Summary of Results ........................................................................................................................................ 78
 5.1.1. Many Observed and Perceived Changes Occurred Nationally ............................................................ 84
 5.1.2. Diff erences in Programmatic Interventions Seen Primarily at Lowest Levels ..................................... 84
 5.1.3. Gender Parity in Secondary School Teacher Employment Declining ................................................. 85
 5.1.4. Programmatic Recommendations ........................................................................................................ 85
REFERENCES .......................................................................................................................................................... 87
APPENDIX A. PS3 IMPLEMENTATION REGIONS AND LGAS................................................................... 88
APPENDIX B. INDICATOR TIME TRENDS STRATIFIED BY PS3 IMPLEMENTATION PHASE .......91
APPENDIX C. INDICATOR DEFINITIONS .................................................................................................... 107
APPENDIX D. DATA SOURCES ...........................................................................................................................112
 Service Utilization Data Sources ...........................................................................................................................112
 DHIS2/HMIS ...............................................................................................................................................112
 Financial Data Sources ..........................................................................................................................................113
 Council Financial Reports (CFRs) ................................................................................................................113
 Human Resource Data Sources ............................................................................................................................113
 Human Resource for Health Information System (HRHIS) ......................................................................113
 Education Data Sources ........................................................................................................................................114
 Basic Education Statistics in Tanzania (BEST) .............................................................................................114
 OPENDATA .................................................................................................................................................115
 Basic Education Statistics in Tanzania (BEST) .............................................................................................115
APPENDIX E. SCOPE OF WORK .........................................................................................................................116
APPENDIX F. QUALITATIVE TOOLS ................................................................................................................141
APPENDIX G. CONFLICTS OF INTEREST..................................................................................................... 160
APPENDIX H. TEAM BIOS ....................................................................................................................................170
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 5
Figures
Figure 1. PS3 implementation strategy (source: PS3 year 2 workplan) ........................................................................ 20
Figure 2. PS3 implementation focus regions ..................................................................................................................21
Figure 3. PS3 results framework (source: PS3 year 2 workplan) ................................................................................... 24
Figure 4. Antenatal care coverage before 12 weeks gestational age ...............................................................................31
Figure 5. Pregnant women attending antenatal care 4+ times ...................................................................................... 32
Figure 6. Prevalence of pregnant women receiving iron and folic acid supplementation ............................................. 33
Figure 7. Pregnant women receiving TT2+ vaccination at antenatal care .................................................................... 34
Figure 8. Women tested for anemia at antenatal care ................................................................................................... 35
Figure 9. Women tested for syphilis at antenatal care ................................................................................................... 36
Figure 10. Deliveries taking place in health facilities .................................................................................................... 37
Figure 11. Births delivered by skilled attendants ........................................................................................................... 39
Figure 12. Mothers receiving postnatal care before 7 days ........................................................................................... 40
Figure 13. Prevalence of low birth weight ..................................................................................................................... 41
Figure 14. Measles vaccination coverage ....................................................................................................................... 42
Figure 15. Penta3 vaccination coverage ......................................................................................................................... 43
Figure 16. Antenatal care partners’ HIV testing rates ..................................................................................................44
Figure 17. HIV-exposed infants receiving fi rst HIV test within 2 months after birth ..................................................45
Figure 18. HIV-exposed infants initiated on cotrimoxazole within 2 months after birth ........................................... 46
Figure 19. Percentage of couples receiving HIV counseling and testing at antenatal care ........................................... 47
Figure 20. Contraceptive prevalence rate ...................................................................................................................... 48
Figure 21. Health facilities with RCH tracer drugs package ........................................................................................ 49
Figure 22. Personnel emoluments per capita in health ................................................................................................. 50
Figure 23. Personnel emoluments per capita in education .............................................................................................51
Figure 24. Other charges per capita in health ............................................................................................................... 52
Figure 25. Other charges per capita in education ......................................................................................................... 53
Figure 26. Nurses per population .................................................................................................................................. 54
Figure 27. Doctors per population .................................................................................................................................55
Figure 28. Assistant medical offi cers per 10,000 people ............................................................................................... 56
Figure 29. Health care workers that are female ............................................................................................................. 57
Figure 30. Primary school student-teacher ratio ........................................................................................................... 58
Figure 31. Secondary school student-teacher ratio ........................................................................................................ 59
Figure 32. Primary school students that are female ...................................................................................................... 60
Figure 33. Secondary school students that are female ....................................................................................................61
Figure 34. Primary school teachers that are female....................................................................................................... 62
Figure 35. Secondary school teachers that are female ................................................................................................... 63
Figure 36. Excerpt from REM Map with LGA councilors, Phase 2 – Kilosa ..............................................................65
Figure 37. Excerpt from REM map with LGA councilors, Phase 1 – Shinyanga ....................................................... 67
Figure B.1. ANC started before 12 weeks of gestation ..................................................................................................91
Figure B.2. ANC, at least 4 antenatal care visits ............................................................................................................91
Figure B.3. Iron and folic acid supplementation in pregnancy ..................................................................................... 92
Figure B.4. Pregnant women receiving TT2+ .............................................................................................................. 92
Figure B.5. Proportion of pregnant women tested for anemia ...................................................................................... 93
Figure B.6. Pregnant women tested for syphilis ............................................................................................................ 93
6 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure B.7. Birth deliveries in health facilities ............................................................................................................... 94
Figure B.8. Births delivered by skilled attendants ......................................................................................................... 94
Figure B.9. Postnatal care within 7 days of delivery, for mothers ................................................................................. 95
Figure B.10. Prevalence of low birth weight .................................................................................................................. 95
Figure B.11. Measles vaccination, inside service area .................................................................................................... 96
Figure B.12. Penta3 vaccination .................................................................................................................................... 96
Figure B.13. ANC partners’ HIV testing rate ............................................................................................................... 97
Figure B.14. Percentage of HIV-exposed infants receiving fi rst HIV test within 2 months after birth ...................... 97
Figure B.15. Percentage of HIV-exposed infants initiated on cotrimoxazole within 2 months after birth ................. 98
Figure B.16. Percentage of couples HIV counseling and testing at ANC .................................................................... 98
Figure B.17. Contraceptive prevalence rate .................................................................................................................... 99
Figure B.18. Health facilities with RCH tracer drugs package ..................................................................................... 99
Figure B.19. PE per capita in health ............................................................................................................................ 100
Figure B.20. PE per capita in education ...................................................................................................................... 100
Figure B.21. OC per capita in health ...........................................................................................................................101
Figure B.22. OC per capita in education .....................................................................................................................101
Figure B.23. Nurses per population, per 10,000 people ............................................................................................. 102
Figure B.24. Doctors per population, per 10,000 people ........................................................................................... 102
Figure B.25. Assistant medical offi cers per 10,000 people .......................................................................................... 103
Figure B.26. Proportion of health care workers that are female ................................................................................. 103
Figure B.27. Primary school student-teacher ratio ...................................................................................................... 104
Figure B.28. Secondary school student-teacher ratio .................................................................................................. 104
Figure B.29. Percentage of primary school students that are female .......................................................................... 105
Figure B.30. Percentage of secondary school students that are female ....................................................................... 105
Figure B.31. Percentage of primary school teachers that are female ........................................................................... 106
Figure B.32. Percentage of secondary school teachers that are female ........................................................................ 106
Tables
Table E.1. Summary of time trends and annualized changes in indicators...................................................................15
Table 1. Phase 1 LGAs ................................................................................................................................................... 23
Table 2. List of indicators .............................................................................................................................................. 28
Table 3. LGAs selected for qualitative sample ............................................................................................................... 29
Table 4. Sampling results for key informant interviews ................................................................................................ 30
Table 5. Annualized average change in the percentage of pregnant women starting antenatal care before 12 
weeks of gestation (in percentage points) .......................................................................................................................31
Table 6. Annualized average change in the percentage of pregnant women with at least four antenatal care visits
(in percentage points) ..................................................................................................................................................... 32
Table 7. Annualized average change in percentage of pregnant women given iron and folic acid supplementation
(in percentage points) ..................................................................................................................................................... 33
Table 8. Annualized average change in percentage of pregnant women receiving TT2+ vaccine 
(in percentage points) .................................................................................................................................................... 35
Table 9. Annualized average change in the percentage of pregnant women tested for anemia at antenatal care 
(in percentage points) ..................................................................................................................................................... 36
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 7
Table 10. Annualized average change in percentage of pregnant women tested for syphilis at antenatal care 
(in percentage points) ..................................................................................................................................................... 37
Table 11. Annualized average change in the percentage of deliveries occurring in health facilities 
(in percentage points) ..................................................................................................................................................... 38
Table 12. Annualized average change in the percentage of births delivered by skilled birth attendants 
(in percentage points) ..................................................................................................................................................... 39
Table 13. Annualized average change in the percentage of mothers who received postnatal care within 7 days 
of delivery (in percentage points) ................................................................................................................................... 40
Table 14. Annualized average change in percentage of newborns with low birth weights (in percentage points) ....... 41
Table 15. Annualized average change in the percentage of children under one year of age who have been 
vaccinated against measles (in percentage points) ......................................................................................................... 42
Table 16. Annualized average change in the percentage of children under one year of age who have had 
Penta3 vaccinations (in percentage points) .................................................................................................................... 43
Table 17. Annualized average change in antenatal partners’ HIV testing rates (in percentage points) ........................44
Table 18. Annualized average change in percentage of HIV-exposed infants receiving fi rst HIV test within 
2 months after birth (in percentage points) ....................................................................................................................45
Table 19. Annualized average change in the percent of HIV-exposed infants initiated on cotrimoxazole 
within 2 months after birth (in percentage points) ....................................................................................................... 46
Table 20. Annualized average change in percentage of couples receiving HIV counseling and testing 
at antenatal care (in percentage points) ......................................................................................................................... 47
Table 21. Annualized average change in contraceptive prevalence rate (in percentage points) .................................... 48
Table 22. Annualized average change in the percentage of health facilities with RCH tracer drugs package 
(in percentage points) ..................................................................................................................................................... 49
Table 23. Average annual change in personnel emoluments per capita in health, in TZ shillings per person ............ 50
Table 24. Average annual change in personnel emoluments per capita in education, in TZ Shillings per person .......51
Table 25. Average annual change in other charges per capita in health, in TZ Shillings per person ........................... 52
Table 26. Average annual change in other charges per capita in education, in TZ Shillings per person ..................... 53
Table 27. Average annual change in the nurses per population ratio (in nurses per 10,000 people) ............................ 54
Table 28. Average annual change in the doctors per population ratio (in doctors per 10,000 people) .........................55
Table 29. Average annual change in assistant medical offi cers per population ratio (per 10,000 people) .................... 56
Table 30. Average annual change in the proportion of health care workers that are female (in percentage points) .... 57
Table 31. Average annual change in the primary school student-teacher ratio (in students per teacher) ..................... 58
Table 32. Average annual change in the secondary student-teacher ratio (in students per teacher) ............................. 59
Table 33. Average annual change in the proportion of primary school students that are female, 
in percentage points. ...................................................................................................................................................... 60
Table 34. Average annual change in proportion of secondary school students that are female, in percentage points ..61
Table 35. Average annual change in proportion of primary school teachers that are female, in percentage points. .... 62
Table 36. Average annual change in proportion of secondary school teachers that are female, in percentage points ...63
Table 37. Integration between system and type of data transferred/exchanged ........................................................... 75
Table 38. Summary of time trend and annualized changes in indicators .................................................................... 79
Table 39. Summary of annualized changes in indicators and diff erences between PS3 regions and other areas ........ 82
Table C.1. Selected health service utilization indicators for routinely collected data – PS3 evaluation ..................... 107
8 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
ACKNOWLEDGMENTS 
Th e authors wish to acknowledge the eff orts and contributions of numerous people who supported this 
evaluation and report. We would like to thank the United States Agency for International Development 
(USAID)/Tanzania for entrusting us with this activity and providing the technical support and fi nancial 
resources that made it possible. We are very grateful for the assistance and time provided by Mission offi cials, 
in particular, Elizabeth Williams, Godfrey Nyombi, Todd Koppenhaver, Nathan Tenny, Milly Kayongo, 
Erasmo Malekela, Aurora Amoah, and Gene Peuse, who currently work at the USAID Mission, as well 
as Shannon Young and Jonathan Young, who were at the Mission during the fi rst years of the evaluation 
and signifi cantly contributed to its design. We also thank Kristen Wares and our colleagues at USAID/
Washington, DC, who supported and provided management guidance to the evaluation. 
Additionally, we would like to acknowledge the Public Sector System Strengthening project’s leadership 
and staff who provided substantial support and input to the evaluation. In particular, we thank Emmanuel 
Malangalila, Sheila O’Dougherty, Desderi Wengaa, Gemini Mtei, Peter Kilima, Conrad Mbuya, and Leah 
Mwainyekule for their contributions. We also recognize and appreciate the support provided by staff at the 
Ministry of Health, Community Development, Gender, Elderly and Children; the Ministry of Education 
and Vocational Training; and at the President’s Offi ce Regional Administration and Local Government 
offi ces (PO-RALG). At MEASURE Evaluation, we thank Ann Marie Fitzgerald, Erin Luben, and Bates 
Buckner for their valuable recommendations, in addition to their technical and operational support. 
We thank the knowledge management team of the USAID-funded MEASURE Evaluation project for 
editorial, design, and production services.
Cover photo: A woman fi nishes drying cassava before going home in Mbamba Bay, Tanzania, the natural 
border between Tanzania and Malawi. © 2012 Riccardo Zennaro, courtesy of Photoshare.
Suggested citation:
MEASURE Evaluation. (2019). Midline Evaluation of the Tanzania Public Sector System Strengthening 
Program: Final Report. Chapel Hill, NC, USA: MEASURE Evaluation, University of North Carolina.
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 9
ABBREVIATIONS
AHSP annual health sector performance profi le 
AHSPR annual health sector performance profi le report 
AMO assistant medical offi cer
ANC antenatal care
BEMIS Basic Education Management Information System
BEST Basic Education Statistics in Tanzania
CFR council fi nancial report 
CO clinical offi cer
COSTECH Tanzania Commission for Science and Technology
CPR contraceptive prevalence rate 
D-by-D “Decentralization by Devolution” policy
DED district executive director 
DFF Direct Facility Financing
DHIS District Health Information Software 
DHRO district health resources offi cer
DMO district medical offi cer
DPLO district planning offi cer
FFARS Facility Financial Accounting and Reporting Systems 
FGD focus group discussion
GOT Government of Tanzania
GOTHOMIS Government of Tanzania Hospital Management Information Systems 
HCMIS human capital management information system
HCW health care worker 
HMIS health management information system 
HR human resources 
HRHIS Human Resources for Health Information System
HSS health systems strengthening
HSSP Health Sector Strategic Plan
ICT information and communications technology
IFA iron and folic acid 
IRB Institutional Review Board
IS information systems 
KII key informant interview
LBW low birth weight
LGA local government authority
LGRCIS local government revenue collection information system
M&E monitoring and evaluation
MOHCDGEC Ministry of Health, Community Development, Gender, Elderly and Children 
10 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
NBS National Bureau of Statistics
O&OD Opportunities and Obstacles to Development
OC other charges
OPRAS Open Performance Review and Appraisal system 
PE personal emoluments 
PMP Performance Management Plan
PNC postnatal care 
POA Prioritization and Optimization Analysis 
POPSM President’s Offi ce Public Sector Management
PO-RALG President’s Offi ce, Regional Administration and Local Government offi ces
PS3 Public Sector System Strengthening project
RBF results-based fi nancing
RCH reproductive and child health
REM ripple eff ects mapping
RHIS routine health information systems 
SBA skilled birth attendant 
SOW scope of work
TAMISEMI Tanzania National Bureau of Statistics and the Presidents’ Offi ce Regional
 Administration and Local Government 
TDHS Tanzania Demographic Health Survey 
TFR total fertility rate
TT2+ tetanus toxin vaccine 
UNC University of North Carolina
USAID Th e United States Agency for International Development
WISN Workload Indicators of Staffi ng Need
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 11
EXECUTIVE SUMMARY 
Evaluation Purpose
Th e Public Sector System Strengthening (PS3) project, funded by the United States Agency for International 
Development (USAID), supports the Government of Tanzania (GOT) in strengthening the public-sector 
system to promote the delivery, quality and use of public services. Th is report presents the fi ndings of a 
midline performance evaluation of PS3 conducted by MEASURE Evaluation which examined time trends 
and pre- and post-program inception changes in the uptake of health services as well as in fi nancial and 
human resources indicators in PS3 regions and in other regions of the country. Th e evaluation also used 
qualitative methods to examine the perceptions from program implementers, community members, and 
other stakeholders about the adoption and performance of the PS3 intervention, its strengths and remaining 
challenges, and the stakeholders’ recommendations on ways to address those challenges. Th e fi ndings of this 
evaluation will contribute to USAID’s portfolio of projects focused on public-sector strengthening while 
informing PS3’s continued program implementation. Th is evaluation was performed at about the mid-point of 
PS3, three years after the project’s inception.
Project Background
In July 2015, USAID awarded the PS3 project in Tanzania to Abt Associates Inc. Th e project has an emphasis 
on system strengthening and improving quality of services for priority health areas (e.g., maternal and child 
health, HIV/AIDS, TB, malaria) as well as improving key multisectoral components at the local government 
level related to education, agriculture, and infrastructure. Specifi cally, the PS3 project aims to improve public 
services delivery and outcomes by strengthening national, district, and local government authority (LGA) 
capacity in the following areas: governance and citizen engagement, human resources, fi nancial management 
and information systems, and operations research. 
PS3 works across national, regional, and local government levels. Th e implementation strategy seeks to 
strengthen each of the fi ve component areas, while working across all sectors and levels of public governance. 
Th e expectation is that strengthening systems at all levels will result in improved service delivery at the lowest 
level—the service provider—which will lead to higher public service utilization by the population and better 
health outcomes. At the national level, emphasis is placed on providing support for key policies and strategies 
as well as strengthening the national system to foster improvements in service delivery at lower government 
levels. At the regional and LGA levels, the PS3 program facilitates the understanding and implementation 
of national policies and guidelines as well as providing targeted assistance to improve the management and 
strengthening of systems and resources.
Evaluation Questions
Th e midline evaluation was designed to answer the following evaluation questions:
1. In PS3 focus regions, to what extent have the uptake of health services (and related indicators of service 
utilization, coverage of health services, quality of services and access) changed over time?
2. In PS3 focus regions, to what extent have fi nancial and human resources indicators changed over time?
3. Are there diff erences in the changes over time observed in certain groups of LGAs, in particular, 
between Phase 1 and Phase 2 LGAs?
4. How do stakeholders perceive the performance and infl uence of PS3 on uptake of health services, 
fi nancial and human resource systems, and community engagement and governance? 
5. In PS3 focus regions, what was the context of any observed or perceived change in uptake of health 
services, fi nancial and human resource systems, and community engagement and governance?
6. What challenges remain in system functioning and community engagement and governance and how 
do stakeholders and community members recommend these be addressed?
12 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Methods
Th is midline evaluation undertook an outcome evaluation to understand how system-level indicators related 
to human resources, fi nancial resources, and health service utilization changed over time in PS3 and in 
other (non-PS3) regions of the country. Data from national routine information systems formed the basis for 
quantitative measurements. Adapted diff erence-in-diff erences models were used to examine time trends before 
PS3 and during the time PS3 has been active, changes in those trends, and to examine diff erences between PS3 
regions and other regions of the country. Potential diff erences between Phase 1 and Phase 2 PS3 LGAs were 
also examined. In addition, qualitative methods provided a more in-depth understanding of the performance 
of the PS3 intervention in human resources, fi nance, and system strengthening, captured unintended, indirect 
eff ects of the intervention, illuminated the process of change, and examined the quality and character of 
the intervention implementation. Th e qualitative component also captured the stakeholders’ perceptions of 
pending challenges in system functioning and community engagement and their recommendations on how to 
address them.
Findings
Q1: In PS3 focus regions, to what extent has the uptake of health services changed over time? And, 
Q3: Are there diff erences in the changes over time observed in certain groups of LGAs, in particular, 
between Phase 1 and Phase 2 LGAs?
We examined quarterly LGA-level time series data for 18 indicators of service utilization in maternal and 
child health, family planning, HIV/AIDs, and drug availability. Data was obtained from District Health 
Information Software version 2 (DHIS2) from the fi rst quarter of 2014 to the last quarter of 2018. For most 
indicators we found positive time trends in PS3 regions throughout the observation period. Antenatal care 
(ANC) started at a low level before 2015 and changed to an increasing trend after 2015. Facility birth delivery 
and skilled birth delivery attendance had signifi cant improvements since 2014; postnatal care had signifi cant 
increases after 2015 in PS3 areas as well. Supply-related indicators such as iron and folic acid supplements and 
vaccines showed a large drop in 2016, followed by a quick recovery in 2017. However, for most indicators, the 
time trends observed in PS3 regions were also observed in the other (non-PS3) regions. Th e trends were similar 
with very few indicators with signifi cant diff erences between PS3 and other areas. Even HIV/AIDS-related 
indicators showed a strong national trend that is shared by PS3 and other areas. Th ere was also no distinction 
in the time trends and changes between Phase 1 and Phase 2 LGAs for most indicators.
Q2: In PS3 focus regions, to what extent have fi nancial and human resources indicators changed over 
time? And, Q3: Are there diff erences in the changes over time observed in certain groups of LGAs, in 
particular, between Phase 1 and Phase 2 LGAs?
We examined annual (fi scal year) LGA-level time series for four indicators of fi nancial resources and yearly 
LGA-level time series data for 10 indicators on human resources. Personnel emoluments (PE) per capita in 
health and in education increased through the observation period, from fi scal year 2012/2013 to fi scal year 
2017/2018, in PS3 regions. However, these levels and time trends were similar to other (non-PS3) regions. Th ere 
were almost no diff erences in time trends between the two types of regions. A similar pattern was observed for 
other charges (OC) per capita in health and education. OC per capita declined before 2015 in both sectors but 
it held at a stable level after 2015. Similar patterns were observed for Phase 1 and Phase 2 LGAs.
For human resources, there were similar increases in the ratios of nurses per 10,000 people and doctors per 
10,000 people since 2012 in both PS3 and other regions. Th e ratio of assistant medical offi cers per 10,000 
people was stable during the observation period in both regions too. No diff erences were observed between Phase 
1 and Phase 2 LGAs. In education, the primary school student-teacher ratio was declining before 2015, but 
afterwards changed to an increasing time trend in both PS3 and other regions. Th e secondary student-teacher 
ratio showed a signifi cant declining trend from 2012 to 2016. It started to increase in 2017 and 2018, perhaps 
due to the certifi cation verifi cation process started by the government. Th ere are no diff erences in the trends and 
levels between PS3 and other regions. In terms of gender balance, the proportion of primary school students 
that are female has been stable at around 50 percent throughout the observation period, from 2012 to 2018. In 
secondary school there has been a clear positive national trend of increasing participation of girls in secondary 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 13
school. By 2018, gender balance has been achieved among students of secondary schools. For teachers, there 
is gender balance in primary schools in both types of regions. However, the proportion of secondary school 
teachers that are female has been under 40 percent, but with an increasing trend until 2016 when it became 
stable. A similar pattern emerges in PS3 and other regions as well as in Phase 1 and Phase 2 LGAs.
Q4: How do stakeholders perceive the performance and infl uence of PS3 on health services, fi nancial 
and human resource systems, and community engagement and governance?
Across PS3 and other regions, stakeholders perceived improvements in governance and civic engagement, 
fi nance, and human resources, as well as the information systems that support these components. Community 
engagement and local governance were identifi ed by stakeholders, particularly LGA councilors, as the most 
signifi cant changes. PS3 area councilors detailed greater capacity to engage citizens in addressing issues as part 
of their roles and responsibilities. Community members discussed a perceived increase in local accountability 
and transparency. Supportive training and information systems, such as those put in place by PS3 and GOT, 
were noted as important in these positive changes. However, stakeholders noted there are challenges that 
infl uence the public sector performance, such as fi nance and human resource shortages.
Q5: In PS3 focus regions, what was the context of any observed or perceived change?
Perceived and observed changes were discussed within the context of larger social and political environments. 
For example, the Opportunities and Obstacles to Development (O&OD) is an intensive participatory process 
used in both rural and urban areas as a bottom-up planning approach. Th is process has, for the most part, 
catalyzed engagement between citizens and local governments even before PS3 program implementation. 
However, there were claims that the planned budget was not always met and sweeping national policies are 
diffi cult to carry out. Additionally, human resource policies to verify public sector employee’s certifi cations 
has led to sweeping human resource shortages, particularly in health. In education, free education policies 
have increased the number of students enrolled in school, but according to stakeholders this has resulted in 
overcrowded classrooms. Other contextual limitations include the poor/lack of availability of internet services 
and electricity services as well as limited capacity to use computer-based information systems. 
Q6: What challenges remain in system functioning and community engagement and governance and 
how do stakeholders and community members recommend these be addressed?
Many of the challenges that remain are closely tied to the supportive context. Although respondents noted 
improvements in general for health and education systems, limited funds, staffi ng shortages and poor job 
performance concerns remained at the service delivery level. Community participants commonly noted 
concerns with stockouts, health service delays, too few staff at health facilities, staff not working, and too few 
teachers, along with growing numbers of students. While many recommendations fall beyond the scope of 
the PS3 program, it was recommended to address the disconnect between central and local governments and 
better align priorities and decision-making. Th ere is also a need for capacity building in numerous groups 
on issues related to management of funds, generation of capital and profi t, and record keeping. Additional 
and expanded trainings, as well as in-service support, were recommended to ensure optimal uptake and 
functioning of positively perceived systems. 
Discussion
Table E.1 summarizes the quantitative evaluation results, describing the types of trends and estimated time 
trend diff erences. Columns under the heading “PS3 areas’ general trend” depict the overall directionality of 
changes over time in PS3 areas. We observed improving trends in PS3 areas for 17 of the 32 indicators, most 
of which pertained to health service utilization. For 12 indicators, trends did not appear to generally improve 
or worsen over time—seven of these indicators had stable trends and fi ve had volatile trends, meaning that 
we observed drastic variation in the levels of indicators at diff erent time points. Indicators with volatile trends 
were supply-side sensitive, and thus likely refl ect changes in supply conditions. Finally, three of the indicators 
worsened over the observed time period, pertaining to fi nance and human resource categories. It is important 
to note, that general patterns between PS3 and other areas were similar for most indicators, despite often 
having diff erent levels.
14 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Also in Table E.1, columns under the heading “PS3 compared with other areas” qualitatively refl ect the 
diff erences in time trends between PS3 and other areas, comparing the diff erences between the before PS3 and 
PS3 active time periods. Estimates were signifi cant for eight indicators. For three of these indicators, PS3 areas 
improved signifi cantly more than other areas between both periods, all of which pertained to maternal health 
service utilization. For another four indicators, we observed signifi cant eff ects that were protective. “Protective 
eff ects” were indicators that had worsening trends in both areas, but PS3 regions worsened at slower rates 
than other areas. Two of the indicators with signifi cant protective eff ects pertained to the prevention of HIV 
transmission from mother to child. Finally, there was one indicator, the percentage of primary teachers that 
were female, that was worse in PS3 areas. However, this result should be taken cautiously as the levels of this 
indicator have been stable, at just below 50 percent in PS3 regions in the PS3 active period and just above 50 
percent in other regions, which would be considered the level of gender parity. 
Many observed and perceived changes occurred nationally. Generally, trends and perceptions show gradual 
improvements nationally, with few diff erences between PS3 and other regions, or between PS3 phases. Th is 
could be a result of various factors. First, many of the PS3 systems were adopted by GOT and rolled￾out nationally. Th is points to a gap that was fi lled early on, but it makes fi nding diff erences in PS3 regions 
diffi cult. In both PS3 and other areas, we found improvements in health outcomes, particularly among health 
center access and attendance-based services, such as ANC, ANC4+, and facility birth deliveries. Qualitative 
results suggest these improvements may have been related to better understanding the importance of visiting 
health centers, as well as improvements in supportive infrastructure such as roads and the building of health 
centers. As such, it is important to note that PS3 is one of several programs contributing to public sector 
development. Th is includes, as noted by qualitative participants, other USAID-funded programs such as 
Boresha Afya, which supports integrated service delivery at the health facility and community levels across 
the country. World Vision and Amref were discussed and credited for improving awareness and education on 
health services, which help to draw patients to health centers.
15
Table E.1. Summary of time trends and annualized changes in indicators
Indicators
Signi
fi cant
PS3 areas’ general trend
(based on graphs)
PS3 compared with other areas
(based on DID estimates)
Improving trend 
About the same 
or stable trend
Volatile trend
Worsening trend 
Signi
fi cantly 
better in PS3 
areas
Better, but no 
signi
fi cant 
difference
Signi
fi cantly 
protective in 
PS3 areas
Protective, but 
no signi
fi cant 
difference
Worse, but 
no signi
fi cant 
difference
Signi
fi cantly 
worse in PS3 
areas
 ANC coverage before 12 weeks gestational age xx
 Pregnant women attending ANC 4+ times xx
 Prevalence of pregnant women receiving IFA 
supplementation for 90+ days x x
 Pregnant women receiving TT2+ at ANC xx
 Women tested for anemia at ANC x x x
 Women tested for syphilis at ANC x x x
 Deliveries taking place in health facilities xx
 Births assisted by skilled attendants x x
 Mothers receiving postnatal care before 7 days x x x
 Prevalence of low birth weight xx
 Measles vaccination coverage x x
 Penta3 vaccination coverage x x
 ANC partners HIV testing rate xx
 HIV-exposed infants receiving first HIV test within 2 
months after birth x x x
 HIV-exposed infants initiated on cotrimoxazole within 2 
months after birth x x x
 Couple HIV counseling and testing at ANC x x
 Contraceptive prevalence rate x x
 Health facilities with RCH tracer drugs package x x
16
Indicators
Signi
fi cant
PS3 areas’ general trend
(based on graphs)
PS3 compared with other areas
(based on DID estimates)
Improving trend 
About the same 
or stable trend
Volatile trend
Worsening trend 
Signi
fi cantly 
better in PS3 
areas
Better, but no 
signi
fi cant 
difference
Signi
fi cantly 
protective in 
PS3 areas
Protective, but 
no signi
fi cant 
difference
Worse, but 
no signi
fi cant 
difference
Signi
fi cantly 
worse in PS3 
areas
 PE per capita in health xx
 PE per capita in education xx
 OC per capita in health xx
 OC per capita in education x xx
 Nurses per population xx
 Doctors per population xx
 AMOs per population x x x
 Health care workers that are female xx
 Primary school student-teacher ratio xx
 Secondary school student-teacher ratio x x
 Primary school students that are female xx
 Secondary school students that are female xx
 Primary school teachers that are female x x X^
 Secondary school teachers that are female x x
Total 8 17 7 5 3 3 7 4 6 11 1
Total signifi cant and favorable 7
Note: None of the indicators were improving in PS3 areas only or worsening in PS3 areas only. 
^: Results of this indicator should be interpreted with caution.
Table E.1. Summary of time trends and annualized changes in indicators (continued)
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 17
Broader policy may have heavily infl uenced observed and perceived changes: In 2015, the Tanzanian 
government established policies that directed public bodies to ensure education is free for all children. Th is 
was met with another policy to ensure staff are properly qualifi ed and certifi ed. As such, we found increases 
in student-teacher ratio trends in both primary and secondary education. Similarly, in health, stakeholders 
commonly discussed a shortage of health care workers. Th ese issues infl uenced community members’ 
perceptions of public sector performance. Despite these national issues, stakeholders perceived systems put in 
place by PS3 and GOT as valuable in creating effi ciencies and improving public sector system performance. 
However, these systems are relatively new, and this perception may be compromised by long-term stagnation or 
issues around commodities. As such, the program may require more time to realize changes in outcomes 
beyond those seen in this midline evaluation. As this is a systems intervention that addresses underlying facets 
of the public sector, it is anticipated that observed changes may still yet be seen. 
Diff erences in programmatic intervention seen primarily at lowest levels: Where PS3 and other areas 
diff ered were often at the lowest levels of intervention. For example, PS3 stakeholders, particularly LGA 
councilors, identifi ed local community engagement and good governance as substantial changes coming 
out of PS3 programs. LGA councilors discussed having the knowledge and skills to attend to community 
members’ needs and complaints. Th ey also discussed improvements in transparency and accountability at the 
local level. 
However, local gains may encounter challenges as they interact with regional and central priorities. A commonly 
noted issue in creating development plans at the local level was the availability of fi nancial and human resources 
to carry out these plans. In response, LGAs developed plans and submitted them, but often learned that the 
central government had diff erent funding priorities. PS3 stakeholders also discussed trainings on information 
systems as important in highlighting the value of these systems and encouraging their use.
Based on the fi ndings, we provide the following recommendations for PS3:
Information systems
 Support internet service nationally or work with central/local government to plan for this in budgets and 
ensure staff are not paying for internet service out of personal funds
 Investigate concerns with Workload Indicators of Staffi ng Need (WISN) staffi ng categories and 
accuracy of input data
 Assess whether additional categories of staff may benefi t from access to Facility Financial Accounting 
and Reporting Systems (FFARS)
 Continue to work towards interoperability of information systems
 Institute a mechanism to regularly communicate system updates/new features to users
Training
 Carry out needs assessment at facilities prior to information system install, ensuring staff have capacity 
to use and maintain information systems; then train as needed
 Ensure supervisors are invited or educated on why their subordinates are invited to trainings; since 
it may not be feasible or necessary to train them to the same depth as their subordinates, PS3 might 
consider developing a shorter, awareness-raising session aimed at this audience so that they at least feel 
engaged and understand the value of the systems
 Consider establishing refresher trainings
 Maintain and expand in-service support
 Integrate how to plan for growing populations into the LGA councilor budget and development training
 Continue good governance work but increase transparency (e.g., with suggestion boxes)
18 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Citizen engagement and governance
 Address disconnect between central and local government in planning and budgeting
o In trainings, set expectations on what to expect from central government with regard to budgeting 
and planning (i.e., they may not receive all requested funds), and share messages to communicate 
with citizens so as to similarly set their expectations
o Increase understanding at LGA level of how central government is making funding allocation 
decisions
o Increased work with central government to encourage better communication (for example, on 
funding priorities and timelines) and collaboration with LGAs
 Increase publicity for new accountability mechanisms and information sources like the new websites, 
health clinic comment boxes, etc.; for example, participants suggested making TV and radio 
announcements and posting notices at health centers and schools.
Gender
 Address secondary school teacher gender parity issues in human resources eff orts; for example, 
investigate potential barriers to recruiting and maintaining secondary school teachers that are female, 
and assess whether gender sensitivity training for teachers, students, and administrators may be needed 
to create environments where women feel respected and safe
Limitations 
MEASURE Evaluation utilized data from routine information systems to examine how key indicators of 
health service uptake, fi nancial resources, and human resources changed over time before and after the 
implementation of PS3. Data sources, including DHIS2, have common challenges that can generally be 
described as: technical, behavioral, or organizational. Technical challenges include burden of data collection 
at the point of care, data management challenges, and incomplete or inaccurate data. Behavioral challenges 
include limited knowledge, lack of value for data, and limited guidance on how to collect and use data. 
Organizational challenges include staffi ng shortages, stockout of standard registrars, and lack of supportive 
supervision. Th ese challenges contribute to data quality problems. 
Qualitative limitations include the possibility that participants did not feel comfortable freely sharing 
their perspectives on a project that is closely aligned with the government, especially if they worked for the 
government in some capacity. Additionally, the qualitative data was not collected in all PS3 areas so the results 
might not refl ect the experiences of those clusters not sampled. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 19
1. INTRODUCTION
Th is document is the report of the midline performance evaluation of the Public Sector System Strengthening 
project (PS3) in Tanzania. It presents fi ndings from the time trends analysis of key service utilization, fi nancial 
and human resources indicators, and the qualitative analysis that were used to answer six evaluations questions 
posed by USAID/Tanzania about PS3. Th is report presents fi ndings as of about the mid-point of the life of the 
PS3, after three years of project inception. 
1.1. Country Context
Tanzania has made important progress in recent years. Th ere has been rapid economic growth of over 7 
percent per year, and signifi cant improvements have been made on some key health indicators. However, 
challenges remain. Tanzania has one of the most rapidly growing populations globally at 2.7 percent per 
year, with the urban population growing approximately 5 percent per year (Economic and Social Research 
Foundation, et al., 2015). Tanzania also has poor living standards, particularly in rural areas, with national 
poverty and extreme poverty levels at 28.2 percent and 9.7 percent, respectively (Economic and Social 
Research Foundation, et al., 2015). Th e Government of Tanzania (GOT) has set an ambitious target to 
reduce poverty signifi cantly and become a middle-income country by 2025. However, to achieve this goal 
Tanzania will have to tackle numerous pressing health issues in its growing population.
1.2. The Development Problem: Health Outcomes
Prior to the start of PS3, Tanzania has made signifi cant headway in various health outcomes. For instance, 
according to the 2015/2016 Tanzania DHS (TDHS), 98 percent of pregnant women attended at least one 
antenatal visit with a skilled provider, the percentage of pregnant women attending at least four prenatal visits 
(ANC4) increased from 43 percent to 51 percent, and the percentage of women receiving timely postnatal 
care (PNC) increased from about 30 percent to 34 percent (Ministry of Health, et al., 2016; National Bureau 
of Statistics (Tanzania), & ICF Macro, 2011). However, challenges remain. Th e 2015/2016 TDHS estimated 
maternal mortality at 556 maternal deaths per 100,000 live births, which was not signifi cantly diff erent from 
2004/2005 TDHS estimates (Ministry of Health, et al., 2016). Th ere is evidence that not enough women 
received recommended levels of care despite general improvements in perinatal care between 2010–2016. 
Despite the improvements in utilization of ANC4 and PNC, adequate prenatal and postnatal coverage was 
still low. Th ese low levels may be due the challenges accessing health facilities: 65.5 percent of women reported 
having problems accessing health care in 2015/2016, which had increased from 35.5 percent in 2010. Th e most 
commonly reported challenges included distance to health facility (42.3%) and having enough money for 
treatment (49.5%) Such barriers may explain under-utilization of maternal health services (UNESCO Institute 
for Statistics, 2019). 
Among other achievements, Tanzania was one of only a handful of African countries to achieve the MDG 4 
objective on child survival. Comparing estimates between the 2004/2005 and 2015/2016 TDHS, the under-5 
mortality rate declined from 112 to 67 per 1,000 live births. Th ere were still areas of child health with potential 
for improvement before PS3 began. Th e percentage of children with diarrhea receiving treatment decreased 
from 62.2 percent at its peak in 2004/2005, to 59.1 percent in 2010, and then to 49 percent in 2015/2016. 
While the proportion of children who were stunted and underweight decreased from 2004/2005, over one￾third of children were stunted in 2015/2016. Similarly, the prevalence of anemia among children remained 
steadily high, at around 58 percent since 2004/2005 (UNESCO Institute for Statistics, 2019). 
Preceding PS3 implementation, net primary school enrollment also declined from 79.7 percent in 2010, to 
75.6 percent in 2015. About 78 percent of primary-age females were enrolled in school in 2015/2016, which 
was a slight decline from 81.2 percent in 2010. Female secondary school attendance remained low at about 29 
percent in both 2010 and 2015/2016 (UNESCO Institute for Statistics, 2019).
20 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
1.2.1. Infl uence of Devolution on the Tanzanian Health System
Th e “Decentralization by Devolution” policy (D-by-D) was passed under Tanzania’s Local Government 
Reform Act in 1996, initiating the decentralization of public sector planning and service delivery from the 
central government to the local governance system. Th e aim of the D-by-D policy was to bring decision￾making for resource allocation and services to the local level so that local needs were better addressed, and 
local people could actively participate in the decision-making process (Masson, L. & Norman, A.S., 2009). 
However, during the time prior to PS3, the D-by-D policy resulted in many challenges at the local level, 
stemming from ineffi ciencies in service delivery related to inadequate fi nancing and fi nancial management 
skills, poor coordination, communication, and unclear scopes of work (SOWs) for district stakeholders. 
Th ese ineffi ciencies extended across all aspects of local governance, such as health service delivery, 
agriculture, infrastructure, and education. Th erefore, they needed to be addressed if Tanzania was to meet its 
developmental goals.
1.3. USAID’s Response to the Need to Strengthen Health Systems in Tanzania
1.3.1. PS3 Project Overview
In July 2015, in response to those systems-level gaps resulting from decentralization, USAID started the PS3 
in Tanzania. Th e overall objective of PS3, led by Abt Associates Inc., is to support the GOT in strengthening 
the public-sector system to promote the delivery, quality, and use of services, particularly for underserved 
populations. Th ere is an emphasis on system strengthening and improving quality of services for priority 
health areas (e.g., HIV/AIDS, maternal and child health, TB, malaria) as well as improving key multisectoral 
components at the local government level related to agriculture, infrastructure, and education. Specifi cally, 
PS3 aims to improve public services delivery and outcomes by strengthening national, district, and local 
government authority (LGA) capacity in the following areas: governance and citizen engagement, human 
resources, fi nancial management and information systems, and operations research. 
PS3 works across national, regional, and local government levels. Th is is refl ected in the PS3 Implementation 
Strategy (Figure 1). Th e strategy for implementation shows that systems are strengthened in each of the fi ve 
component areas, while also working across all sectors and levels of public governance. Th e expectation is 
that strengthening systems at all levels will result in improved service delivery at the lowest level—the service 
provider—which will lead to higher public service utilization by the population and better health outcomes. 
At the national level, an emphasis was placed on providing support for key policies and strategies as well as 
strengthening the national system to foster improvements in service delivery at lower government levels. At 
the regional and LGA levels, the PS3 program aimed to facilitate the understanding and implementation of 
national policies and guidelines as well as to provide strategic assistance and improve the management and 
strengthening of systems and resources.
Figure 1. PS3 implementation strategy (source: PS3 year 2 workplan)
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 21
In order to achieve these outcomes, PS3 has undertaken nine primary interventions:
1. Designing and conducting the world’s fi rst Workload Indicators of Staffi ng Need (WISN) coupled 
with Prioritization and Optimization Analysis (POA) to staff Tanzania’s facilities based upon 
benefi ciary need
2. Establishing Direct Facility Financing (DFF) to pay facilities for priority service outputs and increase 
facility autonomy to meet benefi ciary need
3. Extending a transparent, effi cient, and interoperable PlanRep system to the facility level to allow 
facilities to plan and budget to meet benefi ciaries’ needs
4. Clarifying and strengthening roles and relationships for support service delivery between all levels 
and other governance interventions sharing information and obtaining input from citizens and 
communities 
5. Supporting facilities to use their funds to procure the best mix of inputs to deliver services to benefi ciaries
6. Developing and implementing a new Facility Financial Accounting and Reporting Systems (FFARS) so 
that facilities can manage fi nances to provide value to benefi ciaries
7. Establishing systems, tools, and management processes for LGAs and facility managers to motivate 
their staff to deliver services
8. Extending GOT Hospital Management Information Systems (GOTHOMIS) to facility levels so 
service quality improvement is centered on benefi ciaries
9. Financing and payment systems to ensure GOT policies on exemptions are realized so benefi ciaries 
who cannot pay for services are given equitable access and fi nancial risk protection for catastrophic 
health expenditures
Th ese nine interventions aim to produce systemic change within Tanzania’s health systems to benefi t the aver￾age Tanzanian seeking health care. To institutionalize these changes, PS3 has worked closely with the GOT to 
identify priorities, implement comprehensive solutions, and develop strong partnerships at all levels. 
Figure 2. PS3 implementation focus regions
22 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
1.3.2. Target Population and Geographic Areas
PS3 was implemented in Tanzania (see Figure 2) in 93 LGAs within 13 focus regions (Appendix A). Th e focus 
regions were chosen based on several factors, including the location of current USAID programming eff orts, 
the location of donor partner programming for health systems strengthening activities, areas of highest burden 
of disease, and LGAs with low performance of health systems strengthening (HSS) indicators. At the start of 
the project, PS3 utilized a two-phase approach to identify 26 LGAs (two in each of the 13 regions) that would 
receive a more robust package of PS3 support (Phase 1) compared to the 67 LGAs which would receive a 
standard package of PS3 support (Phase 2).1
 Th e Phase 1 LGAs are shown in Table 1. Interventions in Phase 1 
LGAs started in year one of the project. Interventions in Phase 2 LGAs started in year 2 of the project. 
Th e diff erences between Phase 1 and 2 LGAs vary based on the intervention. For example, under human 
resources (HR), in Phase 1 LGAs, PS3 has assisted LGAs in developing customized staff retention plans. 
Similar intensive work was not carried out in Phase 2 LGAs. It is important to note that as time progressed and 
PS3 adapted the interventions to the actual needs of every LGA, the distinction between Phase 1 and Phase 2 
has faded in terms of project involvement, project resources allocated, and the composition of interventions.
In addition, PS3 has created six “Mentor Clusters” in which they have regionally located their fi nance, 
governance, information systems, and HR mentors. Th e clusters are as follows: Southern Cluster (Lindi and 
Mtwara), Central Cluster (Morogoro and Dodoma), Eastern Lake Cluster (Shinyanga, Mwanza, and Mara), 
Western Cluster (Kagera and Kigoma), Western South Highlands Cluster (Mbeya and Rukwa), and Eastern 
South Highlands Cluster (Njombe and Iringa).
PS3 addressed underperforming areas of national and local governments’ operations. As a result, in 2017 the 
GOT expressed strong support for the PS3 activities and supported the rollout at the national level. Th is led 
to diff erent PS3 activities having diff erent rollout schedules. For example, in year 3, the GOT committed to 
hiring 6,180 health workers, which provided PS3 the opportunity to apply the simplifi ed WISN plus POA to 
all 185 LGAs in Tanzania. 
PS3 and Touch Foundation completed the analysis for all 185 LGAs and developed hiring recommendations for 
all LGAs. While this tool was utilized in all LGAs, PS3 provided direct follow-up mentoring to its 93 LGAs. 
Other PS3 instruments have now been implemented in all 185 LGAs. For example, in December 2017, about 
50 percent of all LGAs, health facilities, and schools were utilizing PlanRep for their plans and budgets. 
Beginning in January 2018, the President’s Offi ce, Regional Administration and Local Government (PO￾RALG) offi ces and PS3 teams launched a concerted eff ort to get all LGAs, health facilities, and schools to 
utilize PlanRep. It is now being utilized in all LGAs, representing use among 24,229 public health facilities, 
15,315 primary schools, and 3,511 secondary schools. In Year 3, GOT and PS3 jointly strengthened the Epicor 
accounting system and then introduced it to all LGAs. Th e updated Epicor system was deployed to all LGAs at 
the beginning of FY2018/2019 on July 1, 2018. According to a September 2018 analysis, all LGAs were using 
the system and 87,468 transactions had been processed, representing the concerted eff orts of the PO-RALG 
and PS3 teams who provided mentoring and troubleshooting help to LGAs (USAID/Tanzania Public Sector 
Systems Strengthening Activity, 2018).
1
 For simplicity, robust LGAs will be referred to as “Phase 1,” LGAs and LGAs receiving the standard package of support will 
be referred to as “Phase 2.”
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 23
Table 1. Phase 1 LGAs
Region LGA
Dodoma Chemba DC
Bahi DC
Kigoma Uvinza DC
Buhigwe DC
Iringa Iringa DC
Kilolo DC
Njombe Wanging’ombe DC
Makete DC
Kagera Kyerwa DC
Biharamulo DC
Mtwara Masasi TC
Nanyamba TC
Mbeya Busokelo DC
Chunya DC
Lindi Kilwa DC
Ruangwa DC
Mara Butiama DC
Rorya DC
Mwanza Kwimba DC
Sengerema DC
Rukwa Sumbawanga DC
Kalambo DC
Morogoro Kilosa DC
Gairo DC 
 Shinyanga Shinyanga DC
Kishapu DC
1.3.3. PS3 Results Framework 
As shown in the PS3 implementation strategy and in the results framework below (Figure 3), the PS3 program 
will strengthen systems with the goal improving service delivery. In addition, the results framework emphasizes 
that better use of existing resources will also lead to improved service delivery.
Th ere are three levels of results in the PS3 results framework. Th e fi rst level is focused on strengthening 
operational systems (fi nance, HR, IS, and governance) across national, regional, and local government levels. 
Th e second level is focused on improved management at the LGA level, which includes linking systems and 
services across levels of government and sectors, in addition to more eff ective communication between the 
government and citizens and the private sector. Increased eff ective communication will result in synergies 
between the public and private sectors, as well as public services that are more transparent and responsive to 
citizens’ needs. Th e third and fi nal level is focused on prioritization of effi ciency and responsiveness of public 
systems. Each level of the results framework feeds directly into USAID/Tanzania’s CDCS, IR 3.2, government 
delivery of services improved. 
24 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 3. PS3 results framework (source: PS3 year 2 workplan)
1.4. Evaluation of the PS3 Project
1.4.1. Evaluation Objectives 
MEASURE Evaluation undertook a midline performance evaluation of the PS3 program with the goal of 
examining pre- and post-program inception changes in the uptake of health services, as well as in fi nancial 
and HR indicators in PS3 regions and in other regions of the country. Th e evaluation also uses qualitative 
methods to capture information on the perceptions from program implementers, community members, and 
other stakeholders about the adoption and performance of the PS3 intervention, its strengths and remaining 
challenges, and the stakeholders’ recommendations of ways to address those challenges.
1 .4.2. Evaluation Questions 
Th e evaluation was designed to answer the following evaluation questions that were provided by USAID/Tanzania:
1. In PS3 focus regions, to what extent has the uptake of health services (and related indicators of service 
utilization, coverage of health services, quality of services and access) changed over time?
2. In PS3 focus regions, to what extent have fi nancial and human resources indicators changed over time?
3. Are there diff erences in the changes over time observed in certain groups of LGAs, in particular, 
between Phase 1 and Phase 2 LGAs?
4. How do stakeholders perceive the performance and infl uence of PS3 on uptake of health services, 
fi nancial and human resource systems, and community engagement and governance? 
5. In PS3 focus regions, what was the context of any observed or perceived change in uptake of health 
services, fi nancial and human resource systems, and community engagement and governance?
6. What challenges remain in system functioning and community engagement and governance, and how 
do stakeholders and community members recommend these be addressed?
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 25
2. EVALUATION METHODS
Th is midline evaluation undertook an outcome evaluation to understand how system-level indicators related 
to human resources, fi nancial resources, and health service utilization changed over time in PS3 and in non￾PS3 regions of the country. Data from national routine information systems formed the basis for quantitative 
measurements. In addition, qualitative methods provided a more in-depth understanding of the performance 
of the intervention in human resources, fi nance, and system strengthening, captured unintended, indirect 
eff ects of the intervention, illuminated the process of change, and examined the quality and character of 
the intervention implementation. Th e qualitative component also captured the stakeholders’ perceptions of 
pending challenges in system functioning and community engagement and their recommendations on how 
to address them.
2.1. Quantitative Evaluation Design
Th e quantitative component of the evaluation was designed to answer the fi rst three evaluation questions. 
MEASURE Evaluation utilized data from routine information systems to conduct time trends analysis to 
examine how key indicators of health service uptake, fi nancial resources, and human resources changed over 
time before and after the implementation of PS3. Th e PS3 program focuses operations in 93 LGAs located in 
13 focus regions of mainland Tanzania, therefore, we collected and compiled data from several information 
systems at the LGA-level for outcome monitoring over time. PS3 is also collaborating with the government of 
Tanzania to implement selected tools and interventions nationally. Th e time trends analysis includes examining 
changes in the group of LGAs that were not assigned to PS3 at the start of the project. We called this non-PS3 
group “other regions.” We examine time trends before and after PS3 was established in July 2015 in both PS3 
and other regions. Time series data were collected from all LGAs in mainland Tanzania from 2013 to 2018. 
Th is also allowed us to conduct additional analysis of the before-and-after time trends to subsets of PS3 LGAs, 
specifi cally to Phase 1 and Phase 2 LGAs.
2.1.1. Quantitative Analytical Approach
Two models are employed for examining time trends. Th e fi rst model is designed to answer the fi rst and 
second evaluation questions regarding changes in trends in PS3 and other regions before and after PS3 was 
established. Model 1 is:
+ ௝ܲܵ3 כ ௧௝ܶ଺ߙ + ௝ܲܵ3 כ ௧݅݉݁ݐହߙ + ௝ܲܵ3ସߙ + ௧௝ܶכ ௧݅݉݁ݐଷߙ + ௧௝ܶଶߙ + ௧݅݉݁ݐଵߙ + ଴ߙ = ௧௝ܻ
 ௧௝ߝ + ௝ܲܵ3 כ ௧௝ܶכ ௧݅݉݁ݐ଻ߙ
Where Yjt represents the service utilization, fi nancial, or human resources outcome of interest for LGA j at time t. 
G iven that we were able to access DHIS2 quarterly data for health service utilization indicators from 2014 to 
2018 and that the PS3 offi cially started in July 2015, we defi ne the “Before PS3” period from the fi rst quarter 
of 2014 to the second quarter of 2015.2
 We defi ne the “PS3 active” period from the third quarter of 2015 to 
the fourth quarter of 2018.3
 Formally, Tjt is an indicator variable defi ned as:
ܶ௝௧ = ൜
 ݈ܽݒݎ݁ݐ݅݊ ݅݉݁ݐ ݁ݒ݅ݐܿܣ ܲܵ3 ݄݁ݐ ݉݋ݎ݂ ݏ݅ ݊݋݅ݐܽݒݎ݁ݏܾ݋ ݄݁ݐ ݂݅ 1,
݈ܽݒݎ݁ݐ݅݊ ݅݉݁ݐ ܲܵ3 ݁ݎ݋݂݁ܤ ݄݁ݐ ݉݋ݎ݂ ݏ݅ ݊݋݅ݐܽݒݎ݁ݏܾ݋ ݄݁ݐ ݂݅ 0,
We intend to examine time trends for the PS3 regions and for the other regions. Th en, we defi ne: 
ܲܵ3௝ = ൜
 ݏ݊݋݁݃݅ݎ ܲܵ3 ݄݁ݐ ݅݊ ݏ݅ ݆ ܣܩܮ ݂݅ 1,
(ܲܵ3 െ ݊݋݊) ݏ݊݋݁݃݅ݎ ݎ݄݁ݐܱ ݄݁ݐ ݅݊ ݏ݅ ݆ ܣܩܮ ݂݅ 0,
2
 In some cases, data were available from time periods prior to 2014. Data for some service utilization indicators was 
available as early as the third quarter of 2013. Financial and human resources data were available from 2012 at yearly time 
intervals. In these cases, earlier quarters/years were also included in time period “Before PS3.”
3
 Financial data were available only through fi scal year 2017/2018.
ܶ௝௧
26 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
And, timet = t, time count in quarters since the fi rst quarter of 2014.2
Model 1 is estimated for each health service utilization indicator using the pooled quarterly time series dataset 
for all LGAs of the country. Th e model is also estimated for each fi nancial and human resources indicator 
using pooled yearly time series data for all LGAs.
Th e model provides estimates of the time trends for specifi c time periods and for each group of interest. Th ey 
are defi ned as the average change per quarter in the health service utilization indicator Yjt for each of the two 
time-intervals of interest for both PS3 and other regions. Specifi cally, we have that:
 ( a1
 + a5 ) is the average change per quarter in Yjt before PS3, in PS3 regions
 ( a1
 + a5 ) + ( a3
 + a7 ) is the average change per quarter in Yjt during the time PS3 active, in PS3 regions
 a1
 is the average change per quarter in Yjt before PS3, in other regions
 ( a1
 + a3 ) is the average change per quarter in Yjt during the time PS3 active, in other regions.
Model 1 also allows us to examine if the time trend before PS3 changed during the time PS3 active. In order 
to examine the time trend changes for each specifi c group of regions, we take the diff erence of the estimated 
time trends above:
 In PS3 regions, the change in time trends before and after PS3 is given by ( a3 + a7 )
 In other regions, the change in time trends before and after PS3 is given by a3
It is important to note that the modeling approach used for models 1 and 2 above is an adaptation of the 
diff erence-in-diff erences strategy to model time trends for two areas (and three areas, in the case of model 
2) and to examine changes in time trends before and after PS3. Th e changes in the time trends estimated 
by the model cannot be interpreted as program eff ects on the outcomes. If we take the diff erence of the two 
diff erences above we have a7
 , which is interpreted as the estimate of how much the before-and-after change in 
time trends in PS3 regions diff ers from the before-and-after change in time trends in other regions.
2.1.1.1. Examining Differences in Time Trends between PS3 Phase 1 and Phase 2 LGAs 
We extend model 1 to expand the answers to evaluation question three regarding diff erences in time trends for 
three groups, PS3 Phase 1, PS3 Phase 2, and other regions. We defi ne:
ܲܵ3ܲ1௝ = ൜
 ݌ݑ݋ݎ݃ 1 ݁ݏ݄ܲܽ ܲܵ3 ݄݁ݐ ݅݊ ݏ݅ ݆ ܣܩܮ ݂݅ 1,
(2 ݁ݏ݄ܲܽ ܲܵ3 ݎ݋ ݎ݄݁ݐܱ) ݌ݑ݋ݎ݃ ݎ݄݁ݐ݋ ݕܽ݊ ݅݊ ݏ݅ ܣܩܮ ݄݁ݐ ݂݅ 0,
ܲܵ3ܲ2௝ = ൜
 ݌ݑ݋ݎ݃ 2 ݁ݏ݄ܲܽ ܲܵ3 ݄݁ݐ ݅݊ ݏ݅ ܣܩܮ ݄݁ݐ 1,݂݅
 (1 ݁ݏ݄ܲܽ ܲܵ3 ݎ݋ ݎ݄݁ݐܱ) ݌ݑ݋ݎ݃ ݎ݄݁ݐ݋ ݕܽ݊ ݅݊ ݏ݅ ܣܩܮ ݄݁ݐ ݂݅ 0,
Th en, Model 2 is:
כ ௧௝ܶ଺ߚ + ௝ܲܵ3ܲ1 כ ௧݅݉݁ݐହߚ + ௝ܲܵ3ܲ1ସߚ + ௧௝ܶכ ௧݅݉݁ݐଷߚ + ௧௝ܶଶߚ + ௧݅݉݁ݐଵߚ + ଴ߚ = ௧௝ܻ
+ ௝ܲܵ3ܲ2 כ ௧௝ܶ଴ଵߚ + ௝ܲܵ3ܲ2 כ ௧݅݉݁ݐଽߚ + ௝଼ܲܵ3ܲ2ߚ + ௝ܲܵ3ܲ1 כ ௧௝ܶכ ௧݅݉݁ݐ଻ߚ + ௝ܲܵ3ܲ1
 ௧௝ߝ + ௝ܲܵ3ܲ2 כ ௧௝ܶכ ௧݅݉݁ݐଵଵߚ
Model 2 is estimated for each health service utilization indicator using the pooled quarterly time series 
dataset for all LGAs of the country. For fi nancial and HR we used pooled yearly time series data. Th is model 
provides estimates of the time trends for specifi c time periods and for each of the three groups of interest. Th e 
interpretation of coeffi cients is similar as in model 1. Specifi cally, we have that:
 ( β1 + β5 ) is the average change per quarter in Yjt before PS3, in Phase 1 LGAs
 ( β1 + β5 ) + ( β3 + β7 ) is the average change per quarter in Yjt during the time PS3 active, in Phase 1 LGAs
 ( β1 + β9 ) is the average change per quarter in Yjt before PS3, in Phase 2 LGAs
ܻ௝௧
ܻ௝௧
ܻ௝௧
ܻ௝௧
ܻ௝௧
ܻ௝௧
ܻ௝௧
ܻ௝௧
ଵߚ
ଵߚ
ଵߚ
ହߚ
଻ߚ ହߚ
ଽߚ
ଷߚ
ଵߙ
ଵߙ
ଵߙ
ଵߙ
ଷߙ
ଷߙ ହߙ
ହߙ
଻ߙ
଻ߙ ଷߙ
ଷߙ
଻ߙ
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 27
 ( β1 + β9 ) + ( β3 + β11 ) is the average change per quarter in Yjt during the time PS3 active, in Phase 2 LGAs
 β1
 is the average change per quarter in Yjt before PS3, in other regions
 ( β1 + β3 ) is the average change per quarter in Yjt during the time PS3 active, in other regions
Model 2 also allows us to examine if the time trends before PS3 changed during the time PS3 active for each 
of the three groups of LGAs. As in the previous model, in order to examine time trend changes for each 
specifi c group of LGAs, we just take the diff erence of the estimated time trends above:
 In Phase 1 LGAs, the change in time trends before and after PS3 is given by ( β3 + β7 ) 
 In Phase 2 LGAs, the change in time trends before and after PS3 is given by ( β3 + β11 ) 
 In other regions, the change in time trends before and after PS3 is given by β3
If we take the diff erence of the changes above for Phase 1 and other regions, we have β7
 , which is interpreted 
as the estimate of how much the before-and-after change in time trends in Phase 1 LGAs diff ers from the 
before-and-after change in time trends in other regions. Likewise, the diff erence of the changes above for Phase 
2 and other regions is β11 , which is interpreted as the estimate of how much the before-and-after change in 
time trends in Phase 2 LGAs diff ers from the before-and-after change in time trends in other regions. For 
obtaining the diff erence of the changes for Phase 1 and Phase 2 we can use ( β7 – β11 ), which is interpreted as 
the estimate of how much the before-and-after change in time trends in Phase 1 LGAs diff ers from the before￾and-after change in time trends in Phase 2 LGAs.
2.1.2. Outcome Measures
Th e PS3 objective of improving the public health system’s operational capacity leading to improved service 
delivery of public services and population health can be defi ned and measured in a number of ways. Th e 
program is expected to promote the delivery and use of health services, through strengthening and supporting 
government systems related to the following components: governance and citizenship engagement, human 
resources, fi nancial resources, and information systems. 
By design, the outcome evaluation approach allows for the analysis of multiple outcomes. Th e fi nal list of 
indicators included in the evaluation is shown in Table 2. To the extent possible, the indicators were chosen 
based on their relevance in refl ecting LGA operational performance and uptake of services infl uenced by PS3 
interventions, quality of data, availability of time series data in the existing information systems, and feasibility 
of accessing the data by the evaluation team. We prioritized inclusion of indicators already included in the PS3 
results framework, particularly related to fi nancial and human resources, where time series data were available. 
Th e selection of indicators was informed by review of several key government documents (Annual Health 
Sector Performance Profi le Reports, Annual Health Statistical Abstract (also called Annual Health Statistical 
Tables and Figures), the Result-Based Financing Operational Manual, and the Health Sector Strategic Plan, 
2015–2020 (HSSP IV) (Ministry of Health, Community Development, Gender, Elderly and Children, 2015, 
2018). Indicators were also selected from feedback from stakeholders at a national meeting conducted in 
Dodoma in April 2018, and from consultation with USAID/Tanzania offi cials. We also reviewed USAID/
Tanzania’s Performance Management Plan (PMP) to align our indicators with those in the PMP. For the 
time series analysis, it was necessary to have data for years before PS3 implementation and for the years PS3 
was active. Indicators were also chosen if data was available starting during the fi rst quarter in 2014 or earlier 
through at least the last quarter in 2018. Indicators were prioritized if data was available quarterly for all LGAs, 
as detailed time series data were necessary for the analysis. Appendix C has defi nitions for each indicator, the 
data source, and the date range used in the analysis. 
2.1.3. Description of Quantitative Data Sources and Data Collection
Tanzania has numerous routine data sources for LGA-level data, such as PlanRep, DHIS2/HMIS, EPICOR, 
HCMIS/Lawson, and BEMIS, though most of these data sources do not have complete time series data from 
2012 to 2018 and data are not necessarily available for all LGAs. We spent a considerable amount of time 
trying to identify fi nance and HR data sources that would be able to provide the type of indicators needed 
ଵߚ
ଵߚ
ଵߚ
ܻ௝௧
ܻ௝௧
ܻ௝௧
ଷߚ ଽߚ
ଷߚ
ଵଵߚ
ଵଵߚ ଷߚ
ଷߚ
ଷߚ
଻ߚ
଻ߚ
ଵଵߚ
ଵଵߚ ଻ߚ
28 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
and for the duration of time required. After substantial review and discourse over availability of complete 
time series data, including consultation with PS3, USAID/Tanzania, and the GOT, we determined that the 
indicators for each of the main domains would come from a variety of data sources (Appendix C). All the data 
on service delivery from 2013–2018 were extracted from the DHIS2 initially. To ensure the data refl ected the 
most up-to-date records, a further data compilation and verifi cation process was undertaken in February 2019 
with staff from the MOHCDGEC’s M&E Unit. Th e data were extracted for all the selected indicators for each 
quarter from 2014 to 2018 and organized in Microsoft Excel fi les.4
 
Table 2. List of indicators
Category Indicator
Service utilization
Maternal health
ANC coverage before 12 weeks gestational age 
Pregnant women attending ANC 4+ times
Prevalence of pregnant women receiving iron 
and folic acid (IFA) supplementation for 90+ days
Pregnant women receiving TT2+ (tetanus/
diphtheria/pertussis vaccination) at ANC
Women tested for anemia at ANC
Women tested for syphilis at ANC
Deliveries taking place in health facilities
Births assisted by skilled attendants**
Mothers receiving postnatal care before 7 days
Infant 
and 
child 
health
Prevalence of low birth weight
Measles vaccination coverage
Penta3 vaccination coverage
HIV/AIDS
ANC partners HIV testing rate
HIV-exposed infants receiving fi rst HIV test within 
2 months after birth
HIV-exposed infants initiated on cotrimoxazole 
within 2 months after birth
Couple HIV counseling and testing at ANC
Other
Contraceptive prevalence rate**
Health facilities with RCH tracer drugs package
Financial 
resources
Personal Emoluments (PE) per capita in health^^
PE per capita in education^^
Other charges (OC) per capita in health^^
OC per capita in education^^
Human resources
Nurses per 10,000 people**,^^
Doctors per 10,000 people
Assistant medical offi cers per 10,000 people
Health care workers that are female
Primary school student-teacher ratio**,^^
Secondary school student-teacher ratio
Primary school students that are female
Secondary school students that are female
Primary school teachers that are female
Secondary school teachers that are female
** Indicator based on USAID PMP Indicator list
^^ Indicator based on PS3 M&E plan Indicator list
4 In some cases, data were available from time periods prior to 2014. Data for some service utilization indicators was 
available as early as the third quarter of 2013. Financial and human resources data were available from 2012 at yearly time 
intervals. In these cases, earlier quarters/years were also included in time period “Before PS3.”
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 29
 Once it was determined that access to the EPICOR fi nancial system was unlikely, and more importantly, 
would likely yield neither the indicators desired nor the complete time series, we engaged the PS3 project to 
understand what fi nance data they were utilizing for project monitoring of key indicators. PS3 had partnered 
with PO-RALG to review and compile LGA level council fi nancial reports (CFR) from 2013/2014 to 
2017/2018. Th eir two primary fi nance-related indicators were being captured in these CFRs. In partnership 
with a key staff member from PS3, we worked directly with PO-RALG to complete the time series and 
compile and clean the CFRs for 2012/2013. 
After review of the routine information systems on human resources, three data sources were selected for use: 
Basic Education Statistics in Tanzania (BEST), OPENDATA for HR for Education, and Human Resources 
for Health Information Systems (HRHIS) for HR for health data. Permission to access the data from these 
databases was obtained from the Ministry of Education Science and Technology and the Ministry of Health 
Community Development Gender Elderly and Children, respectively. For the 2016–2018 HR for education 
data, permission was sought from PO-RALG.
See Appendix D for more background on the data sources utilized in the PS3 evaluation. 
2.2. Qualitative Study Design
A total of six LGAs in PS3 areas and three LGAs in non-program areas were purposefully selected according 
to: (1) a range of system performance, socioeconomic indicators, and disease burden, (2) status as a Phase 
1 versus Phase 2 LGA, (3) status as a newly established LGA versus historical LGA, and (4) location in key 
clusters: Eastern Lake Cluster, Central Cluster, and Eastern Southern Highlands Cluster. 
Table 3. LGAs selected for qualitative sample
Phase 1 Phase 2 Other
Eastern Lake Shinyanga DC (R) Ilemela MC (U) Nyong’hwale DC (R)
Central Bahi (R) Kilosa (R) Manyoni (R)
E. Southern Highlands Wanging’ombe DC (R) Makambako TC (U) Songea MC (U)
R=rural; U=Urban
We intentionally chose a mix of urban and rural wards within LGAs for conduct of community FGDs. 
Adult participants were purposefully selected by sex and location, and based on their participation in health 
or education committees. Two focus group discussions (1 male/1 female) were conducted in each of the nine 
LGAs. Each FGD had 8 to 10 participants and in total there were 90 men and 90 women participants in 
community FGDs. FGDs took from 1 to 2 hours and were conducted in Swahili.
Ripple eff ects mapping (REM) group discussions were conducted in the six program LGAs (one per LGA). 
Each discussion had 8–12 adult participants and there were a total of 29 men and 20 women. Councilors 
were selected based on participation in PS3 trainings and to represent a range of ward vulnerabilities (e.g., 
higher income ward vs. lower income; urban vs. rural, etc.). Discussions took from 1.5 to 2.5 hours and were 
conducted in Swahili.
Th e team conducted 70 key informant interviews (58 men, 12 women) with a range of stakeholders, 
including government staff from LGA, regional, and national levels (e.g., Ministry of Finance, PMO-PO￾RALG, POPSM), international development partners, and program staff . Stakeholders were selected based 
on program exposure and to represent diff erent sectors (although more heavily in the health sector). At the 
LGA level, we selected fi ve to six government stakeholders per each of the 9 LGAs for a total of 42, one in 
each of the three regions (Regional Secretariat), and 14 at the national level, across the technical areas of HR, 
fi nance, health services, and governance and citizen engagement. Program staff were selected based on their 
in-depth knowledge of the program components—IS, HR, fi nance, and governance and citizen engagement. 
We interviewed 11 program staff —fi ve in the clusters plus six at the national level. KIIs lasted from 30 to 60 
minutes. KIIs were conducted mainly in Swahili but some, mostly national level, were conducted in English. 
See Table 4 for the KII sampling plan.
30 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Table 4. Sampling results for key informant interviews
Stakeholder Type Level Subtotal
Program Cluster 5
National 6
Subtotal 11
Government LGA 42
Regional Secretariat 3
National 14
Subtotal 59
Total 70
Researchers used several tools of the Grounded Th eory approach, namely inductive coding, and constant 
comparison of codes, themes, and resultant theoretical models. Pure Grounded Th eory methods were not 
applicable, as this research began with some preconceived hypotheses and orientation related to systems 
strengthening. 
After community FGDs and stakeholder KIIs, the audio tapes were transcribed and translated by the 
interviewers. Stakeholder REM FGDs were not audio-recorded but detailed workshop notes were taken and 
transcribed and photos of mind maps were taken.
Data from completed transcripts was entered into Dedoose, a qualitative data software program. Mind map 
data was entered into the MinJet program. Researchers coded the transcripts from KIIs and FGDs and data 
from REM maps using a combination of deductive codes stemming from the logic model and research 
questions and inductive codes that emerged through the data collection and initial transcript and map review 
processes. Broad code categories were used, rather than detailed, line-by-line coding. Matrices were next 
developed to compare themes across sites and other variables, such as status as a control versus program area, 
types of informants/participants, and status as a Phase 1 versus 2 LGA. Lastly, researchers synthesized the 
results into an initial draft report of qualitative fi ndings. Matrices and selected transcripts were then reread to 
confi rm draft fi ndings and adjustments in the text were made as needed to develop the current version.
2.3. Ethical Considerations
Prior to quantitative and qualitative data collection, ethical clearance for the study protocol and data collection 
instruments was obtained from the National Institute for Medical Research in Tanzania and the University 
of North Carolina (UNC) at Chapel Hill Institutional Review Board (IRB), where required. An exemption 
was received from the UNC IRB for the quantitative protocol given that the project was limited to analysis of 
secondary data at the LGA level. No individual identifi ers were collected from respondents who participated in 
qualitative interviews in order to ensure confi dentiality. Informed consent was obtained from all participants in 
qualitative research.
In addition, all non-Tanzanian researchers obtained a research permit from the Tanzania Commission for 
Science and Technology (COSTECH). We also received additional clearance and research approval from 
the Tanzania National Bureau of Statistics and the Presidents’ Offi ce Regional Administration and Local 
Government (TAMISEMI).
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 31
3. FINDINGS
3.1. Quantitative Evaluation: Evaluation Questions 1 and 3
Th is section provides answers to evaluation questions 1 and 3: 
 Evaluation question 1: In PS3 regions, to what extent has the uptake of health services changed over time?
 Evaluation question 3: Are there diff erences in the changes over time observed in certain groups of LGAs, 
in particular, between Phase 1 and Phase 2 LGAs?
Questions 1 and 3 are answered examining the time trends of key health service utilization indicators in PS3 
regions and in the other (non-PS3) regions of the country. Th e graphs in this section present the trajectory 
of the quarterly weighted average of diff erent service utilization indicators for the LGAs in those two groups 
(PS3 and other regions). Time series quarterly data, largely from the fi rst quarter of 2014 (2014Q1) to the 
last quarter of 2018 (2018Q4), were obtained from the DHIS2 routine information system for each LGA of 
the country. For a few indicators we included quarters in 2013 as the data were available. Th e weights used 
were the LGA female population for 2015. Th e graphs allow us to visualize the evolution, over time, of key 
health utilization indicators for those two groups of LGAs before PS3 started; that is, from 2014 to mid-2015, 
and during the time PS3 has been active—from the third quarter of 2015 to the end of 2018. We have also 
included, where applicable, qualitative information gathered on the context and perception of these trends (for 
evaluation questions 4 and 5). Additional qualitative results are presented in section 3.3.
Figure 4. Antenatal care coverage before 12 weeks gestational age
Table 5. Annualized average change in the percentage of pregnant women starting antenatal care 
before 12 weeks of gestation (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value 
of difference
PS3 -1.99** +6.72** +8.71** 0.000
– Phase I -1.86 +7.26** +9.11** 0.000
– Phase II -2.05** +6.49** +8.55** 0.000
Other -2.41* +4.12** +6.53** 0.000
Difference (PS3 – Other) +2.18 0.183
Difference (Phase I – Phase II) +0.56 0.792
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 169 in 2016; PS3: 87 LGAs (Phase I: 25; Phase II: 62); Other: 82 LGAs.
32 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 4 and Table 5 present the time trends and annualized change for ANC initiated before 12 weeks of 
gestation; that is, during the fi rst trimester of pregnancy. As seen in the graph, during the period from 2014 to 
mid-2015, the indicator had low (less than 15%) and declining levels in both PS3 and other regions (as shown 
in the second column of Table 5, the indicator was declining at a rate of almost -2 percentage points per year 
in PS3 regions and at about the same rate of -2.41 percentage points in other regions.) Th e trend changed 
after mid-2015 from a low level below 15 percent in the country, to over double in PS3 areas and almost 
double in other regions by the end of 2018. Table 5’s third column shows positive and signifi cant increasing 
annual growth rates in this indicator during the PS3 active time interval in both areas. However, there are no 
signifi cant diff erences in the rates of change between PS3 and other regions. Figure B.1 in Appendix B shows 
that time trends in Phase 1 regions were similar to Phase 2 LGAs’ trends. 
Tanzania’s target for ANC initiation <12 weeks is 70 percent by 2020. Th ere have been eff orts to improve 
community outreach activities to encourage women to attend clinics and reinforce the value of ANC for safe 
motherhood since 2015, when the last DHS showed this indicator at only 24 percent of all pregnant women. Th e 
fi ve-year USAID Boresha Afya project, started in 2016, is one of the largest interventions supporting integrated 
service delivery at the health facility and community levels across the country. Similar projects have rolled-out in 
other regions, possibly contributing to the increasing trends in PS3 and other regions in 2017 and 2018.
Another thing which I can talk about the health service is that we have World Vision which has created awareness 
to community in general. It has helped to create awareness to leaders and citizens, and it has selected a group to visit 
households door to door in order to sensitize women, children, and pregnant women so that they encourage women 
and children to attend clinics for their health. [KII with women, other]
Figure 5. Pregnant women attending antenatal care 4+ times
Table 6. Annualized average change in the percentage of pregnant women with at least four 
antenatal care visits (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value 
of difference
PS3 +2.40* +9.07** +6.67** 0.000
– Phase I +0.12 +9.27** +9.15** 0.000
– Phase II +3.34** +8.98** +5.64** 0.000
Other +3.44** +9.31** +5.87** 0.008
Difference (PS3 – Other) +0.79 0.754
Difference (Phase I – Phase II) +3.51 0.165
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 161 in 2016; PS3: 84 LGAs (Phase I: 24; Phase II: 60); Other: 77 LGAs.
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 33
Figure 5 and Table 6 present the time trends and annualized change in the percent of women attending at least 
four ANC visits. Prior to the PS3 implementation period, less than 40 percent of pregnant women in both 
PS3 and other regions had at least four ANC visits, however this indicator signifi cantly increased by 2.4 and 
3.44 percentage points per year within PS3 and other regions, respectively. During the PS3 active period there 
were positive and signifi cant growth rates in both regions, exceeding 9 percentage points per year in both PS3 
and other regions. Th ere were no signifi cant diff erences, however, in the annual growth rates of this indicator, 
before and after PS3 initiation, for PS3 and other regions. At the end of 2018, just under 70 percent of 
pregnant women attended ANC four times in PS3 regions, and just over 70 percent attended in other regions. 
Phase 1 and Phase 2 areas follow similar patterns over time (Figure B.2, Appendix B). 
Improvements in access to services by pregnant women were attributed qualitatively to various program 
and contextual factors, including construction of additional health facilities, funding for ambulances, and 
improved roads for transport. Many of these contextual factors were addressed by local government councilors 
as part of improved governance, as discussed in section 3.3.1. Community members also discussed the 
improvement in primary services themselves as motivation for increased attendance.
Our district hospital has improved services for pregnant women and children. Th ey used to come regularly or twice 
[a] week specifi cally for pregnant women and children, which means the targeted group did not feel compelled to go to 
the hospital. Th ey realized the challenge and worked on it. Th ey do the same in another distant village.
[FGD with men, other]
Figure 6. Prevalence of pregnant women receiving iron and folic acid supplementation
Table 7. Annualized average change in percentage of pregnant women given iron and folic acid 
supplementation (in percentage points)
LGAs
Before PS3
2013Q3 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value 
of difference
PS3 +8.41** +2.27** -6.14** 0.000
– Phase I +6.68** +2.13 -4.55 0.058
– Phase II +9.11** +2.32** -6.79** 0.000
Other +6.44** +1.70* -4.75** 0.002
Difference (PS3 – Other) -1.39 0.463
Difference (Phase I – Phase II) +2.24 0.407
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 184; PS3: 93 LGAs (Phase I: 26; Phase II: 67); Other: 91 LGAs. 
34 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 6 and Table 7 show the time trends and annualized change in the percentage of iron and folic acid 
supplementation (IFA) among pregnant women. Prior to PS3 initiation, levels of this indicator were increasing 
at 8.41 and 6.44 percentage-points per year for PS3 areas and other regions, respectively. By the middle of 
2015, both PS3 and other regions had about 70 percent of pregnant women receiving IFA. During the PS3 
active period, there was a steep, universal decline in this indicator for all areas from early 2016, hitting a 
minimum prevalence of around 30 percent in mid-2017. Within PS3, Phase 1 and Phase 2 trends followed 
similar pattern (Figure B.3, Appendix B). Th is decline rapidly recovered, exceeding the prior peak levels within 
about a year. Refl ecting these trends, the annual increase during the PS3 active period was signifi cantly lower 
in contrast to before PS3 yearly changes for all areas. However, changes in PS3 regions were not signifi cantly 
diff erent to those in other regions. 
Th e annual health sector performance profi le (AHSP) reports, which explain factors infl uencing health 
service indicators, does not make a specifi c mention of folic acid supplementation. However, the 2017/2018 
AHSP report notes challenges of low stocks and, for some facilities, stockouts of vaccines and other essential 
medicines, which may have applied to this indicator as well. It is noted, for instance, that in 2016, the 
government funded only 60 percent of the approved budget for immunization commodities (AHSP, 2016).
Th ese budget constraints and stockouts were felt at the community level. However, systems like those put in 
place by PS3 and the GOT have helped monitor and account for these issues, as noted by one medical offi cer:
Big changes are such as having information at the Municipal on how much medicine has been used, the amount 
used, average consumption within a month, budget, the stock, and ability to press order for the next time. Also 
creating emergency order if you have fi nished your stock. Th ey have also helped us with inventory control system 
which are now installed in our health centers which we didn’t have before, so that health center can order medicines 
for six months and if the stock is about to fi nish even within three months, they can press order for more. So, we have 
the system at the Municipal level and Regional level so that you make sure you are not out of stock
[KII with town medical offi cer, Phase 2]
Figure 7. Pregnant women receiving TT2+ vaccination at antenatal care
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 35
Table 8. Annualized average change in percentage of pregnant women receiving TT2+ vaccine 
(in percentage points) 
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +3.44** +0.72 -2.72** 0.005
– Phase I +3.87* +0.34 -3.53* 0.030
– Phase II +3.27** -0.88* -2.38* 0.046
Other +7.28** +1.87** -5.41** 0.000
Difference (PS3 – Other) +2.69 0.059
Difference (Phase I – Phase II) -1.15 0.567
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 181; PS3: 90 LGAs (Phase I: 25; Phase II: 65); Other: 91 LGAs. 
Figure 7 and Table 8 show the time trends and annualized average change in the proportion of pregnant 
women receiving at least two doses of the tetanus toxoid vaccine (TT2+). From 2014 to the middle of 2015, 
the TT2+ indicator rose by 3.44 and 7.28 percentage points annually in PS3 and other regions, respectively. 
By the middle of 2015, about 65 percent of pregnant women in PS3 regions and 70 percent in other regions 
received TT2+. During the PS3 active period, there was some volatility in this indicator, especially in other 
regions, with a steep decline during 2016 followed by an overall recovery, attaining mid-2015 levels by the end 
of 2018. Generally, there was no signifi cant change over the PS3 active period in PS3 areas, however, there 
was a signifi cant, but small, decline in Phase 2 areas (Figure B.4, Appendix B). Th e change in other regions 
was signifi cant during this period, at 1.87 percentage points per year. Comparing the trends before and 
during PS3, the diff erences in the growth rates was signifi cant for PS3 and other regions, at -2.72 and -5.41 
percentage points, respectively. Nevertheless, the diff erence in the changes over time between both areas was 
not signifi cant. 
Figure 8. Women tested for anemia at antenatal care
36 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Table 9. Annualized average change in the percentage of pregnant women tested for anemia at 
antenatal care (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -1.05** +9.08** +10.13** 0.000
– Phase I -0.65 +11.34 +11.99** 0.000
– Phase II -1.21 -8.16** +9.37** 0.000
Other +4.14* +4.68** +0.54 0.835
Difference (PS3 – Other) +9.59** 0.001
Difference (Phase I – Phase II) +2.62 0.417
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 184; PS3: 93 LGAs (Phase I: 26; Phase II: 67); Other: 91 LGAs. 
Figure 8 and Table 9 show the time trends and annualized change in the proportion of pregnant women tested 
for anemia. Figure 8 shows that initial levels of anemia testing were lower in PS3 regions than in other regions. 
Th e annualized change prior to PS3 implementation was -1.05 and 4.14 percentage-points for PS3 and other 
regions, respectively. During the PS3 active period, the gap lessened toward the end of 2016. By the end of 
2018, PS3 regions had slightly higher levels of anemia testing than other regions, at almost 70 percent. Th e two 
implementation phases followed these similar patterns and trends, but Phase 1 LGAs had consistently lower 
levels of this indicator than Phase 2 LGAs (Figure B.5, Appendix B). During this time, this indicator grew, on 
average, by 9.08 percentage points per year in PS3 regions and 4.68 percentage points per year in other regions. 
Comparing the two time points, other regions’ growth rates did not change, but PS3 regions’ rates were 10.13 
percentage points higher during the PS3 period. Th e diff erence between the two times was 9.59 percentage 
points higher in PS3 regions compared with other regions, which was a signifi cant diff erence. 
Figure 9. Women tested for syphilis at antenatal care
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 37
Table 10. Annualized average change in percentage of pregnant women tested for syphilis at 
antenatal care (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +10.32** +15.44** +5.12* 0.024
– Phase I +4.93 +18.04** +13.11** 0.001
– Phase II +12.52** +14.38** +1.87 0.482
Other +17.53** +7.42** -10.12** 0.001
Difference (PS3 – Other) +15.24** 0.000
Difference (Phase I – Phase II) +11.25* 0.017
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 184; PS3: 93 LGAs (Phase I: 26; Phase II: 67); Other: 91 LGAs.
Figure 9 and Table 10 show the time trends and annualized change in percentage of pregnant women tested 
for syphilis. Figure 9 shows that initial levels of testing were lower in PS3 regions than other regions, with 
an increase followed by a decline, which was less extreme in other regions. Th e annualized change prior to 
PS3 implementation was 10.32 and 17.53 percentage points for PS3 and other regions, respectively. During 
the PS3 active period, the gap lessened, so that by the end of 2018, PS3 regions had slightly higher levels of 
anemia testing than other regions, at about 75 percent. Th e two implementation phases followed these similar 
patterns and trends, but Phase 1 LGAs had consistently lower levels of this indicator than Phase 2 LGAs until 
converging in 2017 (Figure B.6, Appendix B). During this time, this indicator grew, on average, by 15.44 
percentage points per year in PS3 regions, and 7.42 percentage points per year in other regions. Comparing 
changes across the time periods, indicator growth rates were 5.12 percentage points higher in PS3 regions, 
and 10.12 percentage points lower in other regions. Th e diff erence between the two time periods was 15.24 
percentage points higher in PS3 regions compared with other regions, which was a signifi cant diff erence. 
Th e increase in the average rate of syphilis testing in Phase 1 LGAs was also signifi cantly higher—by 11.25 
percentage points—compared with Phase 2 LGAs. 
Figure 10. Deliveries taking place in health facilities
38 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Table 11. Annualized average change in the percentage of deliveries occurring in health facilities 
(in percentage points)
LGAs
Before PS3
2013Q3 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value 
of difference
PS3 +4.43** +2.47** -1.96** 0.000
– Phase I +4.67** +3.22** -1.46 0.220
– Phase II +4.33** +2.16** -2.17** 0.000
Other +3.95** +1.94** -2.01** 0.003
Difference (PS3 – Other) +0.05 0.954
Difference (Phase I – Phase II) +0.71 0.595
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 184; PS3: 93 LGAs (Phase I: 26; Phase II: 67); Other: 91 LGAs. 
Figure 10 and Table 11 show the time trends and annualized change in the percentage of births delivered 
in health facilities. Prior to the middle of 2015, PS3 regions had lower levels of health facility deliveries than 
other regions; however, both areas displayed increasing trends over this time. PS3 regions were increasing 
at 4.43 percentage points annually and other regions were increasing by 3.95 percentage points—changes 
in both areas were statistically signifi cant. During the PS3 active period, the increasing trend was 2.47 and 
1.94 percentage points for PS3 and other regions, respectively. Diff erences in the annualized trends before 
and during PS3 were not signifi cantly diff erent when comparing PS3 and other regions. At the end of 2018, 
just over 95 percent of live births occurred in health facilities in both PS3 and other regions. In terms of the 
implementation phase, Phase 1 areas had lower initial levels of this indicator than Phase 2, but both PS3 
phases converge at a similar level (Figure B.7, Appendix B). 
Th e results for this indicator, whose performance has been improving, are also supported by qualitative 
fi ndings. Several community members and key informants from PS3 and other regions commented on the 
challenges of delivering in facilities in the past and ongoing improvements. 
In the past, mothers refused to give birth at the hospital…but, after being educated, the majority attends clinics 
instead of visiting witch doctors. Th is is what I can say about changes. [FGD with women, other]
Th e convergence of the PS3 and other regions’ trajectories from 2017 may refl ect the eff ect of other ongoing 
programs, such as USAID Boresha Afya, which supports integrated community and health facility level service 
provision, and Amref which primarily works in the Lake Zone.
Amref helped educate pregnant women who used to deliver at home and not at the health centres. But they have also 
helped women and nowadays they conduct a lot of seminars even in the market places with ones that I witnessed. 
Women are now delivering at the health centers and focusing more on the life of mother and child.
[KII with councilor, other]
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 39
Figure 11. Births delivered by skilled attendants
Table 12. Annualized average change in the percentage of births delivered by skilled birth 
attendants (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +5.79** +2.29** -3.50** 0.004
– Phase I +5.04* +3.10** -1.95 0.446
– Phase II +6.10** +1.97** -4.13** 0.002
Other +3.94** +1.58** -2.35 0.078
Difference (PS3 – Other) -1.14 0.525
Difference (Phase I – Phase II) +2.19 0.448
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 184; PS3: 93 LGAs (Phase I: 26; Phase II: 67); Other: 91 LGAs. 
Figure 11 and Table 12 show the time trends and annualized change in the percentage of births delivered by 
skilled birth attendants (SBAs). Prior to the middle of 2015, PS3 regions had lower levels of SBA deliveries than 
other regions (except for the fourth quarter in 2014). Similar to the health facility delivery indicator, both areas 
displayed increasing trends over this time, converging in 2018. Table 12 shows statistically signifi cant increases 
prior to mid-2015, with annualized changes of 5.79 percentage points for PS3 and 3.94 for other regions. 
During the PS3 active period, the increasing trend reduced to 2.29 and 1.58 percentage points for PS3 and 
other regions, respectively. Diff erences in the annualized trends before and during PS3 did not signifi cantly 
diff er between PS3 and other regions. At the end of 2018, SBAs delivered about 93 percent of live births in 
both PS3 and other regions. As for the implementation phases, similarly to the facility delivery indicator, Phase 
1 LGAs had lower initial levels of this indicator than those in Phase 2, but both PS3 phases started to converge 
by the end of 2018 (Appendix B, Figure B.8).
While trends have increased overall, some community members still perceive a shortage of health attendants at 
facilities, particularly among other regions where PS3 was not active. 
When a woman is pregnant, she must attend the clinics. Th ere are very few nurses who can help these women to 
deliver successfully. Sometimes, when you go to the hospital, you fi nd a very long queue for consultation. Th at means 
doctors are not enough here. [FGD with women, other]
40 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 12. Mothers receiving postnatal care before 7 days
Table 13. Annualized average change in the percentage of mothers who received postnatal care 
within 7 days of delivery (in percentage points)
LGAs
Before PS3
2013Q3 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q3 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -1.39 +5.22** +6.61** 0.000
– Phase I -0.39 +6.06** +6.45* 0.044
– Phase II -1.78 +4.88** +6.66** 0.001
Other +3.46** +3.19** -0.27 0.878
Difference (PS3 – Other) +6.88** 0.006
Difference (Phase I – Phase II) -0.21 0.957
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 184; PS3: 93 LGAs (Phase I: 26; Phase II: 67); Other: 91 LGAs. 
Figure 12 and Table 13 show the time trends and annualized change in the percentage of deliveries in which 
the mothers received postnatal care (PNC) within seven days of giving birth. From the end of 2013 until 
the start of 2015, PS3 regions had higher levels of PNC compared to other regions. Th ere was no signifi cant 
change in PNC prior to mid-2015 for PS3 regions, however, in other regions, the average annual increase of 
3.46 percentage-points was signifi cant. During the PS3 active period, PS3 regions had an annual increase in 
PNC of 5.22 percentage points, and other regions averaged a yearly increase of 3.19 percentage points. Th ere 
was a statistically signifi cant 6.61 percentage point increase in the growth rate of PNC in PS3 regions when 
comparing during and prior to PS3 implementation. Th e equivalent change in the growth rate of PNC in 
the other regions was not signifi cant. Comparing the two time periods, the change in the growth rate in PS3 
regions was signifi cantly higher than other regions, at 6.88 percentage points. Phase 1 areas had overall higher 
PNC levels than Phase 2 areas, but the trends over time followed a similar pattern (Figure B.9, Appendix B), 
and diff erences between the two implementation phases were not signifi cant. 
Overall improvements in this indicator can be linked to observed improvements in health facility delivery 
rates. Th e reasons for the diff erences between the two categories of regions are not apparent. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 41
Figure 13. Prevalence of low birth weight
Table 14. Annualized average change in percentage of newborns with low birth weights (in 
percentage points)
LGAs
Before PS3
2013Q3 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -1.89** -0.12 +1.76** 0.000
– Phase I -1.51* -0.55** +0.96 0.168
– Phase II -2.04** +0.06 2.10** 0.000
Other -2.11** +0.18 +2.30** 0.000
Difference (PS3 – Other) -0.53 0.413
Difference (Phase I – Phase II) -1.13 0.203
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 184; PS3: 93 LGAs (Phase I: 26; Phase II: 67); Other: 91 LGAs.
Table 13 and Figure 14 show the time trends and annualized change in the prevalence of infants born with 
a low birth weight (LBW). Figure 13 shows that, prior to PS3 initiation, LBW had declined from around 10 
percent in 2013 to close to 7 percent in the second quarter of 2015 in PS3 and other regions. Th e annualized 
change during this period was -1.89 and -2.11 percentage-points for PS3 and other regions, respectively. 
During the PS3 active period, there was very little change in the time trends or annualized change in this 
indicator. Th e only signifi cant diff erence was among Phase 1 PS3 LGAs, which declined from mid-2015 to the 
end of 2018 by 0.55 percentage points per year (Figure B.10, Appendix B). Th e rate at which LBW prevalence 
signifi cantly declined was similar, comparing before and after PS3 initiation for both areas. However, the LBW 
decline rates in PS3 and other regions did not signifi cantly diff er. 
42 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 14. Measles vaccination coverage
Table 15. Annualized average change in the percentage of children under one year of age who 
have been vaccinated against measles (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +7.57** -1.07** -8.64** 0.000
– Phase I +5.17 +2.27 -2.91 0.471
– Phase II -8.51** -2.40* -10.91** 0.000
Other +9.98** +2.56* -7.41* 0.013
Difference (PS3 – Other) -1.22 0.752
Difference (Phase I – Phase II) +8.00 0.108
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 171; PS3: 85 LGAs (Phase I: 23; Phase II: 62); Other: 86 LGAs. 
Figure 14 and Table 15 show the time trends and annualized change for the prevalence of measles vaccinations. 
For measles vaccination there was some volatility during 2014 and the fi rst part of 2015, but an overall 
increasing trend. Table 15 shows statistically signifi cant increases prior to mid-2015, with annualized changes 
of 7.57 percentage points for PS3 and 9.98 for other regions. A substantial drop in measles provision occurred 
nationwide in 2016. Phase 2 LGAs had higher levels of measles vaccinations than Phase 1 between mid￾2015 to mid-2016; however, by the fourth quarter of 2016, both Phase 1 and Phase 2 LGAs had similar 
levels (Figure B.11, Appendix B). Th is national drop was likely due to system-level supply problems. Measles 
vaccination provision recovered in early 2017 to high levels of close to 100 percent and the time trend returned 
to a relatively stable level during 2018. Th e time trends and volatility over time are similar in PS3 and other 
regions. During the PS3 active period, there was a decreasing trend reduced of -1.07 percentage points yearly in 
PS3 regions and an increasing trend of 2.56 percentage points in other regions. Diff erences in the annualized 
trends before and during PS3 did not signifi cantly diff er between PS3 and other regions.
In addition to the immunization commodity shortages already noted, immunization data may be prone 
to quality problems. Th ere are potential challenges with data accuracy (double-reporting) and appropriate 
denominators (for example, denominators for the health facility catchment area and outside areas, as used in 
the DHIS2), which complicates the interpretation of this and other immunization indicators. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 43
Figure 15. Penta3 vaccination coverage
Table 16. Annualized average change in the percentage of children under one year of age who 
have had Penta3 vaccinations (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +5.43** 0.19 -5.23* 0.012
– Phase I +6.03** +3.10 -2.93 0.226
– Phase II -5.17* -1.01 -6.18* 0.025
Other +6.15* +3.28 -2.86 0.454
Difference (PS3 – Other) -2.36 0.588
Difference (Phase I – Phase II) +3.25 0.377
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 171; PS3: 85 LGAs (Phase I: 23; Phase II: 62); Other: 86 LGAs. 
Figure 15 and Table 16 show the time trends and annualized average changes in the percentage of children 
under one year of age receiving the Penta3 vaccination. As with the other commodity-based indicators, Figure 
15 illustrates some volatility that may be related to supply shortages. Prior to the middle of 2015, PS3 and other 
regions had signifi cant annual average increases in Penta3 prevalence, except for Phase 2 LGAs (Figure B.12, 
Appendix B). Although PS3 regions achieved about 100 percent of children vaccinated in 2017, there was a 
sharp decline across all areas in 2018 that has resulted in very little average change over the PS3 active period 
for both PS3 and other regions. Th ese trends seemed to have occurred at a national level, as there was very little 
diff erence between the two time periods comparing PS3 and other regions. 
2017/2018 AHSP notes that there had been a shortage of immunization commodities, potentially explaining 
the drop seen in 2018. Th e report also notes the need to improve community sensitization and engagement for 
improved uptake of immunization services.
44 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 16. Antenatal care partners’ HIV testing rates
Table 17. Annualized average change in antenatal partners’ HIV testing rates (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +3.56 +7.34** +3.77 0.068
– Phase I +5.37 +8.54** +3.17 0.432
– Phase II +2.83 +6.84** +4.01 0.095
Other +7.81** +6.75** -1.06 0.528
Difference (PS3 – Other) +4.83 0.070
Difference (Phase I – Phase II) -0.84 0.858
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 184; PS3: 93 LGAs (Phase I: 26; Phase II: 67); Other: 91 LGAs. 
Figure 16 and Table 17 show the time trends and annualized average change in ANC partners’ HIV testing 
rates. Th e fi gure shows both PS3 and other regions increasing over time, with PS3 regions increasing at a 
higher level than other regions throughout the pre-program and program stages. Prior to mid-2015, other 
regions had averaged a greater annual increase, at 7.81 percentage points, than PS3 regions, at 3.56 percentage 
points. Th e annual increase prior to PS3 implementation was signifi cant for other regions, but not signifi cant 
for PS3 regions. During the PS3 active period, both PS3 and other regions had signifi cant average yearly 
increases, at 7.34 and 6.75 percentage points, respectively. Th e diff erences in the annual increases over time, 
comparing before and during the PS3 active period, were not signifi cant for either PS3 or other regions. 
Furthermore, the diff erence between the two areas and the two time periods was also not signifi cant. At the 
end of 2018, the average ANC partners’ HIV testing rate was just over 60 percent for PS3 regions, and about 
55 percent for other regions. Within PS3 regions, Phase 1 and Phase 2 followed similar patterns; however, the 
levels of this indicator for Phase 1 LGAs were higher, reaching almost 70 percent in late 2018, compared with 
Phase 2 LGAs, which averaged at just below 60 percent (Figure B.13, Appendix B).
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 45
Figure 17. HIV-exposed infants receiving fi rst HIV test within 2 months after birth
Table 18. Annualized average change in percentage of HIV-exposed infants receiving fi rst HIV test 
within 2 months after birth (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +20.08** -2.70** -22.78** 0.000
– Phase I +19.04** -2.48 -21.52** 0.000
– Phase II +20.50** -2.79** -23.29** 0.000
Other +38.19** -2.64* -40.83** 0.000
Difference (PS3 – Other) +18.06** 0.000
Difference (Phase I – Phase II) +1.78 0.751
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 181; PS3: 92 LGAs (Phase I: 26; Phase II: 66); Other: 89 LGAs. 
Figure 17 and Table 18 show the time trends and annualized average change in the percentage of infants 
exposed to HIV who received their initial HIV test within two months after birth. At the beginning of 2014, 
both PS3 regions and other regions had levels of this indicator under 20 percent. Th e percentage of exposed 
infants who were tested rose through mid-2015 for both groups, however, this change was more rapid in other 
regions, with an annualized average increase of 38.19 percentage points, compared with 20.08 percentage 
points for PS3 regions. After PS3 began, the yearly average growth for this indicator showed that both PS3 
and other areas declined from mid-2015 until the end of 2018 at similar but smaller rates, around 2.6–2.7 
percentage points per year. Compared to before and during the program, the diff erence between the growth 
rates was signifi cant for both areas, with PS3 regions’ growth rates being 22.78 percentage points per year 
lower during the PS3 active period than before mid-2015, and 40.83 percentage points per year lower in other 
regions. Th e diff erence between the two groups over the two time periods was 18.06 percentage points, which 
also is a statistically signifi cant diff erence. Th e reasons for these diff erences are not apparent. In terms of PS3 
implementation, Phase 2 areas had overall slightly higher levels than Phase 1 areas, but both followed similar 
time trend patterns (Figure B.14, Appendix B). 
46 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 18. HIV-exposed infants initiated on cotrimoxazole within 2 months after birth
Table 19. Annualized average change in the percent of HIV-exposed infants initiated on 
cotrimoxazole within 2 months after birth (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +22.4** -3.62** -26.02** 0.000
– Phase I +17.83** -3.82* -21.65** 0.000
– Phase II +24.29** -3.54** -27.82** 0.000
Other +39.88* -2.68* -42.56** 0.000
Difference (PS3 – Other) +16.54** 0.000
Difference (Phase I – Phase II) +6.17 0.221
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 171; PS3: 89 LGAs (Phase I: 26; Phase II: 63); Other: 82 LGAs. 
Figure 18 and Table 19 show the time trends and annualized average change in the percentage of HIV-exposed 
infants on cotrimoxazole within two months after birth. Th e data for this indicator follow a similar pattern 
as the previous indicator on testing of HIV-exposed infants. At the beginning of 2014, both PS3 and other 
regions had levels of this indicator under 20 percent. Th e percentage of exposed infants who were tested 
rose through mid-2015 for both groups, however, this change was more rapid among other regions, with an 
annualized average change at 39.88 percentage points, compared with 22.4 percentage points for PS3 regions. 
After PS3 began, the yearly average showed that both areas fell from mid-2015 through 2018 at similar rates, 
at 3.62 and 2.68 percentage points per year for PS3 and other regions, respectively. Compared to before and 
during the program, the diff erence between the growth rates was signifi cant for both areas, with PS3 regions’ 
growth rate diff erences at -26.02 percentage points during the PS3 active period compared with before mid￾2015, and -42.56 percentage points in other regions. Th e diff erence between the two groups over the two time 
periods was 16.54 percentage points, which was also statistically signifi cant. Th e reasons for these diff erences 
are not apparent. In terms of PS3 implementation, Phase 2 areas had overall slightly higher levels than Phase 1 
areas, but both followed similar time trend patterns (Figure B.15, Appendix B). 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 47
Figure 19. Percentage of couples receiving HIV counseling and testing at antenatal care
Table 20. Annualized average change in percentage of couples receiving HIV counseling and 
testing at antenatal care (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +10.71** +7.29** -3.42** 0.007
– Phase I +12.58** +8.29** -4.28 0.119
– Phase II +9.96** +6.87** -3.08* 0.027
Other +11.38** +8.58** -2.80 0.102
Difference (PS3 – Other) -0.62 0.772
Difference (Phase I – Phase II) -1.20 0.695
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 184; PS3: 93 LGAs (Phase I: 26; Phase II: 67); Other: 91 LGAs. 
Figure 19 and Table 20 show the time trends and annualized average change in the percentage of couples 
who received HIV counseling and testing at ANC. Figure 18 shows that, throughout the pre-program and 
PS3 active periods, PS3 regions had higher levels of couples receiving HIV counseling and testing than other 
regions. Th is mirrors the partners’ HIV testing rate indicator shown in Figure 16. In PS3 regions, there was an 
average annualized growth of 10.71 percentage points, and 11.38 percentage points in other regions, prior to 
the program. During the PS3 active period, the growth rate for PS3 regions signifi cantly fell 3.42 percentage 
points to a rate of 7.29 percentage points per year. In other regions, the growth rate also fell by 2.80 percentage 
points to 8.58 percentage points per year, however this result was not signifi cant. Th ere was no signifi cant 
diff erence in the change in rates from before and during the program between the two areas. Within the PS3 
areas, Phase 1 levels of this indicator were consistently slightly higher than those of Phase 2, and Phase 2 levels 
were consistently slightly higher than those of other areas (Figure B.16, Appendix B.) 
Th e overall increase in both PS3 and other regions refl ects a general perception that partner attendance at 
ANC has increased, which has resulted in increases in couples’ decision-making and partner testing. 
In the past, women were attending clinics without their husbands. Th e parents did not share their problems and 
most men believed that it was not their duty to accompany their wives to the hospital. But, today, men are educated 
and are ready to accompany their wives and sometimes undergo medical examination. Men and women can now sit 
together and make decisions on crucial issues like that. [FGD with women, other]
48 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 20. Contraceptive prevalence rate
Table 21. Annualized average change in contraceptive prevalence rate (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q4 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +0.36 -0.69 -1.05 0.362
– Phase I +0.92 -0.84 -1.75 0.286
– Phase II +0.11 -0.62 -0.74 0.621
Other +0.14 -0.80 -0.94 0.529
Difference (PS3 – Other) -0.11 0.955
Difference (Phase I – Phase II) -1.02 0.646
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 175; PS3: 87 LGAs (Phase I: 25; Phase II: 62); Other: 88 LGAs. 
Figure 20 and Table 21 show the time trends and annualized average changes in the contraceptive prevalence 
rate (CPR). PS3 regions had a higher average CPR compared to other regions both before and during PS3 
implementation. Th ere were no signifi cant annualized changes in the CPR over time or between the two 
groups. Within PS3 regions, the trends in Phase 1 areas were similar to the trends in Phase 2 areas (Figure 
B.17, Appendix B.) Reasons for variations between the PS3 and other regions are not apparent. 
Traditional gender norms and expectations were discussed as a challenge in contraceptive prevalence and 
family planning. 
Now the challenges we face is the small number of those using family planning methods and when you go to the 
villages and advertise them the men will not allow their women to attend the clinics to get those, if only the women 
had the decisions on their own it would have been easy. [KII with DMO, Phase 1]
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 49
Figure 21. Health facilities with RCH tracer drugs package
Table 22. Annualized average change in the percentage of health facilities with RCH tracer drugs 
package (in percentage points)
LGAs
Before PS3
2014Q1 to 
2015Q2 (1)
PS3 active
2015Q3 to 
2018Q3 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +11.00** +4.36** -6.63** 0.000
– Phase I +11.18** +3.86** -7.32** 0.000
– Phase II +10.92** +4.57** -6.35** 0.000
Other +9.82** +4.03** -5.79** 0.000
Difference (PS3 – Other) -0.84 0.645
Difference (Phase I – Phase II) -0.96 0.651
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: 184; PS3: 93 LGAs (Phase I: 26; Phase II: 67); Other: 91 LGAs. 
Figure 21 and Table 22 show the time trends and annualized average change in the percentage of health 
facilities with the RCH tracer drug package. Until 2015, there was a sharp increase in the percent of facilities 
with the package for both PS3 and other regions, followed by maintaining some consistency in early 2015. 
After PS3 started, there was a steep drop in this indicator that reached a minimum level in mid-2016. By the 
end of 2017, health facilities in both PS3 and other regions had recovered and exceeded 2015 levels. By mid￾2018, almost 90 percent of health facilities country-wide had the drug package. Th e indicators over time for 
both PS3 and other areas did not signifi cantly diff er. Within PS3 areas, there were no diff erences between 
trends in Phase 1 and Phase 2 LGAs (Figure B.18, Appendix B). 
3.2. Quantitative Evaluation: Evaluation Question 2
Th is section provides answers to evaluation question 2 and expands the answer for question 3 for fi nancial and 
human resources: 
 Evaluation question 2: In PS3 focus regions, to what extent have fi nancial and human resources 
indicators changed over time?
 Evaluation question 3: Are there diff erences in the changes over time observed in certain groups of 
LGAs, in particular between Phase 1 and Phase 2 LGAs?
50 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Questions 2 and 3 are answered examining the time trends of key fi nancial and human resources indicators in 
PS3 regions and in the other (non-PS3) regions of the country. Th e graphs in this section present the trajectory 
of the annual weighted average of fi nancial and human resources indicators for the LGAs in those two groups. 
Time series fi nancial data were obtained from the council fi nancial reports prepared by each LGA of the 
country. Time series human resources data came from diff erent sources as indicated in section 2.1.3 above. We 
have also included, where applicable, qualitative information gathered on the context and perception of these 
trends. Additional qualitative results are presented in section 3.3.
3.2.1. Financial Resources Indicators
Figure 22. Personnel emoluments per capita in health
Table 23. Average annual change in personnel emoluments per capita in health, in TZ shillings 
per person
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2017 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -316.20 +332.65 +648.85* 0.044
– Phase I -173.52 -96.58 +76.94 0.889
– Phase II -342.31 +507.63 +849.93* 0.029
Other +783.97 +523.61 -260.36 0.629
Difference (PS3 – Other) +909.21 0.147
Difference (Phase I – Phase II) -772.99 0.252
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=166; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Figure 22 and Table 23 show the time trends and annualized change in annual personnel emoluments (PE) 
per capita in health. PEs in health are the total gross salary received by all civil servants and service providers 
working in health services/facilities in the LGA and marked as such in the council fi nancial reports (CFR) 
prepared by each LGA. Th e fi gures used in this report are estimates obtained by adding the PE amounts from 
four quarterly CFRs reported by the LGAs. Population estimates for each LGA were obtained from population 
projections published by the National Bureau of Statistics (NBS). 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 51
Overall, PS3 and other regions show similar levels and time trends in PE per capita in health. With the 
exception of fi scal year 2014–15, this indicator increases every year of the observation period. Th e positive 
trend was maintained during the time PS3 was active. Prior to 2015, levels of this indicator increased from 
below TSh 6,000 per capita to around TSh 9,000 for both groups. During this time, there was a sharp decline 
in 2014, particularly for the PS3 regions, followed by a quick recovery. Th ere was no signifi cant diff erence in 
the annual average change in this indicator prior to 2015. After 2015, PS3 and other areas experienced similar 
levels of PE per capita in health, which increased from 2015–2016, and appear to be steady from 2016–2017. 
However, in PS3 regions, the diff erence in the average annual change in this indicator was signifi cant, when 
comparing the two time points, largely due to signifi cant changes seen in PS3 Phase 2 LGAs. Comparing PS3 
and other regions, there was no diff erence between these areas in the average annual change in PE per capita in 
health across the two time periods (Figure B.19, Appendix B). 
Figure 23. Personnel emoluments per capita in education
Table 24. Average annual change in personnel emoluments per capita in education, in TZ Shillings 
per person
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2017 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -704.69 +4,481.45** +5,186.14* 0.018
– Phase I +1,810.96 +3,029.31 +1,218.35 0.787
– Phase II -1,422.36 +5,078.98 +6,501.34** 0.008
Other +3,256.14* +3,043.04* -213.11 0.902
Difference (PS3 – Other) +5,399.24 0.054
Difference (Phase I – Phase II) -5,282.99 0.302
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=166; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
52 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 23 and Table 24 show the time trends and annualized change in PE per capita in education. 
From fi scal year 2012–13 to 2014–15, levels of this indicator appeared to stay steady around TSh 30,000 
per person, followed by an increase to above TSh 40,000 for both PS3 and other areas. Th e increases 
continued past 2015, maintaining steady levels from fi scal years 2016–17 to 2017–18 for both areas, 
although the average PE per capita for PS3 regions was above that of other areas. During the period of PS3 
implementation, there were signifi cant diff erences seen in the average annual change in PE per capita in 
education in both PS3 and other areas, but these were not signifi cant when stratifying LGAs in PS3 regions 
by implementation phase. Comparing PS3 and other regions, there was no diff erence between these areas in 
the average annual change in PE per capita in education across the two time periods. Trends and annualized 
average changes in PE per capita in education were also similar when comparing Phase 1 and Phase 2 LGAs 
(Figure B.20, Appendix B). 
Figure 24. Other charges per capita in health
Table 25. Average annual change in other charges per capita in health, in TZ Shillings per person
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2017 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -244.52** +9.94 +254.46** 0.000
– Phase I -393.71** +19.45 +413.16** 0.004
– Phase II -187.34** +5.57 +192.91** 0.009
Other -300.82** +96.73 +397.54** 0.000
Difference (PS3 – Other) -143.08 0.269
Difference (Phase I – Phase II) +220.25 0.170
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=166; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Figure 24 and Table 25 show the time trends and annualized change in other charges (OCs) per capita in 
health. OCs are other non-payroll charges fl owing through either LGA or service provider bank accounts in 
the health sector of the LGA and marked as such in the CFRs prepared by the LGAs. Th e fi gures used in 
this report are estimated by adding the OC amounts from four quarters reported by each LGA. Population 
estimates for each LGA were obtained from population projections published by the NBS. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 53
Prior to 2015, there was a drop in the OCs per capita in health from 2013–2014. Th e decline continued 
slightly for PS3 regions and remained relatively steady for other areas until 2016. From 2016–2017, there was 
a slight increase in this indicator for both areas, however PS3 regions had lower levels than other areas. Th e 
average annualized change in OCs per capita in health declined signifi cantly before PS3 implementation 
until 2015. After 2015, there were no signifi cant diff erences in the average annualized changes for any area. 
Comparing across the two time periods, the average annualized change was signifi cantly higher for both 
PS3 and other regions after implementation. However, the diff erences between the two areas across the 
two periods were not signifi cantly diff erent. Stratifying PS3 areas by implementation phase reveals similar 
patterns for both Phase 1 and Phase 2 LGAs (Figure B.21, Appendix B).
Figure 25. Other charges per capita in education
Table 26. Average annual change in other charges per capita in education, in TZ Shillings per person
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2017 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -1,5575.64** -221.66 +1,353.98** 0.000
– Phase I -1,329.78** -204.27 +1,125.51** 0.000
– Phase II -1,652.90** -229.20 +1,423.69** 0.000
Other -841.01** -103.80 +737.21** 0.000
Difference (PS3 – Other) +616.76* 0.043
Difference (Phase I – Phase II) -298.18 0.481
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=164; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Figure 25 and Table 26 show fi ndings for OCs per capita in education. From 2012–2014, levels of this 
indicator for both PS3 and other regions declined, and then remained steady from 2014–2015. Th e average 
annualized changes in OCs per capita in education show statistically signifi cant declines in all areas. After 
PS3 was implemented in 2015, there were no signifi cant diff erences in the average annualized change in any 
area, including when stratifi ed by implementation phase (Figure B.22, Appendix B). Th is stable level is seen in 
Figure 25, which shows little change in the levels of OCs per capita in education from 2015–2017. However, 
the diff erences between the two time periods and the two areas was signifi cant. In PS3 areas, the rate of 
decline lessened by TSh 616.76 more than the rate of decline in other areas. 
54 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
3.2.2. Human Resources Indicators
Figure 26. Nurses per population
Table 27. Average annual change in the nurses per population ratio (in nurses per 10,000 people)
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2018 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +0.46** +0.035 -0.43** 0.000 
– Phase I +0.47** +0.10* -0.36** 0.000
– Phase II +0.46** +0.007 -0.46** 0.000
Other +0.45 ** -0.025 -0.48** 0.000
Difference (PS3 – Other) +0.05 0.480 
Difference (Phase I – Phase II) +0.09 0.338
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=166; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Figure 26 and Table 27 show the time trends and average annual change in the number of nurses per 10,000 
people in the population. Prior to 2015, both PS3 and other areas were increasing at a similar average annual 
rate of 0.45 nurses per 10,000 people. After 2015, these ratios stabilized in both types of region at a level close 
to six nurses per 10,000 people. Both types of region follow a clear national trend in this indicator with no 
diff erences being detected between the regions. Similar patterns in time trends were observed in PS3 Phase 1 
and Phase 2 LGAs (Figure B.23, Appendix B). 
Community members and key informants in all regions discussed concerns around perceived shortages of 
health facility staff and the subsequent eff ects on service quality and health outcomes. Th is shortage was 
commonly discussed within the national context of recent certifi cation rules, whereby staff in various sectors 
were laid off for being unqualifi ed.
[One challenge] is the inadequate workers/employees after the issue of certifi cates verifi cation, so new recruits have 
been posted and soon are going to report. [KII with RMO, other]
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 55
Figure 27. Doctors per population
Table 28. Average annual change in the doctors per population ratio (in doctors per 10,000 people)
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2018 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +0.028** +0.028** 0.000 0.984
– Phase I +0.027** +0.033** +0.006 0.476
– Phase II +0.029** +0.026** -0.003 0.783
Other +0.041** +0.019 -0.022 0.096
Difference (PS3 – Other) +0.022 0.147
Difference (Phase I – Phase II) +0.009 0.496
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=166; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Figure 27 and Table 28 show the time trends and average annual change in the number of doctors (MDs) 
per 10,000 people in the population. Th roughout the entire time period, on average, other regions had higher 
levels of doctors per population ratio than PS3 regions. Both groups of regions have followed similar increasing 
time trends. Prior to 2015, there was a slight increase of 0.028 and 0.041 doctors per 10,000 people per year in 
PS3 regions and other regions, respectively. After 2015, the average annual increase for PS3 regions was almost 
the same and remained signifi cant. Other regions also maintained a small increasing rate. Th ere were no 
diff erences between the time trends of PS3 Phase 1 and Phase 2 LGAs. Th ey followed patterns similar to the 
overall trend (Figure B.24, Appendix B).
56 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 28. Assistant medical offi cers per 10,000 people
Table 29. Average annual change in assistant medical offi cers per population ratio 
(per 10,000 people)
LGAs
Before PS3
2012 to
2015 (1)
PS3 active
2016 to
2018 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -0.008** -0.006** 0.002 0.384
– Phase I -0.003 -0.003** -0.0001 0.933
– Phase II -0.010** -0.007** +0.003 0.342
Other -0.006 -0.010** -0.005** 0.006
Difference (PS3 – Other) +0.007* 0.016
Difference (Phase I – Phase II) -0.003 0.369
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=166; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Figure 28 and Table 29 show the time trends and average annual change in the number of assistant medical 
offi cers (AMOs) per 10,000 in the population. Th ere was a slight decrease in the number of AMOs per 10,000 
in the population throughout 2012–2018 in both areas. Th e small changes in the rates between PS3 areas and 
other regions were statistically signifi cant, with the rate decreasing less, by 0.007 AMOs per 10,000 people 
annually, in PS3 areas. Comparing PS3 phases, Phase 2 LGAs had time trend patterns almost identical to 
other areas. Phase 1 LGAs had similar trends but at a lower level (Figure B.25, Appendix B).
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 57
Figure 29. Health care workers that are female
Table 30. Average annual change in the proportion of health care workers that are female 
(in percentage points)
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2018 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -0.58** -0.67** -0.09 0.503
– Phase I -0.43** -0.81** -0.38 0.103
– Phase II -0.64** -0.61** -0.03 0.859
Other -0.46** -0.47** -0.01 0.964
Difference (PS3 – Other) -0.08 0.678
Difference (Phase I – Phase II) -0.41 0.151
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=166; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Gender indicators were included in this evaluation as gender integration is a component of the PS3 
intervention. Figure 29 and Table 30 show time trends and average annual changes in the proportion 
of health care workers (HCWs) that are female. Across time, there was a slight decline in this indicator, 
particularly in PS3 areas, which could be interpreted as moving towards gender equity, but the exact reasons 
are unknown. In 2015, about 65 percent of HCWs were female in PS3 areas, and almost 68 percent were 
female in other areas. Th e rate of change did not change after 2015. In terms of implementation phases, 
patterns for Phase 1 and Phase 2 seem to refl ect the overall trends, however, Phase 1 LGAs had generally 
lower levels of female HCW participation than Phase 2 LGAs (Figure B.26, Appendix B). 
58 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 30. Primary school student-teacher ratio
Table 31. Average annual change in the primary school student-teacher ratio (in students per teacher)
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2018 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -2.01** +5.30** +7.31** 0.000
– Phase I -1.98* +6.68** +8.65** 0.000
– Phase II -2.04** +4.76** +6.80** 0.000
Other -1.54** +4.92** +6.46** 0.000
Difference (PS3 – Other) +0.85 0.322
Difference (Phase I – Phase II) +1.85 0.088
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=162; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Figure 30 and Table 31 show the time trends and average annual change in the student-to-teacher ratio for 
primary schools in PS3 areas and other regions. Th e trends were similar in both groups of regions. Th ere were 
similar decreases in the student-teacher ratio prior to 2015, and an overall increase between 2015 and 2018. 
PS3 regions consistently had higher levels of this indicator than other regions. Prior to 2015, the rate of decline 
in PS3 areas was by about two students annually, and about 1.5 students per year in other regions. After 
2015, the student-teacher ratio increased by about fi ve students per year in PS3 and other areas. Th ere was no 
signifi cant diff erence in the change in rate when comparing the two areas. Phase 1 and Phase 2 LGAs followed 
similar patterns, with Phase 1 LGAs consistently at higher levels of student-teacher ratios than Phase 2 LGAs 
(Figure B.27, Appendix B). 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 59
Figure 31. Secondary school student-teacher ratio
Table 32. Average annual change in the secondary student-teacher ratio (in students per teacher)
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2018 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 -3.88** +3.03** +6.91** 0.000
– Phase I -4.51** +3.34** +7.85** 0.000
– Phase II -3.62** +2.90** +6.52** 0.000
Other -2.77** +2.04** +4.81** 0.000
Difference (PS3 – Other) +2.10 0.055
Difference (Phase I – Phase II) +1.33 0.414
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=164; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Figure 31 and Table 32 show the time trends and average annual changes in the student-to-teacher ratio for 
secondary schools. Th e fi gure shows similar levels and time trends of student-teacher ratios in PS3 and other 
areas across all time periods. From 2012–2015, there was a steady decline in this indicator. In PS3 areas, the 
student-to-teacher ratios declined by 3.88 students per year, and by 2.77 students in other regions annually, 
prior to 2015. After 2015, student-teacher ratios increased similarly, which was not signifi cant at the 5 percent 
cutoff . Th e trends for LGAs in both PS3 phases followed similar patterns (Figure B.28, Appendix B). 
Qualitative results showed that teacher shortages were seen as a problem locally. Th is occurred simultaneous to 
free-education polices that may have resulted in higher enrollment, as discussed qualitatively.
I will talk about the issue of education, for the past three years up to now there are a lot of changes. Th ere was laziness 
especially to the people who are living beside the town, the children they were not going to school and just stay at home 
looking after livestock. But after the issue was recognized by leaders, now the children are attending school through 
this system of free education. Now a lot of people get education, for example in my school there a lot of students until 
the classes aren’t enough for all the students. [FGD with men, Phase 2]
60 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
“What I can say is, we managed to encourage parents to enroll their children, but we did not consider the issue of 
classes. Our school has very few classes. Save, there are 209 pupils in standard three but there are two classes only. 
We all know that the maximum number of students for each class is 45. In this size, how can pupils understand 
their teacher?” [FGD with women, other]
Th e concern with community members was that while education is free, the perception of the quality and 
performance of these services was aff ected.
Th ere are so many students, but few teachers. You will fi nd that there are 100 students in a class when the average 
proposed number is 45 students. Th ere are classes which are packed with up to 170 students. Th at decreases 
performance. [FGD with men, other]
Figure 32. Primary school students that are female
Table 33. Average annual change in the proportion of primary school students that are female, 
in percentage points.
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2018 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +0.12** -0.26** -0.37** 0.000
– Phase I +0.17** -0.30** -0.47** 0.000
– Phase II +0.10** -0.24** -0.34** 0.000
Other +0.12** -0.28** -0.40** 0.000
Difference (PS3 – Other) +0.03 0.677
Difference (Phase I – Phase II) -0.12 0.295
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=162; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 61
Figure 32 and Table 33 show the time trends and average annual changes in the proportion of primary 
students that are female. Across the entire observation time period, both PS3 and other regions show similar 
levels of female students in primary school, at around 50 percent, which indicates gender parity in primary 
school participation. PS3 regions show only a slightly higher percentage. Th ere is very little variation in that 
level over time. Stratifying by PS3 implementation phases, we observe that Phase 1 and Phase 2 follow the 
same pattern shown in PS3 regions overall (Figure B.29, Appendix B). Estimation of the average annual 
change in this indicator reveals only small changes and of similar magnitude in both regions. 
Figure 33. Secondary school students that are female
Table 34. Average annual change in proportion of secondary school students that are female, 
in percentage points
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2018 (2)
Difference
(2)-(1)
P-value of 
difference
PS3 +1.16** +0.71** -0.44* 0.021
– Phase I +1.35** +0.97** -0.38 0.282
– Phase II +1.07** +0.61** -0.46* 0.046
Other +1.35** +0.76** -0.59** 0.006
Difference (PS3 – Other) +0.15 0.612
Difference (Phase I – Phase II) -0.08 0.858
** Signifi cant at < 1%; * signifi cant at <5%
Number of LGAs: Before PS3=165; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
A diff erent situation is found in secondary schools, where there have been important gains in the participation 
of girls during the period from 2012 to 2018. Figure 33 and Table 34 show signifi cant increases in the 
proportion of secondary students that are female—from a level of about 44 percent in 2012, to a level around 
50 percent by 2018—indicating that parity between genders was achieved. Similar levels and increasing trends 
are found between the PS3 phases (Figure B.30, Appendix B). Table 34 confi rms the similarity of PS3 and 
other regions in the positive annual change in the indicator.
62 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure 34. Primary school teachers that are female
Table 35. Average annual change in proportion of primary school teachers that are female, 
in percentage points.
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 2018
(2)
Difference
(2)-(1)
P-value of 
difference
PS3 +7.12* -0.38** -7.50* 0.010
– Phase I +19.95** -0.30* -20.26** 0.000
– Phase II +2.31 -0.41** -2.72 0.454
Other -11.56* -0.52 +11.03* 0.043
Difference (PS3 – Other) -18.54** 0.003
Difference (Phase I – Phase II) -17.53** 0.000
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=162; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Given the gender parity achieved among primary school students, it is important to examine the gender 
composition among teachers. Figure 34 and Table 35 show the time trends and average annual changes in 
the proportion of primary school teachers that are female. During the entire observation time period, both 
PS3 and other regions show levels close to gender parity among primary teachers. PS3 regions show levels 
lower than other regions in most years, with the exception of 2013, but at over 45 percent. Examining the 
patterns over time by PS3 phases, we observe fl uctuations in PS3 Phase 1 LGAs while Phase 2 LGAs remain 
remarkably stable, at a level of 50 percent (Figure B.31, Appendix B). Th ese two diff erences—between PS3 and 
other areas, as well as Phase 1 and Phase 2 LGAs—over the observed periods were both statistically signifi cant. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 63
Figure 35. Secondary school teachers that are female
Table 36. Average annual change in proportion of secondary school teachers that are female, 
in percentage points
LGAs
Before PS3
2012 to 
2015 (1)
PS3 active
2016 to 
2018(2)
Difference
(2)-(1)
P-value of 
difference
PS3 +1.68** -0.87** -2.56** 0.000
– Phase I +1.53 -0.74** -2.27** 0.005
– Phase II +1.75** -0.92** -2.67** 0.000
Other +1.00* -0.62** -1.61** 0.000
Difference (PS3 – Other) -0.94 0.072
Difference (Phase I – Phase II) +0.40 0.654
Signifi cance levels: ** at 1%; * at 5%.
Number of LGAs: Before PS3=165; PS3 active=184, 93 LGAs (Phase I: 25; Phase II: 68), Other: 91 LGAs. 
Figure 35 and Table 36 show the time trends and average annual changes in the proportion of secondary 
school teachers that are female. Th ere is a clear gender disparity in the composition of secondary teachers. 
In PS3 regions, there were increases in levels of secondary school teachers that are female, from a level of 
25 percent in 2012 to 30 percent in 2016 and 2017, but levels declined in 2018. Consistently higher levels 
of female participation, of at least fi ve percentage points on average, were observed in other regions. PS3 
Phase 1 LGAs show lower female participation among secondary teachers than Phase 2 across all years of the 
observation period. Both areas show similar time trends (Figure B.32, Appendix B).
3.3. Qualitative Results
Additional qualitative results are presented here in terms of the three relevant evaluation questions, and they 
were in line with the main components of the PS3 program, namely: governance and civic engagement, 
fi nance, human resources, and information systems. We found very few discernible diff erences between zones 
(Eastern Lake versus Central and East Southern Highlands) and no diff erences between old and new LGAs. 
We present diff erences between PS3 regions (Phases 1 and 2) and other regions as applicable.
64 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
3.3.1. Evaluation Question 4
Th ese subsections provide the qualitative fi ndings for evaluation question 4: How do stakeholders perceive the 
performance and infl uence of PS3 on health services, fi nancial and human resource systems, and community 
engagement and governance?
In general, and across PS3 and other regions, stakeholders perceived improvements in governance and civic 
engagement, fi nance, and human resources, as well as the information systems that support these components. 
3.3.1.1. Community Engagement and Governance
Community members in PS3 areas and other regions discussed perceived improvements in their ability 
to communicate with local leaders. Th is was in part through a greater understanding and expansion of 
communication channels. Procedures for communication were made clear especially through public meetings, 
notice boards, and in some occasions through suggestion boxes and complaint registers established at service 
facilities. Conversely, LGA websites, such as those established by PS3, were not known or commonly used 
by community members. Physical presence and engagement of community leaders with citizens was also 
perceived to have improved, in both control and intervention areas. 
…it is true that even our leaders today have come back to the rural areas. I’ve worked 13 years in the district, over 
fi ve years ago I had never found a director coming up to people or district offi cials coming...I had never seen, but in 
this case I’ve seen two, three, or four district leaders coming and talking to the elders and talking to the people, so the 
participation has become closer, the leadership has left chairs and are now moving around to the people. 
[FGD with male community members, other]
Local leaders and LGA councilors also refl ected this sentiment, describing a reduction in the gap between 
government and citizens. In intervention areas, LGA councilors credited the improvement to the trainings they 
received from PS3. 
Our behavior has made citizens voice out their opinions, before the training we used to dictate, and citizens were 
afraid and uncomfortable around us we had no cooperation. But after the training citizens see us as a part of their 
community. [REM with LGA councilors, Phase 1]
Th is improved engagement between community members and local government has increased understanding 
in the value of engaging with the government for development. 
…in the past calling citizens for meetings was a big issue, most of them didn’t see the reason to come and the few who 
came were either drunk or disinterested, but due to PS3 the citizens have seen the value of collaborating with their 
leaders to move the development agenda… [FGD with male community members, Phase 2]
Th is was also refl ected by LGA councilors, who felt greater participation from community members both was 
a cause and consequence of development projects, such as construction of infrastructure (schools and health 
facilities) and stocking of those facilities (e.g., with desks). 
Good relations have helped to be in a good position to combat diff erent challenges. For example, there was a time 
period when there was a shortage of desks at schools. I called a meeting that involved diff erent levels of people 
in terms of their occupations or professions. We had businessmen, farmers, animal keepers, etc. We received 20 
desks courtesy of their contributions. We have been through thick and thin in health and education, but we have 
always managed to pull through diffi cult obstacles like that thanks to that kind of collective eff orts. 
[REM with LGA councilors, Phase 1 – Bahi]
One community member even discussed engaging with local government to specifi cally address issues in 
maternal and child health.
Frankly speaking, I was convinced to engage with the local government in order to have chance to visit women 
and under-fi ve years children to sensitize them. As I have said earlier, during the past years they were not attend 
the RCH services of which I then decided to engage with the local government in to help my fellow women who 
have children, pregnant women in order to address that challenge to reach success at the end. 
[FGD with female community members, other]
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 65
In addition to development projects in communities, stakeholders perceived that increased citizen engagement in 
both control and intervention areas has improved transparency in designing, budgeting, and implementing projects. 
In these last three years, citizens participate in decision making and they have power to criticize their government. 
Nowadays, people can present their complaints and make follow-up on whatever matters approved by the 
government. Take an example, if we donate some money for construction purposes, citizens can now monitor the 
expenditure and progress of the program. Decisions are made from the lowest level to the highest level… 
[FGD with female community members, other]
Overall, citizens expressed more comfort in voicing their opinions and holding their local leaders accountable. 
Councilors, in both control and intervention areas, expressed some level of knowledge and awareness of 
community members’ complaints. However, REM and KIIs with LGA councilors found higher levels of 
knowledge on governance issues in terms of how they were prepared to respond to these complaints and issues. 
One illustrative diff erence is in understanding land and fi nance laws and responding to confl icts. 
When discussing eff ects of PS3 trainings on LGA councilors, one REM group discussed how trainings 
improved their governance by learning about land and fi nance laws. Th e trainings also enabled them to 
engage with citizens through appropriate channels to better understand confl icts between pastoralists and 
agriculturalists. Th ese two components allowed the councilors to manage and plan for better use of their 
land management plan and solve land confl icts when they emerge (Figure 36).
Figure 36. Excerpt from REM Map with LGA councilors, Phase 2 – Kilosa
When asked about controlling land issues in a non-PS3 area LGA, one councilor remarked that, to his 
knowledge, there is not a control system on land service and proper supervision is needed. Additionally, there 
seemed to be a gap between the bureaucracy and citizen expectations. 
Th e foremost challenge is that citizens are not educated on issues like compensation, land survey and its costs are 
unbearable. [KII, LGA councilor, other]
When asked what a “new program for assisting in public systems control could do” to support the councilors, 
he responded:
First is transparency, second is to inform on what is going to be done, and through the two issues citizens may 
be convinced to accept the system. Another is to create the friendly environment for the citizens to accept the 
systems. Citizens should be informed about the ongoing issues and understand 1...2...3, should be informed 
about the sources of funds and its spending, they will increase their support in any case their money is used. 
[KII, LGA councilor, other]
Th e new program described above refl ects many of the positive attributes of the PS3 trainings detailed in 
interviews and group discussions with stakeholders. Intervention area councilors detailed greater capacity to 
engage citizens in addressing land issues as part of their roles and responsibilities. 
We learned about our responsibilities as councilors and how to administer people at the ward level, something that we 
were not aware of in the past. Th is has helped reduce confl icts and confrontations during meetings. [REM, Phase 1]
While Bahi received greater support in addressing citizen complaints, we found no discernible diff erence in 
perceptions of performance among intervention areas. 
66 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
As part of their roles and responsibilities, LGA councilors act as a bridge between citizens and various levels of 
offi cials, ranging from civil servants to government offi cials, such as the District Executive Director (DED), 
and coordinate with other wards. Across all areas, councilors’ relations with other leaders seemed to be 
improving. Councilors who were engaged through the REM method commented that they were improving in 
their performance, especially in the understanding of their responsibilities and administrative hierarchies.
Cooperation between councilors and civil servants at the district council is vital. We were told that District council 
is under councilors so a bad relationship with servants will limit development, after the training we have established 
good relationship and we now rectify mistakes by following principles.” [REM with LGA councilors, Phase 2]
“PS3 training has helped to harmonize the relationship between us councilors and DED; before the two were like 
enemies. Th e training helped councilors know their administrative limits towards the DED…now we know there is 
a hierarchical procedure to follow. [REM with LGA councilors, Phase 2]
Perceptions of LGA public service systems and management performance were reported as positive in 
comparison to the past. Across both PS3 and other areas, key informants and councilors described LGA 
management systems as having improved various responsibilities, including managing council funds both at 
council and lower levels. Community members shared similar thoughts, reporting satisfaction with how 
their leaders have been managing the funds compared to the past and how they have been responding to the 
complaints. Th is was presented in terms of satisfaction with the transparency and services provided. For 
example, one community member expressed:
In health facilities, let’s say the dispensary we know the fund that has been brought and how it is used, because the 
fi nancial reports are normally shared during the village assembly meetings and also when there is weakness in the 
services provided we know the channel to follow in reporting…this way the services have improved to a greater extent... 
[FGD with female community members, other]
LGA councilors and other GOT stakeholders echoed this sentiment, noting systems have improved 
effi ciencies, accountability, and performance through managing council funds, reporting progress and 
challenges happening at lower levels, and improved accessibility and availability. 
...thanks to these systems, citizens now can be served on time compared to the previous time. If someone needs a 
driving license, provided that their information is already fed on the system, they will get them within a very short 
time. Th e service is available all the time and uses very short time to serve many people...as I have just said, it 
simplifi es the provision of services to the public. [KII with ICT offi cer, Phase 2] 
Governance and community engagement were overwhelmingly discussed as improved across all areas. It was 
also noted as the most signifi cant change among LGA councilors in most REM sessions. 
3.3.1.2. Finance 
Both PS3 and other areas discussed perceived improvements in fi nancial management and accountability in 
the use of available funds, even when limited. For example, GOT systems, such as Epicor and FFARS, were 
positively perceived in both PS3 and other areas. 
FFARS simplifi es a lot because there it simplifi es to see how money utilization goes, also simply it helps to see 
what is remaining because when you send you see directly how the expenditure goes compared to your activities 
and that money is used according to the budget that you planned. For example, maybe in a budget you planned 
to buy medicines worth 3 or 4 million, so when you deduct it from your system it shows directly that this fund 
has done this work directly through this activity. So, it is simple to see the fund has done what activity. 
[KII with district medical offi cer, other]
Before we were introduced with Epicor we had diffi culties in supervising funds, we used local systems to track down 
the expenditures using guidelines. We used log books and I used to prepare Excel with my own techniques in knowing 
how the money was used. Epicor shows your sections and inform you on how to use your budget and so if there is any 
error I check with my accountants and people of TAMISEMI. But secondly, Epicor helps you limit your expenditure 
and show you your quarterly budget or yearly budget on how to use your money, how much they need to spend and 
how much money has remained quarterly. And second we used to write minutes to get a check, but with Epicor 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 67
its easier because everything has been generated, from voucher to check is being processed online. So, it has helped 
manage the money because the money has to be sent for several issues such as Basket fund, medicines and help you 
plan. So, if the money is from Global Fund it makes you aware of its usage even if it’s on the same account. 
[KII with town medical offi cer, Phase 2]
Appreciation for and usefulness of these systems was also refl ected at the national level, as exemplifi ed through 
this comment from a local government offi cer—TAMISEMI:
Th e positive change in the controls of the services I am currently experiencing is the use of electronic systems in 
revenue and expenditure issues. For example, Epicor has really proved to be much more important because they 
were just lost but now one can stay on the computer. I know how some councils have incorporated it and how 
much it has been…used. Th e control of fi nance has become bigger and the revenue collection has increased. 
[KII with local government offi cer, TAMISEMI]
Stakeholders in both PS3 and other areas described training on these systems—either by GOT or PS3. In 
program areas, PS3’s continued support was appreciated for addressing challenges as they occurred.
PS3, in my work place [for] the training of FFARS they supported us, when we face some challenges, we call them for 
an assistance…they help us on time when there is a challenge in a system. Th ey make sure the problem on [the] system 
solved on time, they will not leave as it is. [KII with district treasurer, Phase 2]
Th e collective perception from stakeholders was that fi nance systems put in place have signifi cantly contributed 
to resource savings. Th ere has been improvement in utilization of the available funds as the responsible offi cer 
can now track or follow-up where unnecessary costs occur. Th is has allowed areas to improve effi ciencies in 
fund use. For example, one KII mentioned they no longer need to travel to Dodoma to submit data, as they 
now can do so through electronic systems; this allowed more funds to be available for development activities. 
You don’t need to travel to Dodoma [for] which we used to spend a lot of money. Of course, last week they designed 
MESO version where you can use electronic fi le data where you print a book and send it to Dodoma...we spent a 
lot of billions, for example last year we spent almost fi ve billion just going to Dodoma for fi ve days but now we are 
working in our areas... [KII with town medical offi cer, Phase 2]
Additionally, as discussed in the previous section, stakeholders perceived that fi nancial systems had improved 
accountability and transparency. LGA councilors in intervention areas noted that transparency and 
accountability had improved by engaging citizens. 
Now that all citizens are involved, there are committees that ensure that the funds allocated by the government 
are well spent for the intended goals. Not all teachers are involved in committee meeting, except very few such 
as the head teacher and the fi nance teacher. Th ere is a system whereby no teacher is allowed to take cash and 
go to buy school needs at a stationery or shop. All this is done to ensure that no teacher misuses the public fund. 
[FGD with men, Phase 2]
Th rough community meetings, councilors read out income and expense reports on respective projects and 
noted defi cits. Th is allowed councilors to emphasize citizen responsibility for developmental contributions 
to increase community ownership and even bring about additional sources of revenue to mutually manage 
ongoing projects (Figure 37). 
Figure 37. Excerpt from REM map with LGA councilors, Phase 1 – Shinyanga
68 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
3.3.1.3. Human Resources
Both PS3 and other areas described HR distribution and allocation as a challenge. HR systems created to 
better address these challenges, such as WISN plus POA (or WISNPOA, colloquially), have been rolled-out 
nationally, but attitudes towards the utility of these systems to solve problems diff ered slightly between control 
and intervention areas. For example, one KI in a non-PS3 area described “going around” the system to manage 
the HR distribution. 
…the challenge happens during arranging workers according to WISNPOA results, it happens that, in one health 
center there is one staff and he or she is overloaded to the extent that he or she is no longer remembering to enter 
some information into the system. And when it comes to analyzing the needs, his or her needs is not seen because it 
seems like he saved few patients. I suggest that, the ministry has to increase the number of staff without looking at the 
WISNPOA system. [KII with DMO, other]
In intervention areas, stakeholders were more positive in their refl ection on the systems, indicating the system 
works well—even when there are challenges related to availability of enough staff ; the system makes it easy to 
identify how the distribution fares in relation to facility needs.
Th ere has been a lot of challenges in allocating the employees to facilities. I have to be honest, there have been a lot 
of challenges on that for a certain employee should go there and the other should go here. But with the use of this 
system, will be able to keenly determine where there is strongly workload or where there is a need to be allocated 
employees. Example, if a certain facility highly needs two clinical offi cers (COs) then this system will identify that 
two COs are strongly required at certain facility after we have entered the data in the system. And if there is a 
workload, for instance in the side of nurses, the system will enable to identify that certain number of nurses are 
needed at a certain facility. Th erefore, this system will highly help in allocating the human resources to facilities... 
[KII with DHRO, Phase 1]
Th is positive perception was also present at the national level, whereby system use had made recruiting, 
allocating, and distributing staff easier.
...in the past yes, but now we are doing good work with data, there is availability of right data to help decision 
making. For example, if a council is allocated ten teachers primary school, let say ten new teachers, the DO will have 
enough data now to look at where to post those teachers, because he has the number of teachers in his schools, he has 
the number of teachers who are in studies, he has the number of teachers who are on sick leaves so he will be able to 
know which school require teachers from those ten, he will be able to make a decision an better allocation...
[KII with PfM advisor – education]
While allocation and distribution of HR seemed to work well with the help of systems, fi ndings across areas, 
regardless of geographical location, showed that there was an HR availability defi cit. Th e number of workers 
was perceived to not correspond well with the capacity needed at the facilities, especially in the health sector. 
Th e system improvement made it easy to follow up on how staff were working and compensated. Systems like 
the Open Performance Review and Appraisal System (OPRAS) and LAWSON signifi cantly contributed to 
make staff at LGAs more accountable and improved effi ciencies in sector performance. 
[LAWSON] is a national system dealing with all employees’ information of which it helps us to perform diff erent 
activities like employment issues. Example, you have seen the newly 14 allocated health sector employees in our council 
who have come to report. Th erefore, their information after they have fi lled the employees’ forms are entered in the 
LAWSON system for them to approve in order to get the check number and to be determined that they are now the 
civil servants…it also enables us for promotions and preparation of budgets with this system. When we are doing 
employees’ promotions after they have fi lled their promotions letters, we enter them in LAWSON, then they are 
approved of which their salaries change. Th erefore, it is a very useful system compared to past years. Th is system helps 
us to perform our responsibilities. [KII with DHRO, Phase 1]
In addition to accountability, some stakeholders expressed that the easy monitoring and follow-up of progress 
for each individual worker has encouraged changes in the way services are provided. However, others felt the 
systems are only able to contribute so much in a context where there is a staffi ng shortage, particularly in terms 
of skilled staff . Shortages have led to other workers completing tasks outside their position or ability. Th ese 
resource constraints are refl ected in quality of services, as citizens perceive public sector services to be of low 
quality with high wait times and frequent stockouts and may choose to avoid them altogether. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 69
Largely, we are obtaining health services from the private facilities because the health services at public facilities is of 
the low quality. Sometimes, you fi nd no medicine or sometimes you are to wait for the medical staff from eight am 
until twelve noon who are at the meetings and have not yet come to attend the patients. Th erefore, we do not know 
does whether the facility management fail to arrange its schedules for things to work well that people to continue being 
served or not. So mostly many people or my community opt to obtain the health services from the private hospital 
rather than public hospitals. [FGD with women, other]
HR capacity, skills, and competencies were expressed more confi dently in intervention areas compared to those 
in control areas. Staff in these areas recognized contribution by PS3/GOT. 
…it has strengthened strongly the services delivery, because even if you look the way we welcome the newly allocated 
public servants they are motivated to perform their jobs. But again, PS3 through trainings has strengthened the way 
we arrange our receptions. Th ese trainings have enabled even the people working at receptions to work according 
to regulations; they are receiving incoming mails, outgoing mails in time to the right authorities and implementing 
them in time. When a customer comes, he/she is cared friendly and receives service he/she requires very smoothly. 
Th erefore, it brings encouragement to our citizens in services delivery. As HROs, we have many customers who are 
local citizens and our public servants. All these are our customers as an institution… [KII with DHRO, Phase 1]
Similar demonstrations in control areas were contributed by staff who moved to these areas from 
intervention districts. 
3.3.1.4. Information Systems
Positive performance of information systems was highly tied to other areas previously discussed (e.g., HR, 
fi nance, governance). For example, respondents noted that HR systems have improved staff allocation, even 
during shortages, to ensure effi ciencies. Information systems have improved fi nancial management and 
budgeting, particularly at facility levels—for example, FFARS system use has facilitated checks and balances 
within facilities.
In general, there was a positive perception that the use of systems has reduced the bulk of tasks to be 
handled manually. Where applicable, IT systems saved time and resources by reducing the use of paperwork. 
Additionally, former paper-based systems were described as burdensome while also leaving room for error and 
loss of information. Th e new digital systems have eased those feelings. 
Changes are there, I mean that the presence of the diff erent systems, we previous had “paper-based.” Th us, the 
presence of systems makes a lot of support that even if you want the information that you have already inserted, even if 
it was years back it is easy to get into the system, you search and fi nd it. Unlike if it was “paper-based” things are easy 
to disappear or lost quickly… [KII with ICT offi cer, Phase 1]
Th ese improved effi ciencies and data availability have opened space for improvements and development in the 
community and at council levels. Th ese system changes have also contributed to perceived improvements in 
data quality and program performance. 
…honestly the system has helped me and my fellow physicians to access information easily, supervise on provision of 
health services in my area, track progress each month, and also compare progress on months before. You can also check 
on facilities and their performances, how many patients they treated, why they have attended to few patients. You can 
see the number of patients in OPD and its challenges, you can notice if health centers are providing fake information. 
So, when supervising you can make follow-ups and solve few problems. Th ey even show trends of patients at certain 
health centers, why they have so many patients, or why they have few patients, how are vaccines provided, and if 
things are going smoothly because we also have targets—local targets, regional targets, and national targets—and you 
as the head, you have to supervise and make sure work performance and provision of services go along the targets and 
make sure they are archived... [KII with town medical offi cer, Phase 2]
Additionally, information systems have changed the sharing and communication atmosphere. Stakeholders 
viewed the systems with which they work as one-stop centers for information. Th ough not interoperable in 
many areas yet, the computer-stored information was considered easy to access. Digital systems allow users 
to access information from anywhere, again reducing the amount of resources spent traveling, as well as 
expanding the reach of information-sharing. 
70 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
As you know we are currently more digital that sometimes notice boards are less useful. Example, if someone is 
somewhere far say in Dar es Salaam or Arusha may not be able to access our information through notice boards. Th e 
notice boards are maybe used by the local people but for those who are far it is only the website that can help them to 
access our council’s information. [KII with DHRO, Phase 1]
Training and guidance were provided for government workers at the council level; by GOT in non-PS3 areas 
and PS3/GOT in intervention areas. Th ese trainings were noted as helpful in building initial capacity among 
those trained. Specifi cally, in intervention areas, sustained and ongoing support was also highly valued.
Th ere are improvements in work performance even to the council or district level and primary level there is 
improvement in the health centers and dispensaries. When you look at the GOT/PS3 sponsored trainings, they have 
highly capacitated our human resources to improve their job performances…generally, there has been a frequently 
big technical assistance from the GoT/PS3 expertise. Especially when there are challenges within the systems, the PS3 
have been getting together with our IT expertise to address the challenges. Also, during installation of these systems 
they have been collaborating with our expertise. [KII with DED, Phase 2]
Many respondents discussed the approach of training specifi c individuals who were expected to go back and 
train others or provide ongoing support to others. While those that were trained directly spoke positively of 
the systems, many viewed the approach as fl awed. Th ere is a suggestion that more training opportunities are 
needed in order to completely change the perception and trust of some workers.
…there is this saying that “no research, no right to speak.” For us, the system itself is a research. We use it to get report 
of revenue collected at a given time. It happens that some leaders reject the report believing that it’s incorrect. Th is is 
due to ignorance since they do not know how the system works. Th ey still rely on written reports which have many 
shortcomings. For instance, people may forget to write some data, so they tend to estimate the value. Th e system is the 
only eff ective method to be used today. Conclusively, we need more training for these kinds of people… 
[KII with ICT offi cer, Phase 2]
3.3.2. Evaluation Question 5
Th ese subsections provide the qualitative fi ndings for evaluation question 5: In PS3 focus regions, what was the 
context of any observed or perceived change?
3.3.2.1. Community Engagement and Governance
Positive perceptions and gains in community engagement and governance must be viewed within the context of 
existing and ongoing initiatives from the GOT. For example, the Opportunities and Obstacles to Development 
(O&OD) is an intensive participatory process used in both rural and urban areas as a bottom-up planning 
approach. Th is process has, for the most part, catalyzed engagement between citizens and local government. 
However, a commonly noted issue in creating development plans at the local level was the availability of 
fi nances and human resources to carry out these plans. 
Th ere is a challenge of not reaching every priority set by the council since there is lack of funds. For instance, you plan 
to build 10 schools, but you have received funds from the central government enough for three schools only. In every 
aspect there is challenge, but we have some step ahead every year. We are moving forward. [KII with DED, Phase 1]
Additionally, councilors perceived their allowances to be low in comparison to their workloads and obligations. 
3.3.2.2. Finance 
Budgetary defi cits and, consequentially, the lack of fulfi llment of planned activities were commonly reported 
challenges. At the council level, there were claims that the planned budget was not always met and sweeping 
national policies were diffi cult to carry out. Also, respondents noted there was improvements in the fi nancing 
of health services and provision of funds for free education; however, ministry funds had been delayed in 
some areas, causing burdens on councils to achieve this requirement. Th ese were seen in terms of equipment 
shortages and insuffi cient infrastructure at the service level, both for schools and health facilities.
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 71
…challenges exist in diff erent levels of our departments, for example in secondary schools we have challenges of 
infrastructures, fi nances, and lack of enough teachers. But in our department the main challenge is fi nances, 
because we have to supervise all the activities and deliver…we have poor performance, which in actual sense is poor 
utilization of resources. In our municipal, we need funds to conduct eff ective monitoring and without funds we are 
working in a tough environment… [KII with education offi cer, other]
…the other challenge is budget defi cit, knowing that our department is the mother department its activities cut across 
all other departments. Th erefore, sometimes there are activities which we are responsible to fulfi ll but we fail due to 
budget defi cit… [KII with DHRO, Phase 1]
Respondents noted that there is increasing demand from a growing population that is not adequately planned 
for in health and education funding.
…for us a challenge is that our district have expanded and the number of people is many and we are serving near 
districts around us, so the problem comes in budget. You may fi nd we have planned maybe the budget of three 
months, but later on it is not enough due to the number of people to be many that comes from within and outside our 
district, so sometimes you fi nd that medicines are not enough. So, that is a challenge we are facing… 
[KII with DMO, central]
Interviews with national-level offi cials confi rmed that budgetary constraints catering to the needs of an 
increasing population were a challenge for LGAs that can be refl ected at the service level. Stakeholders 
discussed gaps in resources due to fi nancing, such as desks and books at schools, and medicine stockouts at 
public health facilities.
Th ese are also met by structural and contextual changes in how health fi nancing at the individual-level works, 
including insurance schemes. Whereby, insurance may cover some testing and/or drugs (and therefore make it 
more available and increase access/use), but not all. 
I: You mentioned increases of tests here, which are they increased?
R1: HIV/AIDS tests is available now.
I: Which challenges that makes increases of various tests and medicine?
R1: Th e main challenges that come is when you need services by your insurance, they will give only few and important 
medicines instead of all.
I: Important medicines like what?
R1: Th ey will give you some and other you are told to go and buy the in local pharmacy. [FGD with men, Phase 1]
Additionally, fi nance challenges intersected with HR as many facilities had shortages of staff , compelling a 
nurse or doctor to act as a fi nancier or bursar/accountant.
3.3.2.3. Human Resources
Widespread staffi ng shortages prove to be the greatest challenges in terms of HR. Th is was in part due to a 
recent nationwide exercise to verify employees’ certifi cates in their places of work. Th ose who were determined 
to be unqualifi ed were removed or had to leave their positions. Th is led to HR shortages. 
We make sure we have proper employees. Th e action of verifying the employees’ certifi cates reduced some employees, 
meaning that we have now defi ciency of human resources… [KII with DHS, other]
Additionally, increasing numbers of schools and health facilities were struggling to meet HR needs with 
competent and skilled staff . A shortage of health facility staff sometimes caused delays, and compromised 
service quality. In education, while stakeholders discussed improvements due to the free education policy, they 
also noted challenges faced due to staff shortages.
Community members and key informants noted, because of the emphasis on qualifi cations and skills and a 
greater encouragement for women to seek job opportunities, that there is a perceived improvement in equity in 
employment selection. 
72 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
3.3.2.4. Information Systems
While information systems were regarded as positive, they may be constrained by contextual factors. For 
example, notable challenges in the systems were related to the poor/lack of availability of internet services and 
electricity services. For instance, while the workers were supposed to use the internet to operate the systems, 
the government had not established how these workers should access the bundle for internet provision and thus 
they reported using their own money to buy internet bundles. Additionally, while communication channels 
have improved, contextual challenges remain. 
…we have improved somehow but we still need more changes. As I said earlier, the citizens had only two options of 
getting the information. Th e fi rst one was through radio and another one was to visit our notice boards to see what 
we have pinned. Now, at least, we have this website. People can visit it to see what is going one and they can send 
us some feedbacks easily via mobile phones. We cannot move forward if we don’t solve the existing challenges that we 
have right now. Th e major challenges are communication networks and transport systems. We cannot get immediate 
responses from the citizens if we have unstable networks. And remember, these systems depend on the strong network. 
Th ere are remote areas which are not reachable, and people are required to spend a lot of money for transport. People 
may pay up to 20 thousand to arrive at our offi ce here… [KII with ICT offi cer, Phase 2]
Additionally, IT challenges intersected with HR. Many facilities had staff shortages, compelling the service 
provider to work as an IT person. Th e transfer of workers sometimes created gaps where trained personnel 
were transferred without an equal replacement. We also found that in some cases the workers trained in an 
intervention area would be transferred to a nonintervention area, thus contributing to the improvement of 
information system use in that new area.
Despite slow growth in the availability of technology, there has been greater understanding of IT services and 
its relation to other issues. For example, one KII discussed information communication technology (ICT) and 
its changing role in governance. 
I can say basing on ICT, people used to call us computer analysts and most people didn’t know what exactly we were 
doing…so we were perceived as useless people. But, due to advancement in technology we are now valuable. We now 
participate in council meetings and are a part of decision making. [KII with ICT offi cer, Phase 1]
3.3.3. Evaluation Question 6 
Th ese subsections provide the qualitative fi ndings for evaluation question 6: What challenges remain in system 
functioning and community engagement and governance and how do stakeholders and community members 
recommend these be addressed?
3.3.3.1. Community Engagement and Governance
In control areas, citizens’ understanding of their responsibilities in development and attending of important 
meetings that involve decision-making has remained limited. Expansion of LGA councilor training and 
support to engage citizens may address this challenge, as LGA councilors discussed these as key successes. 
However, some supportive communication mechanisms, such as comment boxes, are not yet fully accepted by 
communities due to the lack of trust and explanation of use. 
I see that in order to place that comment box fi rst everyone should be given education about the uses of the comment 
boxes through organizing and attending meetings in the society. We need to know the people who are in charge of the 
boxes so that we can be able to know how to post our claims and reach main offi ce. Th is will help to minimize some 
disturbances to people, for example there is a certain leader who came and people expressed all their claims to him but 
later on some people were taken under security for more clarifi cations. Th ose were the disturbances that were facing us 
so we would like the people in charge to work properly to avoid those disturbances. [FGD with men, Phase 2]
In addition to better publicity on comment box use, community participants suggested outreach via means 
like radio and television notices, community meetings, and postings in health facilities and schools in order to 
engage citizens.
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 73
Budget limitations and misaligned priorities created challenges within and among government levels. Some 
promises made by LGA leaders and politicians are not always fulfi lled, making community members’ eff orts to 
participate in governance seem futile. 
At the district level we follow good chain of command, the problem is orders from the central government, they 
fi nance things that are not in the O&OD and this is the reason why projects fail…the response to our O&OD comes 
down from central government as orders, I suggest that central government personnel should also get PS3 training. 
[REM with LGA councilors, Phase 1]
You may sometimes get unwanted help. For example, you may ask for a store construction but their priority is maybe 
in community-based organization, for instance the issue of water and desks or laboratories. [KII with DPLO, other]
Many key informants recommended that central government processes (like O&OD) be better aligned with 
local plans. 
Although many participants reported recent improvements in accountability and transparency, there were still 
many who saw challenges remaining. 
We have a problem here. I have not seen any meeting for a long time. Our member of parliament have not 
arranged for any meeting to talk to his people here in Kilosa. It is very rare to see him and even if he comes for a 
reason that brought him here, it’s not for listening for what people want to tell him or solving the problem of the 
people here. Th ey are not accountable people and not accountable for their people. [FGD with men, Phase 1]
In both control and intervention areas, councilors requested increases in allowances, which falls beyond the 
scope of PS3. 
3.3.3.2. Finance 
Limited funds to achieve activities continue to be a challenge, seen in terms of shortages of supplies, 
equipment, and the required levels of infrastructure in both schools and health facilities. Community 
members discussed the limited availability of health services and service providers, noting there is a shortage 
of medicines in public facilities. Several community members also discussed challenges with housing for 
health providers.
Sometimes your patient needs services in the middle of the night, but you might fi nd only one nurse while doctors stay 
far from the hospitals. Sometimes the nurse in charge of the shift stay far from the hospital, so she can’t operate alone 
in the hospital premises. So, the main challenge is houses for health care providers. [FGD with women, other]
Many of the suggestions provided by stakeholders and community members on this matter fall outside the 
scope of PS3, including ensuring funds are available from the central government to enact policies. However, 
issues arising around budgeting for a growing population can be addressed by trainings.
While community members discussed increased engagement in income-generating activities, there is a need 
for capacity building in numerous groups on issues related to management of funds, generation of capital and 
profi t, as well as with record keeping. Expansion and institutionalizing of the training off ered through LGA 
institutes may improve self-functioning with little supervision and changing of mindsets of citizens and staff . 
...we have a task to do in terms of changing the mentality. Because, one of the important things is most of the 
interventions that are coming through PS3 and the government are going to change the management culture and 
roles and relationships. It is not an easy thing to manage. Like now, if you are saying, okay, the LGAs were responsible 
for basket funding, now the management is straight to the facility, we need to make sure we work together with the 
government to come up with a strategy to make sure that this local government authority management, they don’t 
just move away from supporting the facilities. Th ey have to make sure, like okay, this is a positive change, we have to 
help the facilities to make sure they have good plans and budgets, they have good fi nancial management skills over 
time, and keep on mentoring, so it is something that I feel like there is still a change to make sure it is realized... 
[KII with program]
Additionally, restricted access to fi nance systems was perceived to limit their utility and create challenges. 
74 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
One of the [suggested] changes is that the system users should be allowed to access the system. I faced problems getting 
access to FFARS as a system. We had to use accountants at the health centres. Th ere was a time period when we had 
problems with respect to health centres not uploading revenue reports. I would ask them and they would tell me they 
already did it. I was compelled to ask an accountant so he could help me have access to the system using his account. 
I was supposed to have a user name and password so I could have access to the system. I also have subordinates who 
need to have accounts so they can access information. Th ere is a RBF coordinator who should have an account. I 
also have a secretary whose responsibility is to manage the councils’ revenues with regard to planning, monitoring of 
fi nancial and human resources, as well as infrastructure and things like that. It does not make sense that they are 
the coordinators, yet they do not have accounts. Th at is a challenge, they cannot know what is going on in the system. 
[KII with district medical offi cer, Phase 1]
Th ere are multiple systems, such as FFARS, PlanRep, and Epicor, that perform various functions in fi nancial 
planning and management. For example, stakeholders can budget through systems such as PlanRep, distribute 
funds through Epicor, and track expenses through FFARS. Interoperability of these systems was frequently 
discussed as a possible area for improvement.5
3.3.3.3. Human Resources
Although respondents noted improvements in general for health and education systems, staffi ng shortages and 
poor job performance concerns remained at the service delivery level. Community participants commonly 
noted concerns with health service delays, too few staff at health facilities, staff not working, and too few 
teachers with growing numbers of students.
You get to the hospital and you are told there are only three doctors, but none of them is in the offi ce. You spend a long 
time waiting for them in vain. I suggest that they work on that problem. [FGD with men, other]
Most potential solutions off ered would be outside the scope of PS3. For example, increased housing for health 
staff to ensure their availability would require fi nancial resources from the central government.
Currently, the government has made sure that people are taking the positions according to their qualifi cations. 
Stakeholders recommended that the government continue ensuring well-qualifi ed employees, but also provide 
the right benefi ts to those who remain; an example noted was appropriate promotions, considering that 
promotions were stopped during the verifi cation period.
Th e 2016/2017, 2017/2018 fi nancial years’ implementations are still silent in case of promotions, but recently the 
government has allowed recategorization for the 2015/2016 fi nancial year in the 2017/2018 fi nancial budget. 
Th erefore, the public servants who were to be recategorized in 2016/2017 and 2017/2018 fi nancial year are still 
waiting. I would highly recommend to government to speed up a process in order to allow them get recategorized. 
[KII with DHRO, Phase 1]
Some participants also noted concerns with using the WISN tool. Th ey noted that certain categories of 
personnel are not tracked and that the data being used for the analyses can be incorrect, leading to faulty 
staffi ng needs assessments.
Th e record keeping system was not good. You fi nd out that there is a lot of work that is done at the health centre, 
but only a small chunk of that work is recorded. Th at is what WISN does, analyses recorded data. You look at the 
recorded data and you may be led to believe that the respective health centre does not need more personnel. I think 
they should conduct a new survey. Currently, many centres have become accustomed to keeping records as is refl ected 
in the data. [KII with DCDO, Phase 2]
3.3.3.4. Information Systems
As mentioned, trainings were held with specifi c individuals who were expected to then train others. Some 
stakeholders complained that those trained were not always the best choices; for example, not including 
supervisors in trainings may limit the understanding of system usefulness, trust in the system, and the ability 
to perform job functions. 
5 Interoperability of some of these systems is in process but not yet full rolled-out at the time of data collection. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 75
I got some kind of training when I came here. But you [PS3] now concentrate much on training secretaries, ignoring 
other administrators. I remember that you once conducted a three-days training. DMOs were not there! How dare 
you invite and train the secretaries and recruitment offi cers on how to distribute health sector servants and not 
District Medical Offi cers who head district health departments? [KII with DMO, Phase 2]
Additionally, some stakeholders requested the trainings to be longer, more frequent, and not conducted in or 
near their workplace to avoid being drawn back to their duties during the training.
Interviews with mentors have noted that reaching out to all staff in various cadres with training has been 
a challenge in some areas (especially in low-performing LGAs) over others. While the training-of-trainers 
approach can help some, not all come with the same base-level capacity. 
...yeah, that is why I tell you, you know some are ignorant to computer, many people at the level of primary school 
especially. Head teachers who are accounting offi cers among them studied during ancient time, so computer for them 
is a taboo or problem... [KII with fi nance mentor, Phase 1]
Respondents noted that, for the information systems to properly function, there is need to improve the 
availability of internet services, especially in rural areas. It is important that this is included in the budgeting 
and that staff are not expected to use their own money to cover internet services. Th ere were reports of staff 
having to do so: 
… People are using money from their pockets to purchase the [internet] bundles. Basically, these systems require the internet 
access but there are no well-established infrastructures to support them... [KII with DPLO/intervention, Phase 1]
Other suggestions, such as improving electricity supply and reducing the cost of smart phones may be beyond 
the scope of PS3.
Most notably, many stakeholders discussed how diff erent information is located in diff erent systems. It is 
recommended that systems become integrated and/or interoperable. 
…what I would recommend is that these systems should communicate, meaning that they need to be integrated 
with each other. Example, if possible Lawson should integrate with PlanRep because we do manpower statement in 
PlanRep, it is better if possible to also be integrated in Lawson and again to be implicated in the PlanRep system; 
meaning that these systems should communicate if a person needs to check the employees’ salary statements in the 
budget, check for number of employees either through Lawson which is improved but also can get all this information 
in the PlanRep system. Th e same there should be the possibility to import this information from/or to PlanRep from/
or to Epicor for diff erent usage... [KII with DED, Phase 2]
Th rough the program documents, the evaluation learned that the process of making some of these systems 
interoperable is planned (Table 37), and in some cases underway, though very few LGAs had been reached by 
the time of this evaluation. In the specifi c case of Lawson and PlanRep, it did not appear that PS3 plans to 
integrate these, but these plans may have evolved.
Table 37. Integration between system and type of data transferred/exchanged
Interoperable systems
Direction of data 
transfer Data exchanged
PlanRep and Epicor Epicor to PlanRep Budget (revenue and expenditure)
PlanRep to Epicor Budget changes and reallocation
PlanRep to Epicor Actual expenditure for reporting execution
Epicor to PlanRep Revenue received
Epicor and FFARS FFARS to Epicor Summary of actual expenditure
Epicor and LGRCIS LGRCIS to Epicor Summary of daily actual revenue
LGRCIS to Epicor Prepayments of revenue
PlanRep and FFARS FFARS to PlanRep Summary of actual expenditure
Source: PS3 Year 3, quarter 2 report.
76 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
4. LIMITATIONS
4.1. Limitations of the Quantitative Data
4.1.1. DHIS2 Data Strengths and Limitations
Health care services in Tanzania are provided by public, private, faith-based, and nongovernmental organizations. 
Th e public sector, which includes faith-based organizations, accounts for the bulk (84%) of the service provision 
with an extensive network in both rural and urban areas. Th e point of care data collection and management 
practices vary between the public and private sectors. For the most part, public health facilities rely on paper￾based data collection systems. In the private sector, electronic data capture and management is common. Th e 
public sector is also rolling-out a new initiative to strengthen electronic data capture at the point of care through 
the Government of Tanzania Health Operations Management Information System (GOTHOMIS). 
Th e health management information system (HMIS) provides critical data on those who go to health facilities 
for a variety of services. Besides diagnosis and drug dispensing information, these systems capture essential client 
characteristics, such as sex, age, and residence, that are critical for eff ective service delivery. However, these data 
are often not fully processed, analyzed, and used. 
With specifi c reference to paper-based systems, there are concerns about the quality of data collected in terms of 
completeness, accuracy, and timeliness. Th e data quality problem is exacerbated by the data collection burden 
placed on health care workers for transcribing data, and tallying and reporting data into the HMIS. Pressed for 
time against a long patient waiting list, health care workers may be inclined to record only the most essential data 
for patient care, omitting other critical background characteristics. 
As part of its eff orts to improve the quality of data generated through routine health information systems, the 
Ministry of Health, Community Development, Gender, Elderly and Children (MOHCDGEC) adopted in 2010 
the District Health Information Software (DHIS), developed by the University of Oslo in 1994, for routine data 
management. Th e original version of the DHIS was designed as a tool for aggregating routinely collected data 
across all public health facilities in a country. Th e aggregated data would facilitate data analysis, forecasting of 
required services, and evaluating health care worker performance. In 2013, an improved version of the software, 
called DHIS2, was launched. Th e new version provides advanced features for data capture, including the 
capability of inputting data using various electronic tools, such as computers, laptops, tablets, and smartphones, 
as well as data analysis and visualization. DHIS2 is also interoperable with other data management systems, for 
example, human resource information systems and logistics management information systems, thus improving 
its capacity to serve as a one-stop health data warehouse. Tanzania is among nearly 50 low- and middle-income 
countries that have adopted DHIS2 as a national health management information system. 
Th e introduction of DHIS2 has helped to improve data management by allowing access to data at national 
and subnational levels, enabled by it being a web-based application. DHIS2 off ers the capability of creating 
customized indicators, with accompanying visualization. However, as promising as it is, DHIS2 is aff ected 
by general challenges to the routine health information system (RHIS), which can be grouped into three 
categories—technical, behavioral, and organizational. Th e common technical challenges relate to the ease 
of using data collection tools and information and communication technology. A key technical challenge in 
Tanzania is the burden of data collection at the point of care. With up to 16 registers and other reporting forms 
in use, some of which collect duplicative data, frontline health care workers face enormous data management 
challenges. When data captured into the source document is incomplete or inaccurate, subsequent use of an 
electronic data management system, such as DHIS2, does not ameliorate the quality problems. 
Th ere are other technical challenges related to denominators. DHIS2 uses the health facility as the lowest 
organizational unit for generating indicators. Denominators for calculating health facility level indicators are 
derived from census and population projections. Th e denominators may not be accurate in that we could not 
assess how well the population projected models estimate the actual size of relevant denominators, especially 
considering potential changes in population over time. Successive annual health sector performance profi le 
reports (AHSPR) have alluded to this challenge. For instance, the 2015/2016 AHSPR notes, “data quality issues 
are suspected during capturing of the primary data, and around determination of denominators from population 
projections” (AHSPR, page 39). Additionally, these denominators are diff erent from those reported in the 
Tanzania DHS, so there are likely discrepancies in the values of indicators in this report compared with those 
found in the DHS.
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 77
From a behavioral perspective, a pervasive culture of health data use is yet to be experienced in Tanzania, partly 
due to limited knowledge, poor attitudes, and lack of value for data, as well as a lack of motivation for using 
data. Health care workers at the interface of service delivery and routine health data collection often see their 
role primarily as that of reporting the data collected to higher levels, rather than using the same data to improve 
patient care or service quality. Even among those who may be motivated to use data, clear guidance on their scope 
for taking actions based on the data may be lacking. When data are not adequately used, quality issues may not 
be easily picked out. 
Organizational challenges occur at multiple levels and refl ect information culture, health system structure, 
roles and responsibilities, and resources. Th e key challenges in Tanzania include shortages of staff , in some cases 
leading to task shifting of data management responsibilities to staff with no requisite skills; stockout of standard 
registers, leading to improvisations that can compromise data quality; and lack of supportive supervision and 
feedback mechanisms.
Th e analysis presented in this report identifi es indicators that may have been aff ected by data quality problems. 
Data was often incomplete and did not span the preferred range to establish longer-term trends prior to the start 
of PS3. Although DHIS2 started in 2013, not all LGAs were covered during this year. Th erefore, most service 
utilization indicators only have complete data from 2014, which was 1.5 years prior to PS3. HR and fi nancial data 
were only available yearly, as opposed to quarterly, which was available for service utilization indicators. HR data 
was also not available for all years. All of these challenges reduced the number of time points available for analysis 
in the time series. Furthermore, there were apparent data quality issues refl ected in the values of the original data. 
Some indicators originating from DHIS2 had values well over 100 percent, even extending beyond 1,000 percent 
for some years. Problematic LGAs and quarters/years were identifi ed, but such values call into question the 
quality of entire datasets. Additionally, we could not identify data problems in values less than 100 percent or for 
indicators that were not measured in proportions, such HR and fi nancial ratios. 
Newly implemented information systems also may have aff ected measurement error that is indistinguishable 
from program eff ects on the indicators. Despite quantitative results being presented as averages, there was 
great variability in the indicator values at the LGA level. Th is variability is captured by the confi dence intervals 
presented in the graphs. However, considering the changes in information systems, there was likely measurement 
error, which could have been particularly high when these systems were newly implemented. We cannot 
determine how this measurement error evolved over time or how much measurement error is responsible for the 
variability observed in the indicators. It is impossible to parse out eff ects due to reduced measurement error over 
time from eff ects due to the other factors. 
Additionally, there were issues at the LGA level which imposed limitations on this analysis. Th e total number of 
LGAs diff ered depending on the data source and indicator type, and it also changed over time. Th e inconsistency 
in the total number of LGAs in the diff erent datasets was likely a result of the partitioning of new LGAs from 
old ones. Th e analysis used in this report aggregated data by PS3 and non-PS3 areas, so it is likely that the eff ect 
of these changes is mitigated through the use of averages. However, we cannot know exactly how the creation of 
new LGAs may have aff ected the quality of the data. We also did not have data on LGA characteristics, which 
could have allowed us to identify underlying factors associated with the observed changes over time and with the 
variation of the indicators within PS3 and other regions. 
We recommend interventions for continued strengthening of HMIS through training of existing staff , hiring 
of new staff as necessary, and improvement of data quality through regular data quality assessments. Deliberate 
eff orts are also necessary to promote the use of the data collected through regular data reviews (for example, 
monthly or quarterly), with a view to increasing service coverage and improving data quality. 
4.2. Limitations of the Qualitative Data
It is possible that participants did not feel comfortable sharing negative opinions about PS3, as the project is 
closely aligned with the government and all participants work with or for the government in some manner; they 
may have felt that there could have been retribution if they reported too negatively. Also, it was diffi cult to discern 
diff erences between the PS3 and non-PS3 areas, but this may be because many of the PS3 activities were extended 
to non-PS3 areas, rather than due to poor project performance. Lastly, as a cost-saving measure, the qualitative 
data was not collected in all PS3 areas; it could be that the areas from which we did not sample have had diff ering 
experiences with PS3 and these results may not refl ect the experiences of those clusters outside our sample.
78 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
5. DISCUSSION AND PROGRAMMATIC RECOMMENDATIONS
5.1. Summary of Results
Table 38 (identical to Table E.1 in the Executive Summary) summarizes the evaluation results, describing the 
types of trends and estimated time trend diff erences. Columns under the heading “PS3 areas’ general trend” 
depict the overall directionality of changes over time in PS3 areas. It is important to note, that, although this 
section of the table only describes PS3 area trends, general patterns between PS3 and other areas were similar, 
despite having diff erent levels for most indicators. 
79
Table 38. Summary of time trend and annualized changes in indicators
Indicators
Signi
fi cant
PS3 areas’ general trend
(based on graphs)
PS3 compared with other areas
(based on DID estimates)
Improving trend 
About the same 
or stable trend
Volatile trend
Worsening trend 
Signi
fi cantly 
better in PS3 
areas
Better, but no 
signi
fi cant 
difference
Signi
fi cantly 
protective in 
PS3 areas
Protective, but 
no signi
fi cant 
difference
Worse, but 
no signi
fi cant 
difference
Signi
fi cantly 
worse in PS3 
areas
 ANC coverage before 12 weeks gestational age xx
 Pregnant women attending ANC 4+ times xx
 Prevalence of pregnant women receiving IFA
supplementation for 90+ days x x
 Pregnant women receiving TT2+ at ANC xx
 Women tested for anemia at ANC x x x
 Women tested for syphilis at ANC x x x
 Deliveries taking place in health facilities xx
 Births assisted by skilled attendants x x
 Mothers receiving postnatal care before 7 days x x x
 Prevalence of low birth weight xx
 Measles vaccination coverage x x
 Penta3 vaccination coverage x x
 ANC partners HIV testing rate xx
 HIV-exposed infants receiving first HIV test within 2 months 
after birth x x x
 HIV-exposed infants initiated on cotrimoxazole within 2 
months after birth x x x
 Couple HIV counseling and testing at ANC x x
 Contraceptive prevalence rate x x
 Health facilities with RCH tracer drugs package x x
 PE per capita in health xx
80
Indicators
Signi
fi cant
PS3 areas’ general trend
(based on graphs)
PS3 compared with other areas
(based on DID estimates)
Improving trend 
About the same 
or stable trend
Volatile trend
Worsening trend 
Signi
fi cantly 
better in PS3 
areas
Better, but no 
signi
fi cant 
difference
Signi
fi cantly 
protective in 
PS3 areas
Protective, but 
no signi
fi cant 
difference
Worse, but 
no signi
fi cant 
difference
Signi
fi cantly 
worse in PS3 
areas
 PE per capita in education xx
 OC per capita in health xx
 OC per capita in education x xx
 Nurses per population xx
 Doctors per population xx
 AMOs per population x x x
 Health care workers that are female xx
 Primary school student-teacher ratio xx
 Secondary school student-teacher ratio x x
 Primary school students that are female xx
 Secondary school students that are female xx
 Primary school teachers that are female x x X^
 Secondary school teachers that are female x x
Total 8 17 7 5 3 3 7 4 6 11 1
Total signifi cant and favorable 7
Table 38. Summary of time trend and annualized changes in indicators (continued)
Note: None of the indicators were improving in PS3 areas only or worsening in PS3 areas only. 
^: Results for this indicator should be interpreted with caution, see text.
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 81
Trends were categorized as improving, stable, volatile, and worsening, based on the direction of the lines 
from the fi rst time point for which data was available to the last. For example, some indicators, such as 
ANC attendance, were categorized as improving because levels increased over time, while others, like the 
prevalence of low birth-weight, were considered to be improving due to decreasing levels over time. In fact, we 
observed improving trends in both areas for 17 of the 32 indicators, most of which pertained to health service 
utilization. For 12 indicators, trends did not generally improve or worsen. We observed stable trends for seven 
of these indicators, defi ned as having overlapping confi dence intervals for all adjacent time points, as well as 
the fi rst and last time points. Th e other fi ve indicators were categorized as having volatile trends, meaning that 
we observed drastic variation in the levels of indicators at diff erent time points. Volatile trends were defi ned 
as temporary directional changes occurring for more than one time point, that were so drastic there was no 
overlap with at least one confi dence interval at an adjacent time point.6
 Most of the indicators with volatile 
trends were supply-side sensitive, and thus likely refl ect changes in supply conditions.
Th ree of the indicators worsened over the observed time period. Two of them were the OC per capita indicators, 
which showed declining trends, but they stabilized at the end of the observation period. For primary student￾teacher ratios, worsening meant increasing levels, as an improvement would actually be a reduction in the 
absolute student-teacher ratio value—as long as enrollment has not decreased. For worsening and improving 
indicators, there is no overlap in the confi dence intervals at the fi rst and last observed time points. 
Also, in Table 38, columns under the heading “PS3 compared with other areas,” qualitatively refl ect the 
diff erences in time trends between PS3 and other areas, comparing the diff erences before PS3 and PS3 active 
time periods between the two areas. Estimates were signifi cant for eight indicators. Th e estimated values 
for the changes in the rates for all indicators can be found in Table 39. For three of these indicators, PS3 
areas improved signifi cantly more than other areas between both periods, all of which pertained to maternal 
health service utilization: the percentages of pregnant women tested for anemia and syphilis at ANC and the 
percentage of women attending PNC within a week of delivering.
However, not all of the signifi cant eff ects were absolute improvements. For another four indicators, we 
observed signifi cant eff ects that were protective. “Protective eff ects” were indicators that were worsening in 
both areas, but PS3 regions worsened at slower rates than other areas. Two of these indicators with signifi cant 
protective eff ects pertained to the prevention of HIV transmission from mother to child: the percentages of 
HIV-exposed infants tested for HIV and initiated on cotrimoxazole within the fi rst two months.
Finally, there was one indicator, the percentage of primary teachers that were female, that was classifi ed as 
signifi cantly worse in PS3 areas, but this requires caution in its interpretation. Th e annualized change decreased 
in PS3 regions and increased in other areas between the two periods. However, this result should be taken 
cautiously as the levels of this indicator have been stable, at just below 50 percent in PS3 regions in the PS3 
active period and just above 50 percent in other regions, which would be considered the level of gender parity. 
6
 For some indicators, such variation across time is seen in the fi rst time points of available data. These indicators were not 
considered to be volatile, as the changes might refl ect new data collection systems, and trends remain steady thereafter. 
Additionally, volatile trends were, overall, nonworsening. If confi dence intervals for the fi rst and last time points did not 
overlap and were generally worsening, relevant indicators were categorized as worsening.
Table 39. Summary of annualized changes in indicators and differences between PS3 regions and other areas 
Indicator
PS3 regions Other areas
DID
estimate
[6-3]
DID 
P-value
(1)
Before 
PS3
(2)
PS3 
active
(3)
Difference 
[2-1]
(4) 
Before 
PS3
(5)
 PS3 
active
(6)
Difference
 [5-4]
 ANC coverage before 12 weeks gestational age -1.99** 6.72** 8.71** -2.41* 4.12** 6.53** 2.18 0.183
 Pregnant women attending ANC 4+ times 2.40* 9.07** 6.67** 3.44** 9.31** 5.87** 0.79 0.754
 Prevalence of pregnant women receiving IFA
supplementation for 90+ days 8.41** 2.27** -6.14** 6.44** 1.70* -4.75** 1.39 0.463
 Pregnant women receiving TT2+ at ANC 3.44** 0.72 -2.72** 7.28** 1.87** -5.41** 2.69 0.059
 Women tested for anemia at ANC -1.05** 9.08** 10.13** 4.14* 4.68** 0.54 11.99** 0.001
 Women tested for syphilis at ANC 10.32** 15.44** 5.12* 17.53** 7.42** -10.12** 15.24** 0.000
 Deliveries taking place in health facilities 4.43** 2.47** -1.96** 3.95** 1.94** -2.01** 0.05 0.954
 Births assisted by skilled attendants 5.79** 2.29** -3.50** 3.94** 1.58** -2.35 -1.14 0.525
 Mothers receiving postnatal care before 7 days -1.39 5.22** 6.61** 3.46** 3.19** -0.27 6.88** 0.006
 Prevalence of low birth weight -1.89** -0.12 1.76** -2.11** 0.18 2.30** -0.53 0.413
 Measles vaccination coverage 7.57** -1.07** -8.64** 9.98** 2.56* -7.41* -1.22 0.752
 Penta3 vaccination coverage 5.43** 0.19 -5.23* 6.15* 3.28 -2.86 -2.36 .0588
 ANC partners HIV testing rate 3.56 7.34** 3.77 7.81** 6.75** -1.06 4.83 0.070
 HIV-exposed infants receiving 
fi rst HIV test within 2
months after birth 20.08** -2.70** -22.78** 38.19* -2.64* -40.83 18.06** 0.000
 HIV-exposed infants initiated on cotrimoxazole
within 2 months after birth 22.40** -3.62** -26.02** 39.88** -2.68* -42.56** 16.54** 0.000
 Couple HIV counseling and testing at ANC 10.71** 7.28** -3.42** 11.38** 8.58** -2.80** -0.62 0.772
 Contraceptive prevalence rate 0.36 -0.69 -1.05 0.14 -0.80 -0.94 -0.11 0.955
 Health facilities with RCH tracer drugs package 11.00** 4.36** -6.63** 9.82** 4.03** -5.79** -0.84 0.645
 PE per capita in health -316.20 332.65 648.85* 783.97 523.61 -260.36 909.21 0.147
 PE per capita in education -704.69 4,481.45** 5,186.14* 3,256.14* 3,043.04* -213.11 5,399.24 0.054
 OC per capita in health -244.52** 9.94 254.46** -300.82** 96.73 397.54** -143.08 0.269
 OC per capita in education -1,575.64** -221.66 1,353.98** -841.01** -103.80 737.21** 616.76* 0.043
82
83
Indicator
PS3 regions Other areas
DID
estimate
[6-3]
DID 
P-value
(1)
Before 
PS3
(2)
PS3 
active
(3)
Difference 
[2-1]
(4) 
Before 
PS3
(5)
 PS3 
active
(6)
Difference
 [5-4]
 Nurses per population 0.46** 0.035 -0.43** 0.45** -0.025 -0.48** 0.05 0.480
 Doctors per population 0.028** 0.028** 0.00 0.041** 0.019 -0.022 0.022 0.147
 AMOs per population 0.008** -0.006** 0.002 -0.006 -0.010** -0.005** 0.007* 0.016
 Health care workers that are female -0.58** -0.67** 0.09 -0.46** -0.47** -0.01 -0.08 0.678
 Primary school student-teacher ratio -2.01** 5.30** 7.31** -1.54** 4.92** 6.46** 0.85 0.322
 Secondary school student-teacher ratio -3.88** 3.03** 6.91** -2.77** 2.04** 4.81** 2.10 0.055
 Primary school students that are female 0.12** -0.26** -0.37** 0.12** -0.28** -0.40** 0.03 0.677
 Secondary school students that are female 1.16** 0.71** -0.44* 1.35** 0.76** -0.59** 0.15 0.612
 Primary school teachers that are female 7.12* -0.38** -7.50* 11.56* -0.52 11.03* -18.54** 0.003
 Secondary school teachers that are female 1.68** -0.87** -2.56** 1.00* -0.62** -1.61** -0.94 0.072
** Significant at < 1%; * signifi cant at <5%
Table 39. Summary of annualized changes in indicators and differences between PS3 regions and other areas (continued) 
84 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
5.1.1. Many Observed and Perceived Changes Occurred Nationally
Generally, trends and perceptions show gradual improvements nationally, with very few diff erences between 
PS3 and other regions, or between PS3 phases. Th is could be a result of various factors. First, many of the 
PS3 systems were adopted by GOT and rolled-out nationally. Th is points to a gap that was fi lled early on, 
however does make assessing the eff ects of PS3 diffi cult. 
In both PS3 and other areas, we found improvements in health outcomes, particularly among health center 
access and attendance-based services, such as ANC, 4+ANC, and facility deliveries. Qualitative results suggest 
these improvements may have been related to better understanding the importance of visiting health centers, 
particularly among pregnant women, as well as improvements in supportive infrastructure such as roads and 
the building of health centers. 
As such, it is important to note that PS3 is one in a constellation of programs contributing to public sector 
development. Th is includes, as noted by qualitative participants, other USAID-funded programs such as 
Boresha Afya, which supports integrated service delivery at the health facility and community levels across the 
country. World Vision and Amref were discussed as improving awareness and education on health services, 
which draw patients to health centers. Th ese programs work in both PS3 and/or other areas on outcomes 
similar to those targeted by PS3 and may particularly contribute to the improvements of health center access 
and attendance-based services. 
Health outcomes that are more commodity-based, such as vaccinations, iron and folic acid supplementation, 
and RCH tracer drugs, saw greater variability. Th is was seen quantitatively by steep temporal dips in outcomes 
and refl ected qualitatively in discussions of drug stockouts, insurance limitations, and avoidance of services 
all together when patients expect issues around commodity availability. Additionally, societal context may 
infl uence uptake of services. Just as improvements in facility-based deliveries required education and shifts 
from traditional norms, contraceptive uptake may require shifts around gender norms and decision making. 
Broader policies may have heavily infl uenced observed and perceived changes. In 2015, the Tanzanian 
government issued Circular 5 which implements the Education and Training Policy 2014 and directed public 
bodies to ensure that education is free for all children. Th is was also recently met with another policy to ensure 
staff are properly qualifi ed and certifi ed for their positions. Th ose who were not deemed to be qualifi ed were 
removed from their posts. So even as more students were attending school, fewer teachers were qualifi ed and 
employed in public schools. As such, we found increases in student-teacher ratios after 2015 in both primary 
and secondary education. Similarly, in terms of health, stakeholders commonly discussed a shortage of health 
care workers. Th ese issues infl uenced community members’ perceptions of public sector performance. 
Despite these restrictions, stakeholders perceived systems like those put in place by PS3 and GOT as valuable 
in creating effi ciencies and improving public sector system performance. However, these systems are relatively 
new, and this perception may be compromised by long-term stagnation or issues around commodities. As such, 
the program may require more time to realize changes in outcomes beyond those seen in this midline 
evaluation. As this is a systems intervention that addresses many underlying facets of the public sector, it is 
anticipated that observed changes may still yet be seen. 
5.1.2. Differences in Programmatic Interventions Seen Primarily at Lowest Levels
While we observed similar time trends for many outcomes at the aggregate level, where PS3 and other areas 
diff ered were often at the lowest levels of intervention. For example, PS3 stakeholders, particularly LGA 
councilors, identifi ed local community engagement and good governance as signifi cant changes coming 
out of PS3 programs. LGA councilors discussed having the knowledge and skills to attend to community 
members’ needs and complaints. Th ey also discussed improvements in transparency and accountability at the 
local level whereby citizens know and understand how money is allocated and expended for public services 
in their communities. In areas where PS3 did not provide trainings, councilors reported feeling less sure and 
identifi ed supportive systems in local governance and transparency as a need.
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 85
However, these local gains may fi nd challenges as they interact with regional and central priorities. For 
example, a commonly noted issue in creating development plans at the local level was the availability of 
fi nances and human resources to carry out these plans. So, LGAs developed plans and submitted them but 
often learned that central government had diff erent funding priorities. Th is disconnect between local and 
central authorities was frustrating for LGA staff and councilors and reportedly negatively infl uenced the 
community’s trust.
PS3 stakeholders also discussed trainings on information systems as important in highlighting the value 
of these systems and encouraging their use. Th ere was positive perception of these systems overall, even 
in their nascency, however similar gaps between levels were observed, whereby supervisors of those trained 
may not have understood the value of the system or the data gathered from it. Just as gaps between levels of 
government may undermine development, gaps between levels of supervision as they relate to training may 
undermine system uptake and use. Also, staff turnover and system updates require that there are mechanisms 
for refresher trainings as well as ongoing training for new staff .
5.1.3. Gender Parity in Secondary School Teacher Employment Declining
Qualitative respondents noted a general perception that women were more encouraged to seek employment 
and that gender equity in government had been improving. And while we found gender parity or near gender 
parity for health care workers and primary school teachers, there were clear gender parity gaps for secondary 
school teachers across areas. A low proportion of secondary school teachers were female and there was a 
decreasing trend in this proportion. Th e reasons for this are unclear but these results suggest gender disparities 
in the formation and certifi cation of secondary school teachers across regions. It is also possible that women 
were disproportionately aff ected by the eff orts in recent years to reevaluate qualifi cations for civil servants. Th is 
warrants further investigation in the end line data collection and analysis.
5.1.4. Programmatic Recommendations
We provide the following recommendations for PS3:
Information systems
 Support internet service nationally or work with central/local government to plan for this in budgets
and ensure staff are not paying for internet service out of personal funds.
 Investigate concerns with WISN staffi ng categories and accuracy of input data.
 Assess whether additional categories of staff may benefi t from access to FFARS.
 Continue to work towards interoperability of information systems.
 Institute a mechanism to regularly communicate system updates/new features to users.
Training
 Carry out needs assessments at facilities prior to information system install, ensuring staff have capacity 
to use and maintain information systems, then train as needed.
 Ensure supervisors are invited or educated on why their subordinates are invited to trainings; since 
it may not be feasible or necessary to train them to the same depth as their subordinates; PS3 might 
consider developing a shorter, awareness-raising session aimed at this audience so that they at least feel 
engaged and understand the value of the systems.
 Consider establishing refresher trainings.
 Maintain and expand in-service support.
 Integrate how to plan for growing populations into LGA councilor budgets and development training.
 Continue good governance work but increase transparency (e.g., with suggestion boxes).
86 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Citizen engagement and governance
 Address the disconnect between central and local government in planning and budgeting.
o In trainings, set expectations on what to expect from the central government with regard to 
budgeting and planning (i.e., they may not receive all requested funds), and share messages to 
communicate with citizens so as to similarly set their expectations.
o Increase understanding at the LGA level of how central government is making funding 
allocation decisions.
o Increased work with central government to encourage better communication and collaboration 
with LGAs; for example, on funding priorities and timelines. 
 Increase publicity of new accountability mechanisms and information sources like the new websites, 
health clinic comment boxes, etc.; for example, participants suggested making TV and radio 
announcement and posting notices at health centers and schools.
Gender
 Address secondary school teacher gender parity issues in HR eff orts; for example, investigate potential 
barriers to recruiting and maintaining secondary school teachers that are female, and assess whether 
gender sensitivity training for teachers, students, and administrators may be needed to create 
environments where women feel respected and safe.
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 87
REFERENCES
Economic and Social Research Foundation, United Nations Development Programme, Government of the United 
Republic of Tanzania. (2015). Tanzania Human Development Report 2014: Economic Transformation for Human 
Development. Dar es Salaam, Tanzania: Government of the United Republic of Tanzania and United Nations 
Development Programme.
Masson, L. & Norman, A.S. (2009). Decentralisation by devolution in Tanzania: Refl ections on community 
involvement in the planning process in Kizota Ward in Dodoma. Journal of Public Administration and Policy 
Research; 1(7):133–140.
Ministry of Health, Community Development, Gender, Elderly and Children. (2018). Annual Health Sector 
Performance Profi le Reports (for 2013 up to 2018). Dar es Salaam, Tanzania: Ministry of Health, Community 
Development, Gender, Elderly and Children.
Ministry of Health, Community Development, Gender, Elderly and Children. (2018). Annual Health Statistical 
Abstract (Annual Health Statistical Tables and Figures) (for 2013 up to 2018). Dar es Salaam, Tanzania: Ministry 
of Health, Community Development, Gender, Elderly and Children.
Ministry of Health, Community Development, Gender, Elderly and Children. (2015). Health Sector Strategic Plan, 
2015–2020 (HSSP IV). Dar es Salaam, Tanzania: Ministry of Health, Community Development, Gender, Elderly 
and Children.
Ministry of Health, Community Development, Gender, Elderly and Children. (2015). Result Based Financing 
Operational Manual. Dar es Salaam, Tanzania: Ministry of Health, Community Development, Gender, Elderly 
and Children.
Ministry of Health, Community Development, Gender, Elderly and Children (Tanzania Mainland), Ministry of 
Health (Zanzibar), National Bureau of Statistics, Offi ce of the Chief Government Statistician, and ICF. (2016). 
Tanzania Demographic and Health Survey and Malaria Indicator Survey (TDHS-MIS) 2015–16. Dar es Salaam, 
Tanzania and Rockville, Maryland, USA: Ministry of Health, Community Development, Gender, Elderly and 
Children, Ministry of Health, National Bureau of Statistics, Offi ce of the Chief Government Statistician, and ICF. 
National Bureau of Statistics (Tanzania) and ICF Macro. (2011). Tanzania Demographic and Health Survey 2010. 
Dar es Salaam, Tanzania: National Bureau of Statistics and ICF Macro.
UNESCO Institute for Statistics. (2019). United Republic of Tanzania. Retrieved from http://uis.unesco.org/en/
country/tz?theme=education-and-literacy#slideoutmenu. Date extracted: June 14, 2019.
USAID/Tanzania Public Sector Systems Strengthening Activity. (2018). Public Sector Systems Strengthening (PS3) in 
Tanzania, Annual Report: October 1, 2017–September 30, 2018. Dar es Salaam, Tanzania: USAID/Tanzania. 
88 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
APPENDIX A. PS3 IMPLEMENTATION REGIONS AND LGAS
Regions LGAs/Districts
Dodoma Bahi DC
Dodoma Chamwino DC
Dodoma Dodoma MC
Dodoma Kondoa DC
Dodoma Kongwa DC
Dodoma Chemba DC
Dodoma Mpwapwa DC
Dodoma Kondoa TC
Iringa Iringa MC
Iringa Iringa DC
Iringa Mufi ndi DC
Iringa Mafi nga TC
Iringa Kilolo TC
Kagera Bukoba MC
Kagera Bukoba DC
Kagera Ngara DC
Kagera Biharamlo DC
Kagera Misenyi DC
Kagera Muleba DC
Kagera Karagwe DC
Kagera Kyerwa DC
Kigoma Buhigwe DC
Kigoma Uvinza DC
Kigoma Kigoma/Ujiji MC
Kigoma Kigoma DC
Kigoma Kibondo DC
Kigoma Kasulu DC
Kigoma Kakonko DC
Kigoma Kasulu TC
Lindi Lindi MC
Lindi Lindi DC
Lindi Kilwa DC
Lindi Nachingwea DC
Lindi Liwale DC
Lindi Ruangwa DC
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 89
Regions LGAs/Districts
Mara Musoma DC
Mara Musoma MC
Mara Tarime DC
Mara Bunda DC
Mara Rorya DC
Mara Butiama DC
Mara Serengeti DC
Mbeya Mbeya CC
Mbeya Mbeya DC
Mbeya Mbarali DC
Mbeya Rungwe DC
Mbeya Busokelo DC
Mbeya Kyela DC
Mbeya Chunya DC
Mbeya Mbozi DC
Mbeya Momba DC
Mbeya Tunduma TC
Mbeya Ileje DC
Mbeya Tunduma TC
Morogoro Morogoro MC
Morogoro Morogoro DC
Morogoro Mvomero DC
Morogoro Kilombero DC
Morogoro Ulanga DC
Morogoro Kilosa DC
Morogoro Gairo DC
Morogoro Malinyi DC
Morogoro Ifakaara RC
Mtwara Mtwara/Mkindani DC
Mtwara Mtwara DC
Mtwara Tandahimba DC
Mtwara Newala DC
Mtwara Masasi DC
Mtwara Nanyumbu DC
Mtwara Masasi TC
Mtwara Nanyamba TC
90 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Regions LGAs/Districts
Mtwara Mtwara DC
Mtwara Masai DC
Mwanza Mwanza CC
Mwanza Ilemela MC
Mwanza Sengerema DC
Mwanza Ukerewe DC
Mwanza Misungwi DC
Mwanza Magu DC
Mwanza Kwimba DC
Mwanza Buchosa DC
Njombe Njombe TC
Njombe Njombe DC
Njombe Makete DC
Njombe Ludewa DC
Njombe Makambako TC
Njombe Wanging’ombe
Rukwa Sumbawanga MC
Rukwa Sumbawanga DC
Rukwa Nkasi
Rukwa Kalambo DC
Shinyanga Sinyanga MC
Shinyanga Shinyanga DC
Shinyanga Kishapu DC
Shinyanga Kahama TC
Shinyanga Msalala DC
Shinyanga Ushetu DC
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 91
APPENDIX B. INDICATOR TIME TRENDS STRATIFIED BY PS3 
IMPLEMENTATION PHASE
Figure B.1. ANC started before 12 weeks of gestation
Figure B.2. ANC, at least 4 antenatal care visits
92 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure B.3. Iron and folic acid supplementation in pregnancy
Figure B.4. Pregnant women receiving TT2+
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 93
Figure B.5. Proportion of pregnant women tested for anemia
Figure B.6. Pregnant women tested for syphilis
94 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure B.7. Birth deliveries in health facilities
Figure B.8. Births delivered by skilled attendants
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 95
Figure B.9. Postnatal care within 7 days of delivery, for mothers
Figure B.10. Prevalence of low birth weight
96 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure B.11. Measles vaccination, inside service area
Figure B.12. Penta3 vaccination
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 97
Figure B.13. ANC partners’ HIV testing rate
Figure B.14. Percentage of HIV-exposed infants receiving fi rst HIV test within 2 months after birth
98 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure B.15. Percentage of HIV-exposed infants initiated on cotrimoxazole within 2 months after birth
Figure B.16. Percentage of couples HIV counseling and testing at ANC
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 99
Figure B.17. Contraceptive prevalence rate
Figure B.18. Health facilities with RCH tracer drugs package
100 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure B.19. PE per capita in health
Figure B.20. PE per capita in education
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 101
Figure B.21. OC per capita in health
Figure B.22. OC per capita in education
102 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure B.23. Nurses per population, per 10,000 people
Figure B.24. Doctors per population, per 10,000 people
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 103
Figure B.25. Assistant medical offi cers per 10,000 people
Figure B.26. Proportion of health care workers that are female
104 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure B.27. Primary school student-teacher ratio
Figure B.28. Secondary school student-teacher ratio
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 105
Figure B.29. Percentage of primary school students that are female
Figure B.30. Percentage of secondary school students that are female
106 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Figure B.31. Percentage of primary school teachers that are female
Figure B.32. Percentage of secondary school teachers that are female
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 107
APPENDIX C. INDICATOR DEFINITIO NS
Table C.1. Selected health service utilization indicators for routinely collected data – PS3 evaluation
Indicator 
title
Indicator 
defi nition
Indicator 
calculation
Reporting
frequency
Report data 
source
Date 
range 
available
Antenatal care 
(ANC) coverage 
before 12 weeks 
gestational age 
Percentage 
of pregnant 
women who 
start ANC before 
12 weeks of 
gestational age
N: Number of pregnant 
women who start ANC 
before 12 weeks of 
gestation age*100
D: Estimated number 
of pregnant women
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Pregnant women 
attending ANC 
4+ times
Percentage of 
pregnant women 
who received 
antenatal care 
four or more 
times in a given 
time period.
N: Number of pregnant 
women who received 
antenatal care four or 
more times*100
D: Estimated number 
of pregnant women.
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Prevalence 
of pregnant 
women receiving 
iron and folic 
acid (IFA) 
supplementation 
for 90+ days
Percentage 
of pregnant 
women who 
received IFA for 
90+ days
N: Number of pregnant 
women who received 
IFA supplementation 
for 90+ days for a 
certain period of time
D: Total number of 
pregnant women 
during the same period
Quarterly DHIS2 Q3 2013 – 
Q3 2018
Pregnant women 
receiving TT2+ at 
ANC
The percentage 
of women at the 
fi rst ANC visit who 
received the TT2+ 
vaccine 
N: Total number of 
ANC tetanus two 2+ 
doses*100
D: Total number of 
ANC fi rst visit
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Women tested 
for anemia at 
ANC
The percentage 
of women at the 
fi rst ANC visit who 
were tested for 
anemia 
N: Total number of 
ANC hemoglobin tests 
done*100
D: Total number of 
ANC fi rst visit
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Women tested 
for syphilis at 
ANC
The percentage 
of women at the 
fi rst ANC visit who 
were tested for 
syphilis
N: Total number of 
ANC syphilis tests 
done*100
D: Total number of 
ANC fi rst visit
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Deliveries taking 
place in health 
facilities
Percentage of 
deliveries taking 
place in health 
facilities
N: Number of deliveries 
taking place in health 
facilities during a given 
period
D: Total number of live 
births /deliveries during 
a given period
Quarterly DHIS2 Q3 2013 – 
Q3 2018
108 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Indicator 
title
Indicator 
defi nition
Indicator 
calculation
Reporting
frequency
Report data 
source
Date 
range 
available
Births assisted 
by skilled 
attendants**
Percentage of 
births assisted by 
skilled attendants
N: Number of births 
attended by skilled 
health personnel 
during a specifi ed 
period
D: Total number of 
live births during the 
specifi ed period
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Mothers 
receiving 
postnatal care 
before 7 days
Percentages 
of mothers 
who received 
postnatal care 
within seven 
days of childbirth 
(regardless of 
place of delivery)
N: Number of mothers 
who received 
postnatal care 
within seven days of 
childbirth
D: Total number of 
live births during the 
specifi ed period
Quarterly DHIS2 Q3 2013 – 
Q3 2018
Prevalence of 
low birth weight
Percentage 
of newborn 
registering less 
than 2.5 kg 
weight
N: Number of 
newborns registering 
less than 2.5 kg weight
D: Number of all live 
births
Quarterly DHIS2 Q3 2013 – 
Q4 2018
Measles 
vaccination 
coverage
Percentage 
of children 
under one year 
vaccinated 
against measles
N: Total number of 
children under one 
year vaccinated 
against measles*100
D: Total number of 
children under one 
year targeted in the 
period
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Penta3 
vaccination 
coverage
Percentage of 
children under 
one received 
Penta3 vaccine 
in a given year or 
other period
N: Total number of 
children under one 
year vaccinated 3 
times against DPT￾Hb*100
D: Total number of 
children under one 
year targeted in the 
period
Quarterly DHIS2 Q1 2014 – 
Q4 2018
ANC partners HIV 
testing rate
Percentage of 
partners of ANC 
clients tested for 
HIV during the 
fi rst ANC visit 
N: Number of ANC 
client partners tested 
for HIV during the fi rst 
ANC visit
D: Total number of fi rst 
ANC visits 
Quarterly DHIS2 Q1 2014 – 
Q4 2018
HIV-exposed 
infants receiving 
fi rst HIV test within 
2 months 
after birth
Percentage of 
infants who were 
exposed to HIV 
and were tested 
within 2 months
N: Total number of 
HIV-exposed infants 
receiving fi rst HIV test 
within 2 months after 
birth*100
D: Total number of 
women tested HIV 
positive
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 109
Indicator 
title
Indicator 
defi nition
Indicator 
calculation
Reporting
frequency
Report data 
source
Date 
range 
available
HIV-exposed 
infants initiated 
on cotrimoxazole 
within 2 months 
after birth
Percentage 
of infants who 
were exposed to 
HIV and started 
cotrimoxazole 
within 2 months 
N: Total number 
of HIV-exposed 
infants initiated on 
cotrimoxazole within 2 
months after birth*100
D: Total number of 
women tested HIV 
positive
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Couple HIV 
counseling and 
testing at ANC
Percentage 
of Couple HIV 
counseling and 
testing at ANC
N: Total number of 
ANC clients who 
received HIV testing 
and counselling as a 
couple during the fi rst 
ANC visit
D: Total number of fi rst 
ANC visits
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Contraceptive 
prevalence rate**
The percentage 
of women aged 
15-49 years, 
regardless of 
marital status, 
who have 
received at least 
one modern 
method of 
contraception 
from a health 
facility during the 
year, regardless 
of the method 
used
N: Number of women 
15-49 years of age who 
are currently using a 
modern contraceptive 
method*100
D: Number of women 
between 15 and 49 
years of age in the 
catchment area
Quarterly DHIS2 Q1 2014 – 
Q4 2018
Health facilities 
with RCH tracer 
drugs package
Percentage of 
health facilities 
with RCHs tracer 
drugs package
N: Total number of 
health facilities with 
RCHs tracer drugs 
package
D: Total number of 
health facilities 
Quarterly DHIS2 Q1 2014 – 
Q3 2018
Personal 
Emoluments (PE) 
per capita in 
health
The total amount 
spent on health 
personnel salaries 
per person in the 
population
N: Gross salary in TZ 
Shillings received by 
all health sector civil 
servants in the LGA
D: The number of 
people living in the 
LGAs
Yearly CFR 2012–2017
PE per capita in 
education
The total 
amount spent 
on education 
personnel salaries 
per person in the 
population
N: Gross salary in TZ 
Shillings received by all 
education sector civil 
servants in the LGA
D: The number of 
people living in the 
LGA
Yearly CFR 2012–2017
110 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Indicator 
title
Indicator 
defi nition
Indicator 
calculation
Reporting
frequency
Report data 
source
Date 
range 
available
Other charges 
(OC) per capita 
in health
The total amount 
of other health 
sector-related 
expenditures 
per person in the 
population
N: Total funding in 
TZ Shillings for other 
charges fl owing 
through LGA bank 
accounts for health
D: The number of 
people living in the 
LGA
Yearly CFR 2012–2017
OC per capita in 
education
The total 
amount of 
other education 
sector-related 
expenditures 
per person in the 
population
N: Total funding in 
TZ Shillings for other 
charges fl owing 
through LGA 
bank accounts for 
education
D: The number of 
people living in the 
LGA
Yearly CFR 2012–2017
Nurses per 10,000 
people
The number of 
nurses per 10,000 
people in the 
population
N: The number 
of nurses in the 
LGA*10,000
D: The number of 
people living in the 
LGA
Yearly HRHIS 2012–2018
Doctors per 
10,000 people
The number 
of doctors per 
10,000 people in 
the population
N: The number 
of doctors in the 
LGA*10,000
D: The number of 
people living in the 
LGA
Yearly HRHIS 2012–2018
Assistant medical 
offi cers per 
10,000 people
The number of 
AMOs per 10,000 
people in the 
population
N: The number 
of AMOs in the 
LGA*10,000
D: The number of 
people living in the 
LGA
Yearly HRHIS 2012–2018
Health care 
workers that are 
female
The percentage 
of health care 
workers that are 
female
N: number of health 
care workers that are 
female in the LGA*100
D: Total number of 
health care workers in 
the LGA
Yearly HRHIS 2012–2018
Primary school 
student-teacher 
ratio
The ratio of 
the number of 
primary school 
students to 
the number of 
primary teachers 
N: Number of students 
enrolled in primary 
school
D: The number of 
primary school 
teachers
Yearly MOE BEST 2012–2013
NBS 
OPENDATA
2014–2015
PO-RALG 
BEST
2016–2018
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 111
Indicator 
title
Indicator 
defi nition
Indicator 
calculation
Reporting
frequency
Report data 
source
Date 
range 
available
Secondary 
school student￾teacher ratio
The ratio of 
the number 
of secondary 
school students 
enrolled to 
the number 
of secondary 
teachers 
N: Number of students 
enrolled in secondary 
school
D: The number of 
secondary school 
teachers
Yearly MOE BEST 2012–2013
NBS 
OPENDATA
2014–2015
PO-RALG 
BEST
2016–2018
Primary school 
students that 
are female
The percentage 
of students 
enrolled in 
primary school 
that are female
N: The number of 
female, primary school 
students*100
D: The total number 
of students enrolled in 
primary school
Yearly MOE BEST 2012–2013
NBS 
OPENDATA
2014–2015
PO-RALG 
BEST
2016–2018
Secondary 
school students 
that are female
The percentage 
of students 
enrolled in 
secondary 
school that are 
female
N: The number of 
female, secondary 
school students*100
D: The total number 
of students enrolled in 
secondary school
Yearly MOE BEST 2012–2013
NBS 
OPENDATA
2014–2015
PO-RALG 
BEST
2016–2018
Primary school 
teachers that 
are female
The percentage 
of primary school 
teachers that 
are female
N: The number of 
female primary school 
teachers*100
D: The total number 
of primary school 
teachers
Yearly MOE BEST 2012–2013
NBS 
OPENDATA
2014–2015
PO-RALG 
BEST
2016–2018
Secondary 
school teachers 
that are female
The percentage 
of secondary 
school teachers 
that are female
N: The number of 
female, secondary 
school teachers*100
D: The total number 
of secondary school 
teachers
Yearly MOE BEST 2012–2013
NBS 
OPENDATA
2014–2015
PO-RALG 
BEST
2016–2018
112 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
APPENDIX D. DATA SOURCES
Service Utilization Data Sources
DHIS2/HMIS
DHIS2 is a health data management tool. Data entry is done for a list of data elements or in customized, user￾defi ned forms that are developed to mimic paper-based data capture forms. Th e major data source for DHIS2 
is the routine aggregate data that health facilities collect using HMIS paper-based forms, which are then 
entered into the system at the district level and used to generate routine reports and indicators at the district, 
regional, and national levels. Th e data entry module is where data are manually registered in the DHIS2 
database. Data are registered for a single organizational unit (a health facility), period, and set of data elements 
(dataset) at a time. A dataset often corresponds to a paper-based HMIS reporting form.
Strengths
DHIS2 can also be used to increase or enhance data quality at multiple points in time:
1. At the point of data entry: Checks can be made to ensure that data fall within acceptable minimum 
and maximum values for any given data element. Such checks help identify typing errors at the time of 
data entry.
2. After data entry: Users can defi ne validation rules, which can be compared against the data that have 
been entered to identify any validation rule violations. If there are any violations, staff responsible for 
entering data can determine if any actions need to be taken. Th ese types of checks help to ensure that 
data entered in the system are of good quality.
Limitations
Th ere are concerns about the quality of data collected in terms of completeness, accuracy, and timeliness. Th e 
data quality problem is exacerbated by the data collection burden placed on health care workers for transcribing 
data, as well as tallying and reporting data into the HMIS. With up to 16 registers and other reporting forms 
in use, some of which collect duplicative data, frontline health care workers face enormous data management 
challenges. Th ere are also challenges with denominators for calculating various indicators. DHIS2 uses the health 
facility as the lowest organizational unit for generating indicators. Denominators for calculating health facility￾level indicators are derived from census and population projections. Th e denominators may not be accurate.
Indicators
 ANC coverage before 12 weeks gestational age 
 Pregnant women attending ANC 4+ times 
 Prevalence of pregnant women receiving iron and folic acid (IFA) supplementation for 90+ days 
 Pregnant women receiving TT2+ at ANC 
 Women tested for anemia at ANC 
 Women tested for syphilis at ANC 
 Deliveries taking place in health facilities 
 Births assisted by skilled attendants 
 Mothers receiving postnatal care before 7 days 
 Prevalence of low birth weight
 Measles vaccination coverage
 Penta3 vaccination coverage 
 ANC partners’ HIV testing rate
 HIV-exposed infants receiving fi rst HIV test within 2 months after birth 
 HIV-exposed infants initiated on cotrimoxazole within 2 months after birth 
 Couples HIV counseling and testing at ANC 
 Contraceptive prevalence rate
 Health facilities with RCH tracer drugs package
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 113
Financial Data Sources
Council Financial Reports (CFRs)
CFRs provide and report LGAs’ fi nance information relating to own source revenue (OSR), cashbook balance, 
transfers and loans/borrowing and expenditures, both in summary and details, in a specially designed Excel 
template with each type of information completed on its sheet. Th e process of preparing CFRs by LGAs 
started in 2008/2009. For this particular evaluation, CFR data for all LGAs for six ears from 2012/2013 to 
2017/2018 are involved. 
An Excel template is completed by the LGA and then submitted to the Regional Secretariat (RS) on a 
quarterly basis (within 15 days after the respective quarter’s end). At the RS level, individual LGA CFRs within 
the region are compiled and consolidated to produce a regionally consolidated LGA CFR. After completion 
of the compilation and consolidation by the RS, normally within two weeks, the regionally consolidated 
LGA CFRs are submitted to PO-RALG at the Directorate of Local Government, Local Government Finance 
Section for compilation and consolidation at the national level, and hence nationally consolidated LGA CFRs. 
Access to nationally consolidated LGA CFRs was granted by the Permanent Secretary—PO-RALG—and 
the evaluation team representative worked with three government staff members to clean, extract, and compile 
data from CFRs. Th e staff were appointed by the Assistant Director – Local Government Finance; i.e., one 
fi nance management offi cer, responsible for CFR at PO-RALG HQ and two fi nance management offi cers 
from Mtwara and Coast Regional Secretariats. 
CFRs are the main source of LGA information relating to fi nance, which is used for various purposes, inter 
alia, monitoring, control, and evaluation, and providing information on own-source revenue collection, 
transfers, and expenditures. Th e main users of CFRs are LGAs, RSs, PO-RALG, the Ministry of Finance 
and Planning (MOFP), and other sectoral ministries. Other users include development partners (DPs) who 
collaborate with the government on various matters relating to LGAs as well as particular interests on local 
government fi nance issues.
Limitations
 Several errors and mistakes in some data entered in the “CFR Excel template,” especially at the early 
years (3–4 years), caused by manual system for preparation of CFRs as well as high advanced Excel 
template, thus requiring training to key users. However, the situation improved continuously from 
year to year.
Th is limitation has, however, been solved through PS3 support and beginning 2018/2019 the CFR preparation 
process became automated and CFRs are now produced automatically through the PlanRep System.
Indicators
 Personal emoluments (PE) per capita in health
 PE per capita in education
 Other charges (OC) per capita in health
 OC per capita in education
Human Resource Data Sources
Human Resource for Health Information System (HRHIS) 
Managed by MOHCDGEC
A software for collection, collation, and storage of HR for health information. In 2009, the MOH consulted 
with the University of Dar es Salaam to develop the software which tracks human resources for health 
information in the health sector. Th e system is being used by health secretaries at the district level for the day 
to day tasks of recording and reporting HR in the health sector. HRHIS has currently been rolled-out across 
Tanzania—it is fully operational and used in all district hospitals, regional hospitals, and referral hospitals in 
114 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Tanzania, as well as health training institutions and ministry departments. Data from lower facilities is entered 
at the district level in the case of dispensaries and health centers. 
Records Management
Th e system uses a records management module to control the collection of HR information such as personnel 
particulars, in-service trainings, and employees’ histories. In the data management module several forms are 
available for data entry depending on the type of employee. Th ese include: 
1. Training institution employee form
2. Public employee form
3. Private employee form
4. Referral hospital employee form
Limitations
 Delays in entering new employee records
 Delays in updating the history of employees
 Exit date (day) of employee not recorded
Indicators
 Nurses per 10,000 people
 Doctors per 10,000 people
 Assistant medical offi cers per 10,000 people
 Health care workers that are female
Education Data Sources
Basic Education Statistics in Tanzania (BEST)
Managed by MOEST; data for 2012–2013
Between 2012 and 2013 education statistics were commissioned to diff erent consultants in various parts of 
the country who had the responsibility of gathering education information and submitting the same to the 
ministry of education for publishing. Data were collected, in paper form, from primary and secondary schools, 
and other education institutions. 
Limitations
No raw data was found at the MOE, only limited secondary data that has been published was available. 
Due to diff erent consultants, the quality of the data depended on the vigor exhibited by the consultant and 
therefore the data lacked uniformity in quality. Th e majority of data were at the regional level. 
Indicators
 Primary school student-teacher ratio
 Secondary school student-teacher ratio
 Primary school students that are female
 Secondary school students that are female
 Primary school teachers that are female
 Secondary school teachers that are female
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 115
OPENDATA
Managed by the National Bureau of Statistics; data for 2014–2015
Th is is an online basic statistics portal which contains data for diff erent sectors, including education. For the 
case of HR for education, OPENDATA gathers datasets from diff erent institutions under the education sector. 
Th e Basic Statistics Portal is managed by the Basic Statistics working group, comprising the National Bureau 
of Statistics.
Limitations
Th ere is no specifi c period for updating the portal, and the data have signifi cant gaps.
Indicators
 Primary school student-teacher ratio
 Secondary school student-teacher ratio
 Primary school students that are female
 Secondary school students that are female
 Primary school teachers that are female
 Secondary school teachers that are female
Basic Education Statistics in Tanzania (BEST)
Managed by PO-RALG; data for 2016, 2017, and 2018
Data was retrieved from institutions under the jurisdiction of MOEST and PO-RALG. Th ese include: 
pre-primary, primary, secondary, teacher education, adult and non-formal education, as well as technical, 
vocational, and higher education institutions. Data is collected through annual census of these institutions 
using questionnaires. Filled-in questionnaires/forms are returned to the councils for data entry and processing. 
Data from technical, vocational, and higher institutions is collected directly by the Ministry. Population data is 
obtained from NBS, while examination data is obtained from the National Examination Council of Tanzania 
(NECTA).
Frequency of Data Collection
Data from higher education, technical, and vocational training institutions must be submitted by 15 February 
each year.
Indicators
 Primary school student-teacher ratio
 Secondary school student-teacher ratio
 Primary school students that are female
 Secondary school students that are female
 Primary school teachers that are female
 Secondary school teachers that are female
116 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
APPENDIX E. SCOPE OF WORK
MEASURE Evaluation is a MEASURE project funded by the U.S. Agency for International Development (USAID)
under the terms of Leader with Associates Cooperative Agreement AIDOAA-L-14-00004. Views expressed in this 
report do not necessarily reflect the views of USAID or the U.S. government.
MEASURE Evaluation Phase IV
Updated Scope of Work and Country Work Plan Narrative: 
September 2017 – April 2019
Evaluation of the Tanzania Public Sector System Strengthening (PS3) Program
Revised and Approved January 2018
MEASURE Evaluation
University of North Carolina at Chapel Hill
400 Meadowmont Village Circle, 3rd Floor
Chapel Hill, NC 27517 USA
TEL: 919-445-9350 FAX: 919-445-9353
http://www.cpc.unc.edu/measure
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 117
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 2 
Contents 
Introduction .......................................................................................................................................................... 3
Objective ............................................................................................................................................................... 4
Background: MEASURE Evaluation Phase IV ........................................................................................................ 5
4TZ-002: Evaluation of the Public Sector System Strengthening (PS3) Program in Tanzania .............................. 5
Year 2 Work Plan ............................................................................................................................................. 10
Proposed Timelines and Results ......................................................................................................................... 10
Budget Summary ................................................................................................................................................ 12
Staffing Plan and Management Plan ................................................................................................................... 13
Annex 1: Concept note/evaluation design ......................................................................................................... 16
Annex 2: Cancelled Deliverables and Benchmarks ............................................................................................. 25
 
118 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 3 
Introduction
This SOW is an update of a previously accepted SOW for evaluating the USAID funded Public Sector System 
Strengthening Project (PS3) in Tanzania. Originally designed and planned as an impact evaluation, we 
learned in mid-June 2017 that the Government of Tanzania had decided to expand PS3 interventions to reach 
national coverage. The expansion was expected to occur rapidly, and it was to cover the planned comparison 
areas for the impact evaluation, which rendered the original impact evaluation design invalid. During July￾September 2017, different evaluation design options were discussed with USAID/Tanzania, but most designs 
were infeasible given the new program conditions and availability of data. In September 2017, MEASURE 
Evaluation received new revised evaluation questions from USAID/Tanzania. These questions are answerable 
through an outcome evaluation that includes quantitative and qualitative components. This SOW has been 
fully revised -objectives, proposed methods, deliverables and benchmarks, as well as an updated concept 
note that respond to the new evaluation questions. 
Of the funds received $933,544, $206,877 has been expended. 
Background 
Despite rapid economic growth (7% per annum) and progress for some key health indicators, Tanzania has 
fallen seven positions in the UNDP’s 2014 Human Development Index over the last decade (1). Tanzania has 
one of the most rapidly growing populations globally at 2.7% per year, with the urban population growing 
approximately 5% per year (1). Tanzania also has poor living standards, particularly in rural areas, with 
national poverty and extreme poverty levels at 28.2% and 9.7% (1). The Government of Tanzania has set an 
ambitious target to reduce poverty significantly and become a middle-income country by 2025. However, to 
achieve this Tanzania will have to tackle numerous pressing health issues in its growing population. 
Despite progress, such as reductions in infant mortality rates, Tanzania still faces many health and social 
challenges. HIV/AIDS has had a serious social and economic impact on the country since the epidemic began 
over 30 years ago. This impact cuts across all health sectors and has resulted in the diversion of resources 
from other areas to HIV prevention, care and treatment (1). In 2011/2012, mainland Tanzania had an adult 
HIV prevalence of 5.3%, reflecting a significant decline in prevalence from 7% in 2003/2004 (2). In 2014, only 
640,000 people received HIV/AIDS treatment (3) while approximately 1.5 million people were living with HIV 
(3). In addition to HIV/AIDS, Tanzania faces a significant burden from infectious diseases, such as malaria, 
which contributes to Tanzania’s infant mortality rate of 45 deaths per 1,000 live births (4). Though the 
national malaria prevalence by microscopy for children under five years of age was only 4.1% in 2011/2012, 
there was considerable variation by region, with prevalence in children under five in coastal and lakeside 
areas as high as 20.7% (2). Family planning is another health area that needs improvement to achieve 
Tanzania's 2025 goals. On average, only 32% of married women used a modern family planning method 
during the 2015 Tanzania Demographic Health Survey (TDHS). Total Fertility Rate (TFR) also remains high at 
5.2, with approximately one-quarter of women wanting no more children (5). Further, malnutrition remains 
a significant problem in Tanzania. The 2015 TDHS shows that 34% of children under five experienced stunted 
growth (5). There is an urgent need to understand and overcome these health challenges in order to achieve 
broader national health goals. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 119
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 4 
The “Decentralization by Devolution” policy (D-by-D) was passed under Tanzania’s Local Government Reform 
Act in 1996, initiating the decentralization of public sector planning and service delivery from the central 
government to the local governance system (6). The aim of the “D-by-D” policy was to bring decision making 
for resource allocation and services to the local level so that local needs were better addressed, and local 
people could actively participate in the decision-making process (6). However, the “D-by-D” policy has 
resulted in many challenges at the local level, stemming from inefficiencies in service delivery related to 
inadequate financing and financial management skills, poor coordination, communication, unclear scopes of 
work (SOW) for district stakeholders, and failure to completely decentralize all core functions to the Local 
Government Authorities (LGAs). These inefficiencies extend across all aspects of local governance, such as 
health service delivery, agriculture, infrastructure, and education. Therefore, they must be addressed if 
Tanzania is to meet its developmental goals. 
In response to these systems-level gaps resulting from decentralization, USAID funded the Public Sector 
System Strengthening (PS3) project in Tanzania. The overall objective of the PS3 project is to support the 
Government of Tanzania (GOT) in strengthening the public-sector system to promote the delivery, quality 
and use of services, particularly for underserved populations. There is an emphasis on system strengthening 
and improving quality of services for priority health areas (e.g. HIV/AIDS, maternal and child health, TB, 
malaria) as well as improving key multi-sectoral components at the local government level related to 
agriculture, infrastructure and education. Specifically, the PS3 project aims to improve public services and 
outcomes by strengthening national and district capacity in the following areas: governance and citizen 
engagement, human resources, financial management and information systems. 
There is a paucity of information, both nationally and globally, regarding the efficacy of interventions to 
address system-level gaps and capacity shortcomings. In 2012, an article by Adam and colleagues reviewed 
recent evaluations of health systems strengthening programs, but found that few evaluations were 
comprehensive across multiple health system building blocks and few included evaluation designs that 
considered the complex nature of the programs (7). 
This evaluation will fill present knowledge gaps by supplying critical information about the impact of systems 
strengthening interventions such as Public Sector System Strengthening (PS3). By contributing to the 
country’s evidence base of successful systems strengthening activities, the proposed evaluation will help to 
ensure that high quality public services are available to and meeting the needs of Tanzanians. It will also 
ensure that strong public systems are in place to enable Tanzania to meet its economic and development 
goals. 
Objective
Based on discussions with the USAID/Tanzania and USAID/Washington teams, this scope of work 
proposes an outcome evaluation of the PS3 program with the goal of examining changes in the uptake 
of health services as well as in human resources and finance systems operational indicators in PS3 
regions. 
PS3 is a system-level intervention that seeks to improve the public system's operational capacity with 
the ultimate objective of improving service delivery of public services and population health. This 
evaluation will use data from routine information systems to conduct an analysis of pre-and-post￾program changes over time and trends of finance, human resources and service delivery outcomes in 
120 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 5 
PS3 areas. Quantitative evaluation findings will be contextualized using qualitative methods. Qualitative 
methods will also be used to assess stakeholders’ perceptions of the performance of PS3.
Background: MEASURE Evaluation Phase IV
The primary objective of MEASURE Evaluation is to enable countries to strengthen their systems to 
generate high quality health information that is used for decision making at local, national and global 
levels. MEASURE Evaluation applies a systems approach to achieve this objective in a sustainable way. 
One application of this approach is to increase capacity for rigorous evaluation. MEASURE Evaluation’s 
results framework reflects the overarching implementation strategy whereby the project works through 
distinct activities to achieve results. Achievements in the four result areas shown below contribute to 
the overall project objective. 
Result 1: Strengthened collection, analysis and use of routine health data;
Result 2: Improved country-level capacity to manage health information systems, resources and 
staff;
Result 3: Methods, tools and approaches improved and applied to address health information 
challenges and gaps;
Result 4: Increased capacity for rigorous evaluation.
MEASURE Evaluation’s work under this SOW will address Results 3 and 4. The work described in this 
activity will enhance the capacity of local and regional organizations by providing opportunities for formal 
training and experiential learning. First, we will strive to work with in-country partners to help gather 
systems-level information for the quantitative evaluation and micro-level information for the quantitative 
analyses. For the former, we will work with in-country subcontractor to develop their skills in collecting, 
maintaining and preparing systems-level data and qualitative data for outcome evaluation. Second, to the 
extent possible, we will involve those counterparts in the process of data analysis, so that they can learn the 
analysis methods and become familiar with the tools (e.g. computer programs). Next, we will use the study 
design, data collection, and analysis phases as opportunities for experiential learning for our local data 
collection partners. The experiential learning processes in both cases will be augmented with workshops on 
the methodologies used in the quantitative and qualitative analyses. The work under this SOW will also 
address Result Area 3. 
4TZ-002: Evaluation of the Public Sector System Strengthening (PS3) 
Program in Tanzania
Activity Leader: Gustavo Angeles
Other Staff: Lisa Calhoun, Jessica Fehringer, and Peter Lance, TBD
Objectives:
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 121
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 6 
1. Conduct an outcome evaluation to examine to what extent the uptake of health services (and 
related indicators of service utilization, coverage of health services, quality of services and access) 
changed over time in PS3 regions and to examine to what extent LGAs human resources and 
financial systems strengthened in the same PS3 regions; 
2. Determine whether there are differences in the changes over time observed in certain groups of 
LGAs versus others (e.g., those receiving PS3 versus those in non-PS3 regions but receiving similar 
interventions from the government or from other development agencies; those PS3 LGAs receiving 
the “robust” PS3 package of services versus the “standard” PS3 package; or those with different 
timing of implementation); 
3. Use qualitative approaches to contextualize changes observed in different groups of LGAs and 
assess stakeholders’ perceptions of the performance and impact of PS3 on different aspects of 
service provision, governance and citizen engagement; 
4. Inform policy formation and subsequent systems-level program designs beyond the PS3 program. 
Background: PS3 is a system-level intervention, which began in mid-2015 and is aimed at strengthening 
the devolved (decentralized) public system to provide quality services. The PS3 project works with LGAs 
to strengthen their systems, particularly their governance, financial, information and human resources 
systems. PS3 will operate in 93 LGAs, which fall within 13 targeted regions in mainland Tanzania. The 
PS3 program is strongly aligned with the objectives of both USAID/Tanzania and the Government of 
Tanzania, and, therefore, the success of these objectives and national health policy goals rests in large 
part with the success of efforts under the PS3 program to improve system capacity to deliver health and 
other multi-sectoral services.
Systems-level interventions are increasingly important in global health thinking and strategy 
development. Decentralization of services and systems is also increasingly important. It is a challenge to 
evaluate the impact of systems-level interventions in part because the systemic nature of the 
intervention makes it difficult to find a control group against which to assess impact. MEASURE 
Evaluation hypothesizes that: first, if PS3 provides technical assistance to better manage financial 
resources, then the LGA-level financial system will be strengthened and will have better performance; 
second, if the human resources systems are strengthened, then there will be improved availability of 
human resources in the targeted LGAs; and third, if public systems are strengthened, then the LGAs’ 
delivery of health services will be improved as demonstrated by changes in specific HSS outcomes . 
This activity as originally conceived has been changed at the request of the USAID/Tanzania, given 
changes in the implementation in the PS3 program. Implementation of PS3 program components has 
been rapidly scaled up at the national level meaning that there is no longer a suitable comparison group 
and thus invalidating the evaluation as previously designed. In response to these programmatic 
changes, MEASURE Evaluation proposes to undertake an outcome evaluation of the PS3 program, as 
detailed in this scope of work. For a list of cancelled deliverables and benchmarks, please see Annex 2. 
122 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 7 
Methods
Quantitative
MEASURE Evaluation will extract data from routine information systems and conduct an analysis to 
examine how key service delivery and operational outcomes have changed over time. We will assess 
data from the time frame ranging from 2013 to 2018. This window includes a PS3 pre-implementation 
phase (2013 – July 2015)1
 and a PS3 implementation phase (July 2015 – 2018)2
 which could potentially 
be divided into early- and late-phase implementation categories. Data will be collected from all LGAs of 
the country. 
National, LGA-level data will allow us, first, to identify the LGAs in the PS3 regions and to examine how 
they, as a group, have changed over time. We can then examine if the observed changes in the PS3 
regions vary relative to changes in LGAs where the government or other donors are implementing 
similar HSS interventions3
. We will also examine variations in changes over time for subsets of LGAs, for 
instance, we will compare those receiving the “robust” PS3 package of support versus those receiving 
the “standard” package. 
To conduct the analysis, we will use a multi-year multi-group difference-in-differences (DID) modeling 
approach, which provides great flexibility to examine how trends over time differ for different groups of 
LGAs. The DID approach would also allow to examine changes in the trend before and after the program was 
implemented. 
Routine information systems data will be compiled retrospectively and prospectively. Several key data 
sources will be utilized, including databases with indicators related to human resources (HCMIS/Lawson, 
PlanRep, BEMIS), finance (EPICOR, PlanRep), health (DHIS2/HMIS, PlanRep), and information systems (all 
systems). For data systems in which direct access will not be granted, such as EPICOR and 
HCMIS/Lawson, reports with specific indicators at the LGA level will be requested. Once the complete 
time series has been constructed, prospective data will be collected as it becomes available. Gaps and 
inconsistencies in data will be assessed in-country, and, where possible, efforts will be made to correct 
any issues encountered. MEASURE Evaluation will submit a final report summarizing the findings in April 
2019.
 
1 We will use 2013 as the starting point because that is the year when DHIS2 transitioned completely to 
an electronic system which facilitates accessing and processing the data. It also provides about 1.5 to 2.5 
years of data prior to the PS3 project inception. 
2
 We use 2018 as the upper year for the analysis only because MEASURE Evaluation ends in mid-2019 
and can only collect data through the end of 2018. However, given that PS3 ends in 2019, we strongly 
recommend the Mission continue collecting data through the end of 2019 and to conduct the analysis 
right after PS3 conclusion through a different contractual mechanism. That will allow more time for the 
PS3 interventions to generate changes in relevant outcomes. 
3 We recommend comparing PS3 regions to other 2 or 3 areas, as to make it substantive, simple and 
feasible to implement and to interpret in the empirical quantitative model. The definition of the groups 
to include in the analysis will be discussed with USAID. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 123
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 8 
It will be necessary to engage in-country collaborators to collect, compile and review all data from 
routine information systems. A request for proposals (RFP) was issued and an in-country partner, 
Health and Developmental International Consultants (HDIC), was selected from the pool of applicants. A 
key activity for HDIC will be to apply for and obtain ethical review approval for the study as well as 
obtain approval letters from the Government of Tanzania (GOT) to access all relevant information 
systems. Once all approval letters are obtained and permissions are in place, information systems data 
will be compiled, reviewed and analyzed. 
In addition, we will compile an indicator guide for the key set of indicators for this evaluation. The 
indicator guide will include the definition of each indicator, the source(s) of information, how the 
indicator is calculated, frequency of reporting, and any specific procedures that the information systems 
use to compile and aggregate data to calculate the key indicators. We will also undertake a search and 
review of any data quality assessments (DQA) for the key indicators of interest. This will require us to 
examine DQAs that exist for each of the information systems of interest. We will summarize the 
findings of any DQAs that have been undertaken as they pertain to the indicators of interest. 
There are several key and potentially challenging steps in carrying out this evaluation. First, obtaining 
approvals to retrieve data from the information systems may be challenging. Approval letters from each 
individual GOT office are required for each information system. This process will likely take some time, 
networking and commitment. Thus, support from USAID may be requested to facilitate meetings with 
key figures in the GOT to obtain necessary approvals to access the data. Second, even after receiving 
approval letters, obtaining access to data may be challenging. The evaluation design explicitly requires 
aggregate, LGA-level data, meaning that there is no need to obtain any sensitive individual-level data. 
Some of the databases which we require information from, such as HCMIS, contain sensitive information 
at a more disaggregated level. Our data requirements will be made clear to all GOT offices. 
Qualitative
The qualitative component will provide evidence for the evaluation as well as implementation 
monitoring and program development purposes. We plan (pending protocol re-development) for 
qualitative data collection to include key informant interviews and focus group discussions4. 
Key informant interviews and group discussions will be held with stakeholders, focusing on LGA-level 
government, but also including program, regional administration and national government staff, as well 
as non-state actors (such as the private sector and civil society organizations) that require access to and 
services from the public sector. Interviews and discussions will seek to understand the challenges and 
motivations involved in public systems management, potential solutions and perceptions of whether 
PS3 has addressed these challenges; gather views on activity implementation and components that 
worked well versus those that did not work well and understand the reasons for each; and assess 
 
4 We can only conduct qualitative data collection and analysis in 2018 because MEASURE Evaluation 
ends in mid-2019. However, given that PS3 ends in 2019, we strongly recommend the Mission conducts 
another round of qualitative data collection and analysis after the program ends through a different 
contractual mechanism. That will allow more time for the PS3 interventions to generate changes in 
relevant outcomes.
124 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 9 
perspectives on public systems strengthening and health service outcomes. We will also conduct focus 
group discussions to explore the community’s experience with and attitudes towards accountability of 
and citizen engagement in health planning and monitoring and perceptions of health service quality and 
responsiveness and any recent changes in quality. 
In addition, MEASURE Evaluation will review program monitoring data, which will be critical to 
understanding whether the interventions were implemented as planned, as changes in implementation 
could affect the causal chain leading to the outcomes. Program monitoring data review will also aid in 
understanding challenges and successes of the activities as they evolved over time. We will contact the 
program staff regularly to collect program monitoring data and to ask about any changes in 
implementation plans and reasons behind changes. 
For qualitative data collection, the evaluation will work with the same local collaborator as contracted 
for the quantitative data collection. Similar to the quantitative component, letters of support or 
approval from individual agencies will be required in addition to the basic ethical approval. These letters 
will facilitate the participation of stakeholders in key informant interviews. After letters and ethical 
approval are secured, data collection will take place, followed by analysis and reporting. 
In addition, we will track implementation of the PS3 interventions in the project LGAs as well as 
implementation of PS3-like programs in the rest of the country. In the rapid expansion of the PS3 
program, several other organizations, including the Government of Tanzania, have taken over 
implementation of PS3 interventions in specific geographic areas. We will keep track of which group or 
organization is implementing the program, where they are implementing it, what specific components 
of the program they are implementing and the time frame of implementation. This will also be 
undertaken for other PS3-like programs, which are programs that are implementing similar public-sector 
systems strengthening activities. 
Gender Integration 
USAID and PEPFAR require gender integration throughout the program cycle. Under this evaluation, we 
will ensure that gender is integrated into data collection, analysis, and reporting, as applicable. For 
example, under the quantitative component, when data is available disaggregated by sex, we will 
investigate potential differences in outputs/outcomes by sex. Similarly, under the qualitative 
component, qualitative tools and data analysis will explore potential differences in program 
implementation and outputs/outcomes related to gender. Also, qualitative data collection will explore 
with stakeholders their perceptions of the gender integration work carried out by PS3. 
This document covers the revised SOW and workplan for activities in year two, including: completion of 
the year 2 work plan, revision of the overall evaluation design, the collection of qualitative and 
quantitative data in 2018; quantitative analysis; report development, including qualitative analysis and 
quantitative; and a dissemination meeting. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 125
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 10 
Year 2 Work Plan 
Summary: The proposed workplan describes the benchmarks necessary to assess progress in conducting 
the evaluation, workplan and deliverables, including the collection of qualitative data, compilation of 
quantitative data, data analysis, and completion of the report.
Benchmark 1.1: Workplan for Year 2 completed
Benchmark 1.2: Evaluation design finalized
Benchmark 1.3: Qualitative data collection completed
Benchmark 1.4: Debrief on qualitative data collection carried out
Benchmark 1.5: RHIS and supplementary quantitative data collection completed
Benchmark 1.6: Report completed
Benchmark 1.7: In-country presentation of quantitative and qualitative findings 
Benchmark 1.8: Transition and closeout. 
About Benchmark 1.8 (Transition and closeout): MEASURE Evaluation will coordinate with the 
organization identified by the Mission the transfer of data and associated documentation (including 
qualitative tools) for the continuation of the evaluation through the end of the PS3 project. It is 
recommended that coordination start at least 6 months in advance of MEASURE Evaluation completion 
of activities. The transition process will include orienting the identified partner to the evaluation 
protocol, methodologies, and study tools as well as providing an opportunity for the incoming team to 
build partnerships with the organizations involved in this project and evaluation. In addition, the 
identified organization will be present for the in-country presentation of qualitative and quantitative 
findings. 
126 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 11 
Proposed Timelines and Results 
Timeline5
: 
Since this activity is funded under MEASURE Evaluation Phase IV, all fieldwork must end by April 2019. 
(MEASURE Evaluation ends June 2019). Therefore, the workplan and timeline covers activities through 
April 2019. Due to the changes in the evaluation design, the list of benchmarks and deliverables has 
been updated below. For a list of cancelled deliverables and benchmarks, please see Annex 2. 
Benchmarks Responsible 
Person(s)/Enti
ty
Q5
Oct
-
Dec 
17
Q6
Jan￾Ma
r 18
Q7 
Apr
-Jun
18
Q8
Jul 
– 
Sep 
18
Q9 
Oct￾Dec 
18 
Q10 
Jan￾Mar 
19 
Q11 
Apr 
19
Deliverables
1.1 Workplan for 
Year 2 completed 
X Year 2 workplan approved 
1.2 Evaluation design 
finalized
MEASURE 
Evaluation X Evaluation protocol
1.3 Qualitative data 
collection completed
MEASURE 
Evaluation 
In-country 
subcontractor 
 X 
1.4 Debrief 6
 on 
qualitative data 
collection carried out 
MEASURE 
Evaluation 
In-country 
subcontractor 
 X Presentation 
1.5 RHIS and 
supplementary 
quantitative data 
collection completed
MEASURE 
Evaluation 
In-country 
subcontractor 
 X 
1.6 Report 
completed 
MEASURE 
Evaluation X Final report
1.7 Presentation of 
findings
MEASURE 
Evaluation X Presentation 
1.8 Transition and 
close out 
MEASURE 
Evaluation X X X Data transfer completed 
Qualitative tool transfer 
completed 
 
5 The timeline is subject to discussion and approval of this SOW by the Mission. 6
 This responds to the Mission’s need for rough early findings so as to inform design of the follow-on to PS3 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 127
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 12 
Budget Summary 
This budget covers the 19-month period from October 2017-April 2019. Based on recent discussions 
with USAID/Tanzania and the PS3 program, adjustments to the timeline and budget have been made to 
accommodate personnel time, subcontracting of in-country consultants to support the quantitative 
work and travel to Tanzania. 7
Items Assumptions Amount $USD
Personnel 
38.3 mos.8
. LOE for technical support 
 (Senior Health Economist (3.2), Health Economist (.4) 
Quantitative Component Manager (2.4) Qualitative component 
manager (3.2), two research managers/analysis (4.6), one 
graduate assistant (6); HQ support (.5) Local Research 
coordinator: (15); Quantitative Analyst (3). 
418,705
Travel
Eight trips total (1-3 people for 10 days each trip) from Chapel 
Hill to Dar es Salaam, with internal travel budgeted as well. -----
-Three trips for the evaluation design meeting and training for 
qualitative and quantitative data collection; 
-One trip for qualitative debrief meeting; and 
-Three trips for data analysis and dissemination in Tanzania 
-plus, local travel in Tanzania.
169,514
Subcontracting* Local research contract for data collection, cleaning and initial 
analysis. 
51,7099
ODC Communication costs, publication costs, meeting costs 86,735
Total $726,662
 
7
 MEASURE Evaluation has received $933,544 in funds for this activity. $206,877 has been expended, including 
advance payment to the subcontractor. That contract and budget will be revised once this workplan is approved. 8
 Please note this level of effort is an estimate, pending final budgets, an estimated 18 months of local MEASURE 
Evaluation staff (Research Coordinator and Quantitative Analyst). 9 There is already a subcontract in place, though the initial subcontract will require modification. 
` 
128 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 13 
Staffing Plan and Management Plan 
Name Role/Title Technical/Managerial Focus
Country based staff
 TBD Research Coordinator Coordinate the indicator selection, 
data collection, permits, 
coordination of data analysis w UNC 
 TBD Quantitative Research 
Analyst
Work with the local contractor and 
Mission in data quantitative data 
analysis.
US-based Advisors 
Gustavo Angeles Senior Health Economist 
and Evaluation Point-of￾Contact (POC)
Principal Investigator and Activity 
Lead
Lisa Calhoun Quantitative Component 
Manager
Management of quantitative data 
collection and analysis
Jessica Fehringer Qualitative Component 
Manager
Co-Principal Investigator, qualitative 
lead 
Peter Lance Health Economist Advisor for evaluation design and 
data analysis 
Ann Marie Fitzgerald Country Portfolio Manager Project management
Data manager or analyst, TBD Data manager/analyst Data management and analysis
Qualitative research analyst, 
TBD
Qualitative Research 
Analyst
Qualitative data management and 
data analysis 
Gustavo Angeles, PhD is the Activity Lead and Senior Evaluation Advisor for the PS3 evaluation. His 
responsibilities for this work include development of the overall evaluation design, with a particular 
focus on design of a rigorous quantitative component. He will oversee information collection for the 
quantitative component and carry out data analysis. He will produce a report describing evaluation 
findings (and aim for publication). He is a health economist and faculty member of UNC’s Gillings School 
of Public Health, and a Fellow at the Carolina Population Center. He has over 20 years of experience 
evaluating health and social development programs in Latin America, Africa and Asia and is currently 
Senior Evaluation Advisor of MEASURE Evaluation, where he leads the impact evaluation of the 
USAID/Guatemala Western Highland Integrated Program, and co-leads impact evaluations of two health 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 129
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 14 
programs in Bangladesh. He also participates as an instructor on a number of training and capacity 
building activities on impact evaluation conducted in Mexico, South Africa, India and Nepal. With 
UNICEF and 3ie funding, he is currently co-investigator for the impact evaluations of Malawi’s Social 
Cash Transfer Program and Zimbabwe’s Harmonized Social Cash Transfer Program. He has ample 
expertise in social cash transfer programs from his current work in Malawi, Ghana and Zimbabwe, and 
the mid-term evaluation of Mexico’s PROGRESA/Oportunidades program he led when he was Director of 
the Center for Evaluation Research and Survey in Mexico’s National Institute of Public Health (INSP) in 
2010-2011. Dr. Angeles worked on impact evaluations of Mexico’s Secretary of Social Development cash 
transfers to the elderly and childcare support to poor working mothers’ programs, both with 3ie 
funding. He served as Deputy Director of MEASURE Evaluation (2000-2010), providing strategic and 
technical advice to USAID on evaluation and capacity building. In that role, he was technical lead of 
health program evaluations in Bangladesh, Tanzania, Indonesia, Nicaragua, Paraguay, Mexico, Ecuador 
and Peru. Additional research includes measurement and estimation methods for impact evaluations of 
health programs.
Lisa Calhoun, MPH, is the Quantitative Component Manager for the PS3 Evaluation. She is responsible 
for the development and implementation of the quantitative component of the PS3 Evaluation. Her 
responsibilities include development of a framework for collecting quantitative data; oversight of the 
collection of quantitative data and information; liaising with key stakeholders in Tanzania to obtain 
quantitative data; and the organization and development report describing findings (and aim for 
publication). She will also work closely with the data manager and principal investigators to clean 
analyze and report on evaluation findings. Ms. Calhoun has more than 10 years’ experience in 
international health research and evaluation with a focus on maternal and child health. Most recently, 
at the University of North Carolina at Chapel Hill (UNC), she provided technical oversight and 
management of an impact evaluation of family planning programs in India and Kenya. Prior to her work 
at UNC, Ms. Calhoun lived and worked in East Africa as a Fulbright Scholar, at the U.S. Centers for 
Disease Control and Prevention and the University of California San Francisco, and the Carter Center, 
where her research focused on infectious diseases such as malaria, guinea worm disease and HIV/AIDS. 
Ms. Calhoun has a MPH in Epidemiology from the University of Michigan.
Jessica A. Fehringer, PhD, MHS, is the Qualitative Component Manager. Her responsibilities for this 
work include management and design of all aspects of the qualitative component, including tool design, 
training and collaboration with the qualitative local research partner; oversight of analysis; and report 
generation. Dr. Fehringer is responsible for the oversight of gender-related research activities as well as 
for leading or co-leading several evaluation and research projects in South Asia and sub-Saharan Africa. 
She has a PhD from Johns Hopkins University School of Public Health in Population, Family and 
Reproductive Health. She has worked in international public health for over 15 years, has designed, and 
implemented qualitative and quantitative research and evaluation in South and Southeast Asia, South 
America, and sub-Saharan Africa. She is currently leading a quasi-experimental mixed methods 
evaluation examining the MNCH/FP and health service impacts of integrating gender and social inclusion 
130 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 15 
into health governance activities in Nepal; leading an experimental impact evaluation of a household 
economic strengthening and HIV health services project in Rwanda; and acting as gender advisor and 
qualitative lead for the design and survey implementation of a mixed methods quasi-experimental 
evaluation examining the gender dimensions of impacts of ground nut value chain interventions in 
Zambia.
Peter M. Lance, PhD is an economist. He will provide advice for the development of the quantitative 
design; oversee information collection for the quantitative component; and will participate in the 
production of a report describing findings (and aim for publication). Dr. Lance earned his PhD in 
Economics from the University of North Carolina at Chapel Hill with a focus on health, econometrics and 
economic development. Since earning his PhD, Dr. Lance’s work has focused on program impact 
evaluation, modelling of health behaviors and outcomes and survey sampling. He has served as a 
sampling expert and analyst at all phases of many surveys. His work has involved China, India, Thailand, 
Bangladesh, Nigeria, Senegal, Kenya and Indonesia.
Data Manager/Analyst (TBD). S/he will manage data, produce tables, and assist with report writing.
Qualitative Data Analyst (TBD). S/he will assist with data collector training, help monitor data collection, 
contribute to qualitative data analysis and assist with report writing.
Ann M. Fitzgerald, MPH, MSTOM is the UNC Country Portfolio Manager (CPM) for Tanzania. Ms. 
Fitzgerald will monitor compliance with MEASURE Evaluation Phase IV Agreement conditions and 
approved work plan deliverables. Ms. Fitzgerald has more than 30 years of experience in the field of 
international health and development as chief of party and technical advisor. Her technical areas of 
experience include integrated HIV policy, prevention and NGO strengthening; social and behavior 
change communication and youth programming. Her skill sets include operations research, survey 
planning and implementation, capacity building and training, and integrated approaches to health and 
medicine, project management and contracts.
 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 131
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 16 
Annex 1: Concept note/evaluation design 
COP16 Impact Evaluation Concept Note
USAID Tanzania
REVISED Oct 20, 2017 
IE Title:
Evaluation of the Public Sector System Strengthening (PS3) Program in Tanzania
Name of program/intervention being evaluated:
Public Sector System Strengthening (PS3) Program
Principal investigator:
Gustavo Angeles, PhD, MEASURE Evaluation, University of North Carolina (Contact: 
gangeles@email.unc.edu)
Country team contacts:
Shannon Young/USAID shyoung@usaid.gov
Jonathan Young/USAID jwyoung@usaid.gov
Todd Koppenhaver/USAID tkoppenhaver@usaid.gov
Moses Busiga/USAID mbusiga@usaid.gov
Implementing Agency:
USAID
 
Implementing mechanism (evaluation): MEASURE Evaluation (Contact: heidi_reynolds@unc.edu)
Implementing mechanism (PS3 intervention): Abt Associates
Start and end dates of agreement for the IE implementing mechanism:
Start date: July 1, 2014
End date: June 28, 2019 
 
132 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 17 
Specific Aims 
PS3 is a system-level intervention that seeks to improve the devolved public system’s operational 
capacity with the ultimate objective of improving the delivery of public services. MEASURE Evaluation 
will extract data from routine information systems and conduct time series analysis at the Local 
Government Authority (LGA)-level to understand how system-level indicators related to human 
resources, finance and health service utilization have changed over time in PS3 regions; MEASURE 
Evaluation will also examine if there are differences in the changes over time observed in LGAs of PS3 
regions and in LGAs where the government or other agencies implement similar PS3-like system 
strengthening interventions. Key informant interviews and focus group discussions with stakeholders 
and focus group discussions with community members will be held to contextualize the quantitative 
results for a more comprehensive understanding of the PS3 program outcomes and illuminate potential 
causal mechanisms associated with the observed outcomes, as well as to assess the quality of 
implementation. 
Specifically, MEASURE Evaluation seeks to answer the following questions: 
1. In PS3 regions, to what extent the uptake of health services (and related indicators of service 
utilization, coverage of health services, quality of services and access) changed over time? 
2. In PS3 regions, to what extent were financial and human resource systems strengthened over 
time? 
3. Are there differences in the changes over time observed in certain groups of LGAs versus 
others (e.g., those receiving PS3 versus those in non-PS3 regions but receiving similar 
interventions from the government or from other development agencies; those PS3 LGAs 
receiving the “robust” PS3 package of services versus the “standard” PS3 package; or those with 
different timing of implementation) 
4. How do stakeholders perceive the performance and impact of PS3 on different aspects of 
service provision, governance and citizen engagement? 
The operational outcomes of interest will focus on the human resources and finance systems of the 
LGAs that are targeted by the key components of PS3 intervention. Service delivery outcomes 
considered will be primary services provided by the LGAs, such as MNCH, HIV/AIDS, malaria and TB. We 
will examine service utilization, coverage of health services, quality of services and access, to the extent 
that relevant indicators on those service aspects are available in routine information data sources of the 
country. The underlying hypotheses to be examined in this evaluation are, first, if PS3 provides technical 
assistance to better manage financial resources, then the LGA-level financial system will be 
strengthened and will have better performance; second, if the human resources systems are 
strengthened, then there will be improved availability of human resources in the targeted LGAs; and, 
third, if public systems are strengthened, then the LGAs’ delivery of health services will be increased. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 133
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 18 
Quantitative methods: 
MEASURE Evaluation will utilize data from routine information systems to conduct an analysis to 
understand how key human resources, finance systems, and service delivery indicators have changed 
over time before and after the implementation of PS3. Given that the PS3 project works at the LGA-level 
to strengthen their systems, we will collect and compile data from several information systems at the 
LGA-level for outcome monitoring over time. Data will be collected from all LGAs in the country, and 
from 2013 to 2018. This will allow us to examine changes before-and-after program implementation and 
across subsets of LGAs. 
National, LGA-level data will allow us, first, to identify the LGAs in the PS3 regions and to examine how 
they, as a group, have changed over time; then, second, we will examine if any observed changes in the 
PS3 regions vary relative to the changes in LGAs in non-PS3 regions where the government or other 
development agencies are implementing similar systems strengthening interventions. Third, we will also 
examine variations by subsets of LGAs, such as those LGAs which receive the “robust” package of PS3 
support compared to those LGAs that will receive the minimum “standard” package of PS3 support. We 
could also consider creating groupings based on the timing of implementation, such as LGAs that 
received early implementation of PS3 versus those that received later implementation of PS3. 
We will monitor indicators from 2013 to 2018 to examine if there are any changes in the trend at the 
time of the introduction of the PS3 interventions. This time frame includes a PS3 pre-implementation 
phase (2013 – July 2015) and a PS3 implementation phase (July 2015 – 2018)10. 
To conduct the analysis of trends we will use a multi-year multi-group difference-in-differences (DID) 
modelling approach. This modelling framework provides great flexibility to examine trends over time for 
each of different groups of analysis and to examine differences of those trends between groups. It also 
allows to examine if there were changes in trends before and after the program was implemented. This 
analysis will show if observed trends in indicator change significantly over time and whether these 
trends are positive or negative. 
This analysis will be conducted in the statistical software Stata. Data will first be extracted from routine 
information systems, cleaned and imported into Stata. 
In addition, we will compile an indicator guide for the key set of indicators for this evaluation. The 
indicator guide will include the definition of each indicator, the source(s) of information, how the 
indicator is calculated, frequency of reporting, and any specific procedures that the information systems 
use to compile and aggregate data to calculate the key indicators. We will also undertake a search and 
 
10 We will use 2013 as the starting point of the analysis because that is the year when DHIS2 transitioned 
completely to an electronic system which facilitates accessing and processing the data. It also provides 
about 1.5 to 2.5 years of data prior to the PS3 project inception. We use 2018 as the upper year for the 
analysis only because MEASURE Evaluation ends in mid-2019 and can only collect data through the end 
of 2018. However, given that PS3 ends in 2019, we strongly recommend the Mission continues collecting 
data through the end of 2019 and to conduct the analysis right after PS3 conclusion through a different 
contractual mechanism. That will allow more time for the PS3 interventions to generate changes in 
relevant outcomes.
134 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 19
review of data quality assessments for the key indicators of interest. This will very likely require 
examining existing data quality assessments for the each of the information systems containing the key 
indicators. We will summarize the findings of any DQAs that have been undertaken as they pertain to 
the key indicators.
Results framework and indicators: The operational and service indicators which will be utilized for the 
time series analysis are driven by the results framework for the PS3 program (Figure 1). The results 
framework shows the expected relationship between program exposure and the outcome measures. 
The program is expected to promote the delivery and use of health services, such as increasing the 
numbers of people tested and treated for HIV and other communicable diseases, through strengthening 
and supporting government systems related to the following components: governance and citizenship 
engagement, human resources, finance and information systems. Broadly, these are proximate LGA￾level capacity indicators and the ultimate outcome of LGA-level service use. 
Figure 1. PS3 Results Framework (source: PS3 Year 2 workplan)
A preliminary short list of proposed indicators by area are listed below. Additional indicators will be 
explored and included in the analysis once availability of data has been confirmed.
1. Human resources: 
a. Percentage of targeted LGAs with nurse/nurse-midwife ratio >= 3/10,000
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 135
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 20 
b. Percentage of targeted LGAs with pupil-teacher ratio <= 45 in primary schools 
c. Proportion of approved permits that are vacant, by sector 
2. Finance:
a. Percentage of targeted LGAs with clean audit reports 
b. Percent change in own source revenue generation in targeted LGAs Budget execution 
rate by LGA (by sector) 
c. Percentage of public payments made through contracts with private providers 
d. Percentage of approved budget transferred from national level to LGA level 
3. Health service delivery
a. Numbers of persons living with HIV (adults and children) newly initiated on treatment 
(ART) by LGA 
b. Antenatal care visit coverage by LGA 
c. Proportion of OPD confirmed malaria cases by LGA 
Data sources: Tanzania has numerous routine sources for LGA-level data to support this evaluation. 
These include the PlanRep, DHIS2/HMIS, EPICOR, HCMIS/Lawson, and BEMIS databases. Table 1 
describes these data sources and the routine information included in the database. We will also obtain 
information outputted from the above-mentioned databases to collect indicators that USAID/Tanzania 
deems critical but are not presently available to us in established routine data systems. We will collect 
routine data from 2013 to present in order to cover the pre-implementation time period (2013-July 
2015) as well as the program implementation period (July 2015-present). 
This evaluation relies heavily on data from existing routine information sources, which is why the budget 
is not higher. However, some de novo data collection of qualitative data will be needed, as explained in 
detail below. 
Threats to validity: We do not anticipate problems with retention in the study because the unit of 
analysis is the LGA, and the data is routine existing data. Inadequate exposure to PS3 is a possibility, but 
exposure will be carefully monitored by the intervention implementing partner for PS3, Abt Associates. 
It’s possible that the quality of the routine data to be used is so low that it will present challenges for 
use in the DID modelling framework. 
Table 1. Data sources, frequency of data collection and information available
Data sources Frequency 
of data 
collation
Information contained in data 
sources
Outcomes Summary of 
information 
availability
DHIS2/HMIS Monthly Information on: infectious 
diseases (HIV, TB, Malaria), MNCH, 
IPD, OPD, tracer Medicines, HR, 
GBV, cervical cancer, and other 
aggregate routine data, etc. 
Proximate 
outcomes 
related to HR 
Complete 
information in 
DHIS2 from 2013 
to present (some 
136 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 21 
Intermediate 
outcomes 
related to service 
use 
in excel some in 
database) 
2012 and earlier 
will be in Excel 
Hard copies of 
disaggregated 
data available 
EPICOR Monthly Detailed information on LGA level 
budgets and expenditures, can be 
disaggregated by category of 
expenditure 
Proximate 
outcomes 
related to 
Finance 
System rolled 
out in 2012, prior 
to this will be 
Excel 
Likely can 
receive outputs 
from the system 
but not access to 
the system 
Reasonable 
quality data 
PlanRep Yearly Information on: Plans for MNCH, 
IPD, OPD, training institutions, CBI, 
status of health facilities, P4P 
annual performance indicators, 
HRH, drugs & supplies, sources of 
funds, Council health performance 
indicators, financial reports, etc. 
Proximate 
outcomes 
related to HR 
and finance 
Intermediate 
outcomes 
related to service 
use 
System revised 
in 2017 (by PS3) 
and in 
2010/2011 so 
will have data 
from them, 
available in 
hardcopy prior 
to 2010 
Managed by PO￾RALG and will 
need to request 
at that level 
HCMIS/Lawson Monthly Human resources Proximate 
outcomes 
related to HR 
Currently using 
v9 which started 
in 2012 
Used v7 since 
2000 
Possible to 
request reports 
of specific 
aggregated 
information 
Good quality 
data 
4 focus sectors: 
Education, 
agriculture, 
health, livestock 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 137
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 22 
BEMIS Monthly 
Annually 
Data on schools, teachers, pupils Proximate 
outcomes 
related to 
education 
Annual BEST 
report produced 
Appears to have 
data from 2006 
Sample size: The sample of analysis will be all of the LGAs of mainland Tanzania. Currently, there are 
approximately 169 LGAs. 
Qualitative methods: Qualitative methods will allow the evaluation to gain a more in-depth 
understanding of the intervention outcomes, to capture unintended, indirect effects of the intervention, 
to contextualize the quantitative findings, to illuminate the process of change, and to better assess the 
quality and character of the intervention implementation. They will also explore stakeholders’ 
perceptions of intervention impact and provide data to inform PS3 program follow-on development. 
MEASURE Evaluation-led qualitative data collection will take place in mid-2018 and will include key 
informant interviews, and community and stakeholder focus group discussions. The protocol will 
describe additional endline qualitative data collection recommended; this data collection would be 
outside the timeline of MEASURE Evaluation Phase IV. 
We plan for 60-70 key informant interviews with a range of stakeholders – focusing on LGA-level 
government, but also including program, regional administration and national government – to 
understand challenges and motivations involved in health systems management, potential solutions and 
perceptions of whether PS3 has addressed these; views on activity implementation and components 
that worked well versus did not and why; and perspectives on system strengthening and health service 
outcomes. We also plan to conduct 12 focus group discussions (two per each of six selected LGAs to 
explore the community’s experience with and attitudes towards accountability of and citizen 
engagement in planning and monitoring, and perceptions of health service quality and responsiveness 
and any recent changes in quality. We will explore potential use of creative approaches, such as 
projective techniques, so as to better elicit participation and get at potentially hidden attitudes. 
We will conduct stakeholder focus group discussions using Ripple Effects Mapping (REM) in LGAs (total 
of six – one per LGA) and at the national level (one). REM is a qualitative participatory group method 
useful for evaluating complex programs. It engages stakeholders to retrospectively visually map the 
intended and unintended changes due to a program and uses elements of mind mapping, Appreciative 
Inquiry, and group interviewing (8-10). A total of six LGAs in 3 PS3 regions will be purposefully selected 
according to: 1) a range of systems indicator performance and disease burden; 2) status as a “robust” vs. 
“standard” package recipient; 3) status as newly established LGA versus historical LGA; and 4) program 
geographical priority areas (Eastern Lake Cluster, Central Cluster, and Eastern Southern Highlands 
Cluster) 
We will code the transcripts from KIIs and FGDs and data from REM maps using a combination of 
deductive codes stemming from the results framework and research questions and inductive codes that 
emerge through the data collection and initial transcript and map review processes (11). The goal of 
this coding is to break up the data and then to organize it into categories that allow the comparison of 
data within and between these categories and help to develop theoretical concepts (12). Matrices will 
138 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 23 
next be developed to compare themes across sites. Prior to developing first drafts of the qualitative 
findings, researchers will process the coded transcripts, matrices, REM maps, and field notes through 
analytic memos; analytic memos are mainly conceptual narratives, documenting reflections on and 
thinking processes about qualitative data and often bringing together different data into a clear 
grouping (11). 
In addition, review of program monitoring data will be critical to understanding whether the 
interventions were implemented as planned, as changes in implementation could affect the causal chain 
leading to the outcomes. If the evaluation does not find changes in the key outcomes, this program 
monitoring data could help to explain why. Program monitoring data review will also aid in 
understanding challenges and successes of the activities as they evolved over time. We will contact the 
program staff regularly to collect program monitoring data and to ask about any changes in 
implementation plans and reasons behind changes. 
In addition, we will track implementation of the PS3 interventions in the project LGAs as well as 
implementation of PS3-like programs in the rest of the country. In the rapid expansion of the PS3 
program, several other organizations, including the Government of Tanzania, have taken over 
implementation of PS3 interventions in specific geographic areas. We will keep track of what group or 
organization is implementing the program, where they are implementing it, what specific components 
of the program they are implementing and the time frame of implementation. This will also be 
undertaken for all other PS3-like programs, which are programs that are implementing similar public￾sector systems strengthening activities. 
References 
1. Economic and Social Research Foundation, United Nations Development Programme, 
Government of the United Republic of Tanzania. 2015. Tanzania Human Development Report 
2014: Economic Transformation for Human Development. Dar es Salaam, Tanzania: GOT and 
UNDP. 
2. Tanzania Commission for AIDS (TACAIDS), Zanzibar AIDS Commission (ZAC), National Bureau of 
Statistics (NBS), Office of the Chief Government Statistician (OCGS), and ICF International 2013. 
Tanzania HIV/AIDS and Malaria Indicator Survey 2011-12. Dar es Salaam, Tanzania: TACAIDS, 
ZAC, NBS, OCGS, and ICF International. 
3. http://aidsinfo.unaids.org/ (accessed 26 April 2016) 
4. World Health Statistics 2013. World Health Organization. 
5. Ministry of Health, Community Development, Gender, Elderly and Children (MoHCDGEC), 
[Tanzanian Mainland], Ministry of Health (MoH) [Zanzibar], National Bureau of Statistics (NBS), 
Office of the Chief Government Statistician (OCGS), and ICF International 2016. Tanzania 
Demographic and Health Survey and Malaria Indicator Survey (TDHS-MIS) 2015-16. Dar es 
Salaam, Tanzania, and Rockville, Maryland, USA: MoHSW, MoH, NBS, OCGS, and ICF 
International. 
6. Masson L and Norman AS, 2009. Decentralization by devolution in Tanzania: Reflections on 
community involvement in the planning process in Kizota Ward in Dodoma. Journal of Public 
Administration and Policy Research 1(7): 133-140. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 139
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 24 
7. Adam T, Hsu J, de Savigny D, Lavis JN, Rottingen J, Bennett S, 2012. Evaluating health systems 
strengthening interventions in low-income and middle-income countries: are we asking the 
right questions? Health Policy and Planning 27(Suppl 4): iv9-iv19. 
8. Darger M, 2014. Capturing the ripples from community-driven business retention and 
expansion programs. Journal of Extension, 52(2). Available at 
http://www.joe.org/joe/2014april/tt6.php 
9. Hansen Kollock DA, Flage L, Chazdon S, Paine N, and Higgins L, 2012. Ripple effect mapping: A 
"radiant" way to capture program impacts. Journal of Extension, 50(5). Available at 
http://www.joe.org/joe/2012october/tt6.php
10. Kollock, DA, 2011. Ripple effects mapping for evaluation. Pullman, WA: Washington State 
University Extension. Available at http://extension.wsu.edu/stevens/wp￾content/uploads/sites/19/2013/12/REM.Complete.pdf 
11. Miles MM, Huberman AM, and Saldana J, 2014. Qualitative Data Analysis: a methods 
sourcebook. Arizona State University – Third Edition. 
12. Maxwell JA, 1996. Qualitative Research Design: An Interactive Approach. Thousand Oaks, CA: 
Sage Publications – Second Edition. 
 
140 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
MEASURE Evaluation - PS3 Evaluation
Revised and approved Jan 2018 25 
Annex 2: Cancelled Deliverables and Benchmarks 
Benchmarks Responsible 
Person(s)/Entity
Deliverables Current Status 
1.1 Scoping 
visit completed
MEASURE 
Evaluation
Trip report Completed 
1.2 Evaluation 
design finalized
MEASURE 
Evaluation
Evaluation 
protocol
Completed, will need to be revised based on new 
evaluation design 
1.3 Year one 
qualitative data 
collection 
completed
MEASURE 
Evaluation
In-country 
subcontractor
 Cancelled 
1.4 Year one 
RHIS and 
supplementary 
quantitative 
data collection 
completed
MEASURE 
Evaluation
In-country 
subcontractor
 Cancelled 
1.5 Workplan 
for Year 2 
completed
MEASURE 
Evaluation
Year 2 
workplan 
approved
Cancelled
1.6 Year one 
report 
completed 
MEASURE 
Evaluation
Final year one 
report
Cancelled
1.7 Year two 
RHIS and 
supplementary 
quantitative 
data collection 
completed
MEASURE 
Evaluation
In-country 
subcontractor
 Cancelled 
1.8 Year two 
qualitative data 
collection 
completed 
Cancelled
1.9 Year two 
report 
completed
MEASURE 
Evaluation
Final year two 
report 
Cancelled
1.10 
Presentation of 
findings
MEASURE 
Evaluation
 Cancelled 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 141
APPENDIX F. QUALITATIVE TOOLS
 1 
EVALUATION OF THE PUBLIC SECTOR SYSTEMS 
STRENGTHENING (PS3) PROJECT IN TANZANIA 
TOOL 1: Key Informant Interview Guide for Stakeholders 
General Information: 
Date 
Start Time 
Finish Time 
Region/LGA (if not National) 
Name of Key Informant 
Sex 
Position/Title 
Length of time in position 
Ministry/Organization/Institution (if 
applicable) 
Moderator’s Name 
Introduction: 
READ: Thank you for being here. My name is ____________ and I work with _______________. As I 
explained during the informed consent process, we are trying to understand stakeholder experience of Government of 
Tanzania/PS3 activities as well as your perceptions on impact of Government of Tanzania/PS3 activities on public 
system strenthening and service delivery outcomes. The information gathered from today’s interview will inform mid-course 
changes in PS3, development of similar programs in the future, and contribute to the evaluation of PS3. We will not use 
your name or otherwise identify you personally in the analysis and reporting of the results of our discussion today. Our 
discussion today should take about 45 minutes. 
142 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
 2 
For All GOT Stakeholders 
First, I would like to learn about the nature of your work, what is going well, and what would help to make your work 
more effective and/or efficient.
OPENING 
1. Please tell me about your main job 
responsibilities. 
1.1 Anything else?
2. With which information systems 
do you work, if any? 
2.1 For each mentioned, ask: 
o How is the system useful in your work? 
o What changes in the system, if any, would 
help to improve your work? 
o What training or guidance have you 
received, if any, on use of the system? 
o Who provided this training/guidance? 
o What, if anything, was useful in the 
training/guidance? 
o What, if anything, would you recommend 
be changed in the training/guidance? 
2.2 Probe on these systems, if not mentioned: 
o PlanRep 
o Epicor 
o HCMIS/Lawson 
o FFARS 
o LGRCIS 
o GOT-HoMIS 
o WISN plus POA 
o GWF 
o DHFF/DFF 
3. (for councilors only) What 
trainings or guidance have you 
received on your roles and 
responsibilities as a councilor? 
3.1 Who provided this training/guidance? 
a. What, if anything, was useful in the 
training/guidance? 
b. What, if anything, would you 
recommend be changed in the 
training/guidance? 
RELATIONSHIPS/INTERACTION WITH OTHER STAKEHOLDERS 
4. What other GOT 
ministries/offices/etc. do you 
interact with regularly? 
4.1 For each mentioned, ask: 
o What is working well in your 
relationship with that office? 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 143
 3 
o Tell me about any challenges in your 
relationship with that office. 
o What could help to improve your work 
with that office? 
4.2 Any other ministries/offices that you work 
with regularly? 
CHALLENGES/SUCCESSES 
5. Other than any you have already 
mentioned, what major challenges 
do you face in your work? 
4.1 What could be done to address these 
challenges? 
4.2 If not mentioned, probe on: 
o Making decisions based on 
evidence/results 
o Attention to gender issues and how 
they relate to your work 
o (for councilors) Citizen engagement 
6. Other than any you have already 
mentioned, what major success(es) 
have you had in your work? 
5.1 What helped you to achieve this success? 
5.2 If not mentioned, probe on: 
o Making decisions based on evidence 
o Attention to gender issues and how 
they relate to your work 
o (for councilors) Citizen engagement 
For GOT KIs from Program Areas 
Now we would like to learn about your experience working with Government of Tanzania/PS3. This can include 
Government of Tanzania/PS3-sponsored trainings or workshops you have attended, Government of Tanzania/PS3-
developed systems, operational manuals, and technical assistance you have used, technical assistance provided to you by 
Government of Tanzania/PS3 staff, or any other exposure you have had to Government of Tanzania/PS3 activities. 
144 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
4 
EXPERIENCE WITH PROGRAM 
7. Please tell me about your 
experience working with PS3. 
6.1 Probe: Please tell me about any PS3-
sponsored trainings or workshops you have 
attended. 
6.2 Probe: Please tell me about any PS3-
developed systems and operational manuals 
you may have used. 
6.3 Probe: Please tell me about any technical 
assistance you may have received from PS3 
staff outside of formal trainings. 
6.4 For each system, operational manual, or 
technical assistance instance mentioned, ask: 
o What, if anything, was useful? 
o What, if anything, would you 
recommend be changed? 
CHANGES DUE TO PROGRAM 
8. In your opinion, what positive 
changes, if any, have there been 
due to PS3’s work? 
7.1 Probe: On public service management? 
7.2 Probe: On service delivery? 
7.3 Probe: On systems strengthening? 
7.4 Probe: In your own work? 
7.5 Probe: In the work of others? 
7.6 For each change mentioned, ask: 
o To what do you attribute this 
change? 
o What evidence have you seen 
for this change? 
9. (if positive changes mentioned) Of 
the positive changes you 
mentioned, which do you think is 
the most significant? 
8.1 Probe: What makes this the most 
significant? 
8.2 Probe: How has this change affected 
your work, if at all? 
10. In your opinion, what negative 
changes, if any, have there been 
due to PS3’s work? 
9.1 Probe: On public service management? 
9.2 Probe: On service delivery? 
9.3 On systems strengthening? 
9.4 Probe: In your own work? 
9.5 Probe: In the work of others? 
9.6 For each change mentioned, ask: 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 145
5 
o To what do you attribute this 
change? 
o What evidence have you seen 
for this change? 
11. (if negative changes mentioned, 
ask) Of the negative changes you 
mentioned, which do you think is 
the most significant? 
10.1 What makes this the most significant 
negative change? 
10.2 Probe: How has this change affected 
your work, if at all? 
CONCERNS/FUTURE PROGRAM/OTHER PROGRAMS 
12. What concerns, if any, do you have 
about the work of PS3? 
11.1 Probe: Is there anything else? 
13. In your opinion, how could the 
next version of PS3 (a “follow-on” 
project) be more effective in 
strengthening public system 
management? 
12.1 Probe: Is there anything else? 
14. Lastly, we would like to know 
about any other public system 
management strengthening 
activities in your field. Tell me 
about any programs/activities you 
are aware of that are currently 
addressing your area of work. 
13.1 Probe: What are they doing to address 
the challenges in your work? 
13.2 How well are the programs/activities 
addressing the challenge(s)? 
a. What changes, if any, have you 
observed due to these activities? 
For GOT KIs from Control Areas 
RECENT CHANGES IN PUBLIC SYSTEM MANAGEMENT AND 
SERVICE DELIVERY 
Now we would like to learn about your opinion on public system management, service delivery and public 
systems strengthening changes in the past three years.
1. In your opinion, what positive 
changes, if any, have there been in 
these areas in the last 3 years? 
1.1 Probe: On public system management? 
1.2 Probe: On service delivery? 
1.3 Probe: On systems strengthening? 
1.4 Probe: In your own work? 
1.5 Probe: In the work of others? 
1.6 (for councilors) Probe: In citizen 
engagement? 
1.7 For each change mentioned, ask: 
146 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
6 
o To what do you attribute this 
change? 
o What evidence have you seen 
for this change? 
2. (if positive changes mentioned) Of 
the positive changes you 
mentioned, which do you think is 
the most significant? 
2.1 What makes this the most significant? 
3. In your opinion, what negative 
changes, if any, have there been in 
these areas in the past 3 years? 
3.1 Probe: On public system management? 
3.2 Probe: On service delivery? 
3.3 Probe: On systems strengthening? 
3.4 Probe: In your own work? 
3.5 (for councilors) Probe: In citizen 
engagement? 
3.6 Probe: In the work of others? 
3.7 For each change mentioned, ask: 
4 To what do you attribute this change? 
5 What evidence have you seen for this 
change? 
4. (if negative changes mentioned, 
ask) Of the negative changes you 
mentioned, which do you think is 
the most significant? 
4.1 What makes this the most significant 
negative change? 
FUTURE PROGRAM/OTHER CURRENT PROGRAMS 
5. Imagine there is a new program to 
support public system 
management. Besides anything 
you have already shared, what 
would you recommend this 
program do to support you in your 
work? 
5.1 Probe: Anything else? 
5.2 (for councilors) Probe: Anything on 
citizen engagement? 
6. Lastly, we would like to know 
about any current public system 
management strengthening 
activities in your field. Tell me 
about any programs/activities you 
are aware of that are currently 
addressing your area of work. 
6.2 Probe: What are they doing to address 
the challenges in your work? How well 
are the programs/activities addressing 
the challenge(s)? 
6.3 Probe: What changes, if any, have you 
observed due to these activities? 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 147
7 
For PS3 Staff Only 
OPENING 
1. First, I would like to learn about 
the nature of your work, what is 
going well, and what could be 
improved. 
1.1 Please tell me about your main PS3 job 
responsibilities. 
2. Please tell me about any major 
changes you have needed to 
make in implementation plans. 
2.1 For each change ask: 
o What led to the need for this 
change? 
o What effects of the change 
have you seen, if any? 
CHALLENGES/SUCCESSES 
3. Please tell me about your major 
successes in your work to date. 
3.1 Probe: Anything else? 
4. Please tell me about the main 
challenges you have faced in 
your work. 
4.1. Probe: Any other challenges to share? 
CHANGES DUE TO PROGRAM 
5. In your opinion, what positive 
changes, if any, have there been 
due to PS3’s work? 
5.1 Probe: Any other positive change(s)? 
5.2 Probe: In public system management? 
5.3 Probe: In service delivery? 
5.4 For each change mentioned, ask: 
o To what do you attribute 
this change? 
o What evidence have you 
seen/heard for this 
change? 
4. (if positive changes mentioned) Of 
the positive changes you mentioned, 
which do you think is the most 
significant? 
4.1 Probe: What makes this the most 
significant? 
148 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
8 
5. In your opinion, what negative 
changes, if any, have there been due to 
PS3’s work? 
5.1 Probe: Any other negative change(s)? 
5.2 Probe: In public system management? 
5.3 Probe: In service delivery? 
5.2 For each change mentioned, ask: 
i. To what do you attribute 
this change? 
ii. What evidence have you 
seen/heard for this 
change? 
6. (if negative changes mentioned) Of 
the negative changes you mentioned, 
which do you think is the most 
significant? 
6.1 Probe: What makes this the most 
significant negative change? 
CONCERNS/FUTURE PROGRAM/OTHER PROGRAMS 
7. What concerns, if any, do you have 
about the work of PS3? 
7.1 Probe: Anything else? 
8. How could the next version of PS3 
(a “follow-on” project) be more 
effective in strengthening public 
system management? 
8.1 Probe: Is there anything else the next 
program could do to improve public system 
management? 
9. Lastly, we would like to know 
about any other public system 
management strengthening 
activities in your field. Tell me 
about any programs/activities you 
are aware of that are currently 
addressing public system 
management. 
9.1 Probe: What are they doing to address the 
challenges in public system management? 
9.2 How well are the programs/activities 
addressing the challenge(s)? 
8.3 What changes, if any, have you observed 
due to these activities? 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 149
 9 
Before we end our session, I would like to ask if you have any questions or 
other thoughts you would like to share with me. (Allow room for thoughts, 
questions, comments) 
Thank you so much for your time.
For International Organization Staff 
OPENING 
1. First, I would like to learn about the nature of your 
work. 
1.2 Please tell me about your main 
job responsibilities. 
2. Please tell me about your 
experience working with PS3. 
6.1 Probe: Please tell me about any 
PS3-sponsored trainings or 
workshops you have attended. Any 
meetings? 
6.2 Probe: Please tell me about any 
PS3-developed tools, including 
guidelines, online data dashboards 
and information systems, you may 
have used. 
6.3 Probe: Please tell me about any 
technical assistance you may have 
received from PS3 staff outside of 
formal trainings. 
6.4 For each 
training/workshop/tool/technical 
assistance instance mentioned, ask: 
o What, if anything, was 
useful? 
o What, if anything, 
would you 
recommend be 
changed? 
150 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
 10 
CHANGES DUE TO PROGRAM 
3. In your opinion, what 
positive changes, if any, have 
there been due to PS3’s 
work? 
5.1 Probe: On public service 
management? 
5.2 Probe: On service delivery? 
5.3 Probe: In your own work? 
5.4 Probe: In the work of others? 
5.5 For each change mentioned, 
ask: 
o To what do you 
attribute this change? 
o What evidence have 
you seen for this 
change? 
4. (if positive changes 
mentioned) Of the positive 
changes you mentioned, 
which do you think is the 
most significant? 
8.3 Probe: What makes this the 
most significant? 
5. In your opinion, what 
negative changes, if any, have 
there been due to PS3’s 
work? 
9.7 Probe: On public service 
management? 
9.8 Probe: On service delivery? 
9.9 Probe: In your own work? 
9.10 Probe: In the work of others? 
9.11 For each change mentioned, 
ask: 
o To what do you 
attribute this change? 
o What evidence have 
you seen for this 
change? 
6. (if negative changes 
mentioned, ask) Of the 
negative changes you 
mentioned, which do you 
think is the most significant? 
10.3 What makes this the most 
significant negative change? 
CONCERNS/FUTURE PROGRAM/OTHER PROGRAMS 
7. What concerns, if any, do 
you have about the work of 
PS3? 
11.1 Probe: Is there anything else? 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 151
 11 
8. How could the next version 
of PS3 (a “follow-on” 
project) be more effective in 
strengthening public system 
management? 
12.1 Probe: Is there anything else? 
9. Lastly, we would like to 
know about any other public 
system management 
strengthening activities in 
your field. Tell me about any 
programs/activities you are 
aware of that are currently 
addressing your area of work. 
5.6 Probe: What are they doing to 
address the challenges in your 
work? 
5.7 How well are the 
programs/activities 
addressing the challenge(s)? 
b. What changes, if any, 
have you observed due to 
these activities? 
Before we end our session, I would like to ask if you have any questions or other 
thoughts you would like to share with me. (Allow room for thoughts, questions, 
comments) 
Thank you so much for your time.
152 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
June 28, 2018 
 1 
EVALUATION OF THE PUBLIC SECTOR SYSTEMS 
STRENGTHENING (PS3) PROJECT IN TANZANIA 
TOOL 2: LGA Councilor Group Discussions using Ripple Effects 
Method 
Date: 
Location (LGA): 
Participant number (m/f): 
Wards of participants: 
Participant 
Number 
Sex (M/F) Ward 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
Team: 1 Facilitator, 1 “Mapper”, 1 Note/Quote-taker 
To Bring: 
 Sign-in sheet 
 Agendas for participants 
 Informed Consent forms 
 Facilitator Guide 
 Watch/clock/timer 
 Flip chart 
 Multiple color marker(s) 
 Tape/sticky tack 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 153
June 28, 2018 
 2 
Introductions [2 minutes] 
READ: Welcome. Thank you for taking the time and making the effort to be with us. Our names are 
____________ and we work with _______________. As we explained during the informed consent process, we 
are trying to understand councilors’ experience of Government of Tanzania/PS3 activities as well as your perceptions 
on impact of Government of Tanzania/PS3 on public systems strengthening and public service delivery outcomes. The 
information gathered from today’s discussion will inform mid-course changes in Government of Tanzania/PS3, 
development of similar programs in the future, and contribute to the evaluation of PS3. The method we will use today 
provide a method of illustrating to stakeholders the ripple effects of Government of Tanzania/PS3, validating the 
effects of PS3, and creating stronger support and public value. We will not use your names or otherwise identify you 
personally in the analysis and reporting of the results of our discussion today. 
Ground Rules [1 minute] 
We encourage everyone to participate, knowing there are no right or wrong answers. Each of you was invited here 
because you have been a participant and are aware of things that have happened in your community as an LGA 
councillor. Speak up if you agree or disagree with someone, as we want to understand everyone’s experiences and 
opinions. We have note-takers, to record this discussion and help us capture everything we need for the analysis and 
summary, but we will not identify you personally. This discussion will probably take about two hours. Do you have 
any questions before we get started? 
Appreciative Inquiry interview [20 minutes] 
Today will be composed of two parts: first a peer interview, and second a group reflection. In the first part we will ask 
you to pair up with someone who you don’t know or is not a good friend. You will interview each other, asking each 
other a set of standard questions that are on your agenda. You may ask tailored follow-up questions for clarification of 
your partner’s response. You’ll have 20 minutes to interview each other- so each interview should last 10 minutes. 
We’ll announce when it is time to switch. We’ll all come back together and share in the next part, so feel free to take 
notes as you’re interviewing. Do you have any questions before we break out and start our interviews? 
Instructions for participants: 
1. Find a partner (not a good friend) 
2. Share your experiences with PS3 training and mentoring following these questions: 
a. Tell me a story about how you have used the information, and/or tools received 
through Government of Tanzania/PS3 councilor training and mentoring. 
b. What is an achievement or success you had based on your experience with/learning 
through Government of Tanzania/PS3 training and mentoring—what made it 
possible? What did this achievement lead to? 
c. What new or deepened collaborations with others (individuals, community 
organizations, government, etc.) have you made as a result of these efforts? What did 
these connections lead to? 
154 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
June 28, 2018 
 3 
d. What unexpected things have happened as a result of your involvement in 
Government of Tanzania/PS3 councilor training and mentoring? 
e. Describe any challenges you have observed with the PS3 activities. 
Mapping [45–75 minutes] 
Now that everyone has had a chance to interview and be interviewed, we’re going to pull all this information together. 
This will give us a chance, as a group, to reflect on the various strengths and challenges associated with the program. 
You may want to look at your own experience and how it compares to others’ experiences with the program. If you’re 
listening to someone else share and have a similar experience but maybe a different explanation or would like to add 
something, I encourage you to raise your hand and share. We’ll also be drawing out these experiences on the flip-chart 
so we can see how one event effects other things or ripples out. 
Instructions for facilitator: 
1. Ask each pair to offer one story (only one at a time so everyone has an opportunity to 
share), and ripple it out (draw out some of the details), welcoming input from all. Use a 
different marker color for each ripple. 
Probing questions can include: 
x Then what happened? 
x Who was involved? 
x What skill, approach, or tool from Government of Tanzania/PS3 training, if any, 
was involved? 
x What are people doing differently? 
x How have relationships changed as a result? 
2. Continue until at least one story from each pair has been captured and rippled. 
Tips for successful mapping: 
x Individual learning and action items may be the easiest to get the conversation going. 
x When mapping, get as many details as possible. 
x After you collect information for the “map,” allow opportunities for participants to add 
further and give greater detail. This provides ideas about how to dig deeper. You may also 
get people who have different ideas- those should be noted on the map. 
x People may be shy about sharing their own experiences, perceptions, or successes, but may 
be more willing to talk about those of other councilors exposed to PS3 training and 
mentoring. 
Reflection [5–15 minutes] 
Before we leave, we want you to reflect on this mapping process. Think on all the stories and experiences we heard. 
Instructions for facilitator: 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 155
June 28, 2018 
 4 
1. *Ask the group to identify the most significant change(s) on the map. 
2. What is most interesting about the map? 
3. How might we use the map to help us tell our story about how Government of 
Tanzania/PS3 is making a difference? 
4. *What other effects or impacts would you like there to be? What do you wish would 
happen? (are there things from your action plan that have not yet happened you think are 
important?) 
5. What should we do next? (can refer to use of mapped information or, with more time, use of 
the map to identify impact concentration and gaps) 
Closing [10–15 minutes] 
Thank you again for joining us today and participating! The pictures we drew today will be digitized and anlayzed. 
They will also be photographed if you would like the photo. The final report from this study will also be available 
electronically in mid-2019 and we will send that to you as well. If you would like, please provide your email contact so 
that we can make sure to send these materials to you. 
NOTE TO FACILITATOR: TAKE A PHOTO OF THE MAP 
156 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
June 21, 2018 
 1 
EVALUATION OF THE PUBLIC SECTOR SYSTEMS 
STRENGTHENING (PS3) PROJECT IN TANZANIA 
TOOL 3: Community Focus Group Guide 
Names of facilitators: 
Date: 
Location: 
Number men: 
Number women: 
Approximate ages: 
Introduction: 
READ: Welcome. Thank you for being here. Our names are ____________ and we work with 
_______________. Today we are in ____ region, ____ district, _____ division, ______ ward, _____ village. 
We wish to talk to the group now. We are talking to _________ women/men, approximate ages _____. As we 
explained during the informed consent process, we are trying to understand the quality of public services in Tanzania 
and how citizens are able to influence these services. We are especially interested in if people are able to hold the local 
government authorities accountable and how those authorities respond. You have been recruited to participate in this 
discussion because of your role on public service committees in your community. The information you provide will be used 
develop a program that aims to better respond to citizens needs and concerns and incorporates community-level opinions 
into policy and planning. We will not use your names or otherwise identify you personally in the analysis and reporting 
of the results of our discussion today. 
Ground Rules: 
READ: We encourage everyone to participate, knowing there are no right or wrong answers. Speak up if you agree or 
disagree with someone, as we want to understand everyone’s experiences and opinions. We will record this discussion to 
help us capture everything we need for the analysis and summary, but we will not identify you personally. Does everyone 
agree to record this discussion? This discussion will probably take about an hour. Do you have any questions before 
we get started? 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 157
June 21, 2018 
 2 
Opening Questions-intended to build rapport 
1. Let’s start with thinking about how you and your family have personally used health services in 
this area. How would you describe them? 
a. What are the strengths of the health services in this area? 
i. How are those strengths helpful to you? 
b. How could the health services in this area be improved to better meet your needs? 
Main Questions 
 
HEALTH CARE SERVICES 
2. I am going to tell you a story. 
Let’s say one of your children 
were sick and needed to go to 
the clinic, but you arrive and 
there are no medical staff to see 
you that day and you have to 
return the next day. A few days 
later you hear from a neighbor 
they have had a similar 
experience. Another friend went 
to the clinic and the dispensary 
was out of the medicine they 
needed. What would you do in 
this situation? 
2.1 If they indicate they would do something: 
i. Who would you go talk to and why? 
o Are there any other offices or committees you 
would go to? (for example 
ii. What would happen as a result of (the response they 
gave above for action taken) 
iii. If it isn't mentioned, probe for specifics based on rural urban 
location
x Urban areas: Ward development committees, 
Mtaa committee, Community Health 
Management Committee, Health Facility 
Committee, LGA, health facility 
x Rural areas: Ward development committees, 
Village Executive, Hamlet leader, LGA, health 
facility 
2.2 For those who wouldn't do anything if the health 
services were inadequate, why wouldn't you take 
action? 
2.3 We talked about actions that you might take if you
were in this situation. As a committee member, what 
would you do if someone came to you with this 
information? 
i. Who would you talk to and why? 
ii. What decisions would need to be made? Who 
would make those decisions? What might happen 
as a result? 
158 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
June 21, 2018 
 3 
LGA PUBLIC SERVICE SYSTEMS and MANAGEMENT 
3. As a community committee 
member, how have the LGA 
public service systems and 
management changed in the past 
five years, if at all? 
3.1 Probe for changes within: 
x Health systems 
x Education systems 
x Your committee functioning 
3.2 Tell me about any successes in these system changes. 
3.3 Tell me about any challenges in these system changes. 
3.4 For each change, probe, to what do you attribute 
these changes? 
ENGAGEMENT WITH THE LOCAL GOVERNMENT 
4. What, if anything, has been done 
in the last 5 years to help you 
engage with local government. 
4.1 For everything listed, ask: 
i. Was it effective? 
ii. If yes, what made it effective? 
iii. If no, What made it fail? 
4.2 If Opportunities and Obstacles to Development not mentioned, 
probe: Was there an Opportunities and Obstacles to 
Development process? What was involved in that 
process? 
4.3 Some people have talked about (I have heard about) 
programs like complaint registers or suggestion boxes 
posted at facilities or in public buildings to get 
people’s suggestions about how to improve services. 
What has been your experience with those? 
i. Does/did anyone use complaint registers or 
suggestion boxes? 
ii. What works/worked about that process? 
iii. What didn’t work? 
iv. If they have never had suggestion boxes: What do 
you think of the idea of suggestion boxes? 
What would make you contribute to the 
complaint register or suggestion boxes vs. 
Talking to an elected official? 
CITIZEN ENGAGEMENT AND TRANSPARENCY 
As I mentioned earlier, what you share today will be used to help a program to strengthen citizen engagement and improve 
transparency of local government authorities in Tanzania. One of the keys to doing this is better communication with 
citizens. 
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 159
June 21, 2018 
 4 
5. As committee members, what 
changes, if any, have you seen in 
the last five years related to 
citizen engagement and 
accountability? 
5.2 What are the preferred communication channels? 
i. If they are not mentioned, probe for: media, drama, street 
drama, Hamlet/vitongoji leaders 
x What makes this communication 
effective/successful? 
x What are some of the challenges? 
ii. What would you change, if anything, about the way 
you communicate with citizens? 
iii. In your opinion, how can GOT best use and respond 
to information communicated to them from citizens? 
5.1 Part of this project involves a website to help share 
information and decisions that are being made at the 
LGA level. What do you think of this idea? 
i. Have you used or tried to use the website? If yes, 
for what purpose? 
ii. What, if anything, do you think will make this a 
successful tool for communicating with 
Tanzanians? 
Closing Question 
Does anyone have anything else to share today? Thank you so much for your time and sharing your 
thoughts with us. They will be very helpful as the Government of Tanzania/PS3 work to strengthen the 
public services available in this community. 
160 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
APPENDIX G. CONFLICTS OF INTEREST
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 161
Disclosure of Conflict of Interest for USAID Evaluation Team Members 
Name 
Title 
Organization 
Evaluation Position? Team Leader Team member 
Evaluation Award Number (contract 
or other instrument)
USAID Project(s) Evaluated (Include 
project name(s), implementer 
name(s) and award number(s), if 
applicable)
I have real or potential conflicts of 
interest to disclose. 
 Yes No 
If yes answered above, I disclose the 
following facts: 
Real or potential conflicts of interest may include, 
but are not limited to: 
1. Close family member who is an employee of the 
USAID operating unit managing the project(s) 
being evaluated or the implementing
organization(s) whose project(s) are being 
evaluated.
2. Financial interest that is direct, or is significant 
though indirect, in the implementing 
organization(s) whose projects are being 
evaluated or in the outcome of the evaluation.
3. Current or previous direct or significant though 
indirect experience with the project(s) being 
evaluated, including involvement in the project 
design or previous iterations of the project.
4. Current or previous work experience or seeking 
employment with the USAID operating unit 
managing the evaluation or the implementing 
organization(s) whose project(s) are being 
evaluated.
5. Current or previous work experience with an 
organization that may be seen as an industry 
competitor with the implementing 
organization(s) whose project(s) are being 
evaluated.
6. Preconceived ideas toward individuals, groups, 
organizations, or objectives of the particular 
projects and organizations being evaluated that 
could bias the evaluation. 
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this 
disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, 
then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and 
refrain from using the information for any purpose other than that for which it was furnished. 
Signature 
Date 
Hannah Silverstein
Graduate Research Assistant
MEASURE Evaluation
MEASURE EVALUATION. AID-OAA-L-14-00004
Public Sector Systems Strengthening Activity
Abt Associates
AID-621-C-15-00003
 
11/12/19
162 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Disclosure of Conflict of Interest for USAID Evaluation Team Members 
Name 
Title 
Organization 
Evaluation Position? Team Leader Team member 
Evaluation Award Number (contract 
or other instrument)
USAID Project(s) Evaluated (Include 
project name(s), implementer 
name(s) and award number(s), if 
applicable)
I have real or potential conflicts of 
interest to disclose. 
 Yes No 
If yes answered above, I disclose the 
following facts: 
Real or potential conflicts of interest may include, 
but are not limited to: 
1. Close family member who is an employee of the 
USAID operating unit managing the project(s) 
being evaluated or the implementing
organization(s) whose project(s) are being 
evaluated.
2. Financial interest that is direct, or is significant 
though indirect, in the implementing 
organization(s) whose projects are being 
evaluated or in the outcome of the evaluation.
3. Current or previous direct or significant though 
indirect experience with the project(s) being 
evaluated, including involvement in the project 
design or previous iterations of the project.
4. Current or previous work experience or seeking 
employment with the USAID operating unit 
managing the evaluation or the implementing 
organization(s) whose project(s) are being 
evaluated.
5. Current or previous work experience with an 
organization that may be seen as an industry 
competitor with the implementing 
organization(s) whose project(s) are being 
evaluated.
6. Preconceived ideas toward individuals, groups, 
organizations, or objectives of the particular 
projects and organizations being evaluated that 
could bias the evaluation. 
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this 
disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, 
then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and 
refrain from using the information for any purpose other than that for which it was furnished. 
Signature 
Date 
Aimee Benson
Data Manager
MEASURE Evaluation
MEASURE EVALUATION. AID-OAA-L-14-00004
Public Sector Systems Strengthening Activity
Abt Associates
AID-621-C-15-00003
Aimee Benson
2 Dec 2019
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 163
164 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Disclosure of Conflict of Interest for USAID Evaluation Team Members 
Name
Title
Organization
Evaluation Position? Team Leader Team member
Evaluation Award Number (contract 
or other instrument)
USAID Project(s) Evaluated (Include 
project name(s), implementer 
name(s) and award number(s), if 
applicable)
I have real or potential conflicts of 
interest to disclose.
Yes No
If yes answered above, I disclose the 
following facts:
Real or potential conflicts of interest may include, 
but are not limited to:
1. Close family member who is an employee of the 
USAID operating unit managing the project(s) 
being evaluated or the implementing
organization(s) whose project(s) are being 
evaluated.
2. Financial interest that is direct, or is significant 
though indirect, in the implementing 
organization(s) whose projects are being 
evaluated or in the outcome of the evaluation.
3. Current or previous direct or significant though 
indirect experience with the project(s) being 
evaluated, including involvement in the project 
design or previous iterations of the project.
4. Current or previous work experience or seeking 
employment with the USAID operating unit 
managing the evaluation or the implementing 
organization(s) whose project(s) are being 
evaluated.
5. Current or previous work experience with an 
organization that may be seen as an industry 
competitor with the implementing 
organization(s) whose project(s) are being 
evaluated.
6. Preconceived ideas toward individuals, groups, 
organizations, or objectives of the particular 
projects and organizations being evaluated that 
could bias the evaluation. 
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this 
disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, 
then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and 
refrain from using the information for any purpose other than that for which it was furnished. 
Signature
Date
for any purpose other than that fo
Jessica Anne Fehringer
Senior Technical Advisor for Evaluation
MEASURE Evaluation
Public Sector Systems Strengthening Project
Nov 22 2019
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 165
Disclosure of Conflict of Interest for USAID Evaluation Team Members 
Name
Title
Organization
Evaluation Position? Team Leader Team member
Evaluation Award Number (contract 
or other instrument)
USAID Project(s) Evaluated (Include 
project name(s), implementer 
name(s) and award number(s), if 
applicable)
I have real or potential conflicts of 
interest to disclose.
Yes No
If yes answered above, I disclose the 
following facts:
Real or potential conflicts of interest may include, 
but are not limited to:
1. Close family member who is an employee of the 
USAID operating unit managing the project(s) 
being evaluated or the implementing
organization(s) whose project(s) are being 
evaluated.
2. Financial interest that is direct, or is significant 
though indirect, in the implementing 
organization(s) whose projects are being 
evaluated or in the outcome of the evaluation.
3. Current or previous direct or significant though 
indirect experience with the project(s) being 
evaluated, including involvement in the project 
design or previous iterations of the project.
4. Current or previous work experience or seeking 
employment with the USAID operating unit 
managing the evaluation or the implementing 
organization(s) whose project(s) are being 
evaluated.
5. Current or previous work experience with an 
organization that may be seen as an industry 
competitor with the implementing 
organization(s) whose project(s) are being 
evaluated.
6. Preconceived ideas toward individuals, groups, 
organizations, or objectives of the particular 
projects and organizations being evaluated that 
could bias the evaluation. 
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this 
disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, 
then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and 
refrain from using the information for any purpose other than that for which it was furnished. 
Signature
Date
ion for any purpose other than that for wh
Brittany Iskarpatyoti
Research Associate
MEASURE Evaluation
MEASURE EVALUATION. AID-OAA-L-14-00004
Public Sector Systems Strengthening Activity
Abt Associates
AID-621-C-15-00003
11/11/19
166 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 167
168 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Disclosure of Conflict of Interest for USAID Evaluation Team Members 
Name
Title
Organization
Evaluation Position? Team Leader Team member
Evaluation Award Number (contract 
or other instrument)
USAID Project(s) Evaluated (Include 
project name(s), implementer 
name(s) and award number(s), if 
applicable)
I have real or potential conflicts of 
interest to disclose.
Yes No
If yes answered above, I disclose the 
following facts:
Real or potential conflicts of interest may include, 
but are not limited to:
1. Close family member who is an employee of the 
USAID operating unit managing the project(s) 
being evaluated or the implementing
organization(s) whose project(s) are being 
evaluated.
2. Financial interest that is direct, or is significant 
though indirect, in the implementing 
organization(s) whose projects are being 
evaluated or in the outcome of the evaluation.
3. Current or previous direct or significant though 
indirect experience with the project(s) being 
evaluated, including involvement in the project 
design or previous iterations of the project.
4. Current or previous work experience or seeking 
employment with the USAID operating unit 
managing the evaluation or the implementing 
organization(s) whose project(s) are being 
evaluated.
5. Current or previous work experience with an 
organization that may be seen as an industry 
competitor with the implementing 
organization(s) whose project(s) are being 
evaluated.
6. Preconceived ideas toward individuals, groups, 
organizations, or objectives of the particular 
projects and organizations being evaluated that 
could bias the evaluation. 
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this 
disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, 
then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and 
refrain from using the information for any purpose other than that for which it was furnished. 
Signature
Date
tion for any purpose o
Courtney McGuire
Research Associate
MEASURE Evaluation
MEASURE EVALUATION. AID-OAA-L-14-00004
Public Sector Systems Strengthening Activity
Abt Associates
AID-621-C-15-00003
November 18, 2019
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 169
Disclosure of Conflict of Interest for USAID Evaluation Team Members 
Name
Title
Organization
Evaluation Position? Team Leader Team member
Evaluation Award Number (contract 
or other instrument)
USAID Project(s) Evaluated (Include 
project name(s), implementer 
name(s) and award number(s), if 
applicable)
I have real or potential conflicts of 
interest to disclose.
Yes No
If yes answered above, I disclose the 
following facts:
Real or potential conflicts of interest may include, 
but are not limited to:
1. Close family member who is an employee of the 
USAID operating unit managing the project(s) 
being evaluated or the implementing
organization(s) whose project(s) are being 
evaluated.
2. Financial interest that is direct, or is significant 
though indirect, in the implementing 
organization(s) whose projects are being 
evaluated or in the outcome of the evaluation.
3. Current or previous direct or significant though 
indirect experience with the project(s) being 
evaluated, including involvement in the project 
design or previous iterations of the project.
4. Current or previous work experience or seeking 
employment with the USAID operating unit 
managing the evaluation or the implementing 
organization(s) whose project(s) are being 
evaluated.
5. Current or previous work experience with an 
organization that may be seen as an industry 
competitor with the implementing 
organization(s) whose project(s) are being 
evaluated.
6. Preconceived ideas toward individuals, groups, 
organizations, or objectives of the particular 
projects and organizations being evaluated that 
could bias the evaluation. 
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this 
disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, 
then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and 
refrain from using the information for any purpose other than that for which it was furnished. 
Signature
Date
ation for any purpose 
Willis Omondi Odek
Dr.
MEASURE Evaluation
MEASURE EVALUATION. AID-OAA-L-14-00004
Public Sector Systems Strengthening Activity
Abt Associates
AID-621-C-15-00003
November 12, 2019
170 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
APPENDIX H. TEAM BIOS
Gustavo Angeles, PhD, is the activity lead and main investigator for the quantitative component of the 
Tanzania PS3 evaluation. An economist by training, he is faculty at the Gillings School of Global Public 
Health and at the Department of Public Policy, both at UNC. He is also a Fellow at the Carolina Population 
Center. Dr. Angeles has over 20 years of experience on evaluation of health and social development programs 
in Latin America, Africa, and Asia, and is currently Senior Evaluation Advisor of MEASURE Evaluation 
where he led the impact evaluation of USAID/Guatemala Western Highland Integrated Program, and coleads 
impact evaluations of health programs in Bangladesh. He also participates as instructor on a number of 
training and capacity building activities on impact evaluation methods conducted in Mexico, South Africa, 
India, Ghana, Ethiopia, and Nepal. In addition to his work with MEASURE Evaluation, he is co-PI of the 
Transfer Project, a research-learning collaboration with UNICEF, FAO, and UNICE for evaluating the 
eff ectiveness of national cash transfer programs in Africa. During 2010–2011 he was Executive Director of the 
Center for Evaluation Research and Survey at Mexico’s National Institute of Public Health (INSP), where he 
led evaluations, surveys, and analytical work for the health and social development agencies of the Government 
of Mexico. He participated in the midterm evaluation of Mexico’s PROGRESA/Oportunidades cash transfer 
program and in evaluations of cash transfers for the elderly and of childcare support to poor working mothers. 
He has conducted additional research in measurement and estimation methods for impact evaluations of 
health programs. Dr. Angeles served as Deputy Director of MEASURE Evaluation from 2000–2010. 
Jessica A. Fehringer, PhD, MHS, designed and oversaw implementation of the qualitative component 
of the Tanzania PS3 evaluation. Dr. Fehringer is responsible for leading several evaluation and research 
projects in sub-Saharan Africa. She has a PhD from Johns Hopkins University School of Public Health in 
Population, Family, and Reproductive Health and a master’s in health sciences in international health, with 
a focus on social and behavioral interventions, from the Johns Hopkins University School of Public Health. 
She has worked in international public health for over 15 years, designing and carrying out qualitative and 
quantitative research and evaluation in South and Southeast Asia, South America, and sub-Saharan Africa 
on a range of health topics, including HIV prevention and treatment, local health governance, maternal, 
neonatal, and child health and nutrition, and gender-based violence.
Gideon Kwesigabo, MD, MSc, MEd, PhD, is former Dean of the School of Public Health, and former 
Director of Continuing Education and Professional Development, Muhimbili University. Dr. Kwesigabo is 
an epidemiologist by training whose research interests span from HIV epidemiology and trends, maternal 
and child health, violence against women and children, nutrition, to evaluation of interventions. He has 
authored/coauthored over 50 publications in peer reviewed journals and over 45 technical and research 
reports in the areas of oncology, HIV/AIDS, female genital mutilation, violence against women, violence 
against children, stigma, social capital and maternal and child health, health systems research, and teaching 
and learning. Apart from working as the Managing Partner for the Health and Development International 
Consultants, Dr. Kwesigabo teaches and supervises both undergraduate and postgraduate students in the area 
of epidemiology. 
Willis Odek, PhD, MSc, is a multidisciplinary social scientist and strategic information expert. He has 20 
years of experience in research, monitoring, and evaluation in HIV, malaria, population and development, 
nutrition, and economic strengthening programs and health information systems. From December 2013 
to May 2019, Dr. Odek worked in Tanzania as Chief of Party/Senior Technical Advisor for MEASURE 
Evaluation, a global project funded by the U.S. Agency for International Development (USAID), focusing on 
malaria surveillance and response, health management information systems, data quality, data use, orphans 
and vulnerable children information systems, and M&E and research capacity building with academic and 
research institutions. He is currently a Strategic Information Specialist with the United Nations Population 
Fund’s East and Southern Africa Regional Offi ce (UNFPA/ESARO), in Johannesburg, South Africa. Dr. 
Odek holds a PhD in Sociology from the University of Aberdeen, Scotland, UK, an MSc in Demography and 
Health from the London School of Hygiene and Tropical Medicine, UK, and a Bachelor of Education degree 
from the University of Nairobi, Kenya.
Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report 171
Lisa Calhoun, MPH, has more than 12 years of experience in research and evaluation with a focus on 
maternal and child health globally. Ms. Calhoun brings expertise in evaluation of health programs, collection 
of population- and facility-based data, quantitative analysis, and, in addition, has a successful track record of 
publication in peer-reviewed journals. She currently provides leadership and strategic decision-making for a 
Gates Foundation-funded project aimed at generating evidence on expanded contraceptive method choice for 
adolescents and youth as well as providing monitoring, learning, and evaluation support and standardization 
for a pool of 20 projects spanning 11 countries. In addition, her recent work history focuses on management 
of an impact evaluation of family planning programs in India, Kenya, Nigeria, and Senegal, as well as an 
outcome evaluation of a Public Sector Systems Strengthening Project in Tanzania. Previously, Ms. Calhoun 
worked at the U.S. Centers for Disease Control and Prevention (CDC), the University of California San 
Francisco, the University of Michigan, and the Carter Center.
Hannah Silverstein, MPH, is a doctoral student in the Department of Maternal and Child Health and 
a Graduate Research Assistant for the PS3 midline evaluation. Ms. Silverstein is interested in policy and 
program evaluation in low- and middle-income countries, specifi cally, the impact of social protection on 
people with disabilities. She has a Master’s degree from the Gillings School of Global Public Health at the 
University of North Carolina at Chapel Hill in Maternal and Child Health. In 2016, she was awarded an 
FHI 360-UNC Research Fellowship, and worked on several projects, including USAID’s YouthPower Action 
SMART Connections online support group for adolescents living with HIV in Nigeria and the Abundant 
Health Community-Based Care for Non-Communicable Diseases P roject in Vietnam. Ms. Silverstein
also co-produced the fi lm, “Th e Footnote Film Project,” a documentary exploring the global eff ort to treat 
clubfoot. 
Brittany Iskarpatyoti, MPH, is a Technical Specialist for Monitoring and Evaluation with MEASURE 
Evaluation. Ms. Iskarpatyoti provides technical expertise in evaluation planning, training, and qualitative 
research. She has led activities triangulating program data with routine health data for improved case 
management, DQA, and capacity building in qualitative research and gender integration. Ms. Iskarpatyoti 
holds a Master’s in Public Health from Emory University and previously worked on the Gender and 
Empowerment Team at CARE and as a US Peace Corps Volunteer in Namibia.
Courtney McGuire, MPH, is a Research Associate at the Carolina Population Center at the University of 
North Carolina at Chapel Hill. Ms. McGuire has worked in this position since 2017. She has worked with 
the Measurement, Learning & Evaluation project to implement the NURHI Sustainability Study (2017–
2019). She is currently working on the Full Access, Full Choice project (2017–2021) and supports the project’s 
collaboration with the African Institute for Development Policy (AFIDEP) in Kenya as well as the 19 BMGF 
partner investments that the project supports. Before joining the Carolina Population Center, Ms. McGuire 
worked with the Johns Hopkins Center for American Indian Health as an Evaluation Manager. Based on 
the White Mountain Apache Reservation in eastern Arizona, she supported two programs: the Arrowhead 
Business Group, a youth entrepreneurship program; and Respecting the Circle of Life, a teen pregnancy 
prevention program. Ms. McGuire received her Bachelor of Arts from Th e George Washington University in 
2011 and her Masters of Public Health from Emory University in 2014. 
Dr. Joe L.P. Lugalla, is a Professor and Dean of the Institute for Educational Development, EA of the 
Aga Khan University (AKU). He is also the Associate Vice Provost of the Aga Khan University, Tanzania. 
Dr. Lugalla joined AKU from the University of New Hampshire in June 2015. He attained a Doctor of 
Philosophy Degree in Social Science from the University of Bremen in early 1990 and then a Post-Doctoral 
Diploma in Higher Education and International Development from the University of Kassel. Dr. Lugalla is 
a Professor of medical sociology, medical anthropology, globalization, global health, and development. He 
has researched and published extensively in these areas of specialization, including education. Dr. Lugalla 
has contributed to seven books (coauthored, edited volumes, and single authored) and four monographs, 
as well as many papers in peer-reviewed international journals. In addition to his teaching, Dr. Lugalla is 
also actively involved in a variety of outreach community development programs in sub-Saharan Africa, 
specifi cally in Tanzania.
172 Midline Evaluation of the Tanzania Public Sector System Strengthening Program: Final Report
Aimee M. Benson is a data processing specialist with almost 20 years of experience in developing both 
paper-and-pencil (PAPI) and computer-assisted (CAPI) data capture systems, as well as training and 
supporting fi eld personnel in their implementation. In addition to creating the data capture systems, Ms. 
Benson has participated in all stages of the data lifecycle, including questionnaire development, indicator 
harmonization across multi-country surveys, data quality assessment, data cleaning and management, 
production of analysis tables and graphs, and data archiving. She has extensive experience with survey data 
(household-level, individual-level, and facility-level), as well as experience with routine health system data 
(DHIS2 data, RDQA). Ms. Benson has also cofacilitated capacity building workshops in data management 
and analysis, including indicator construction, using STATA. Her most recent work has involved a complex 
primary CAPI data collection eff ort for an observational study of pregnancy outcomes among women in 
resource-poor environments. Ms. Benson has worked in the United States and in international data collection 
projects, including projects in Malawi, China, Bangladesh, the Caribbean, Zambia, India, Nigeria, Kenya, 
Senegal, and Rwanda. She is currently the Data Manager at UNC’s Global Women’s Health, housed in the 
Department of Obstetrics and Gynecology at the School of Medicine. Ms. Benson holds a Master’s degree in 
Experimental Psychology from Duke University.
This publication was produced with the support of the United States Agency 
for International Development (USAID) under the terms of MEASURE Evaluation 
cooperative agreement AID-OAA-L-14-00004. MEASURE Evaluation is implemented 
by the Carolina Population Center, University of North Carolina at Chapel Hill in 
partnership with ICF International; John Snow, Inc.; Management Sciences for Health; 
Palladium; and Tulane University. Views expressed are not necessarily those of USAID 
or the United States government. TRE-19-26
ISBN: 978-1-64232-214-9 
MEASURE Evaluation 
University of North Carolina at Chapel Hill
123 West Franklin Street, Suite 330
Chapel Hill, NC 27516 USA
Phone: +1 919-445-9350
measure@unc.edu
www.measureevaluation.org