PMI VECTORLINK POST-SPRAY DATA QUALITY AUDIT REPORT 2021 ROUND II SPRAY CAMPAIGN March 2022 This publication was produced for review by the United States Agency for International Development. It was prepared independently by ME&A, Inc. through the Data for Development Activity. PMI VECTORLINK POST-SPRAY DATA QUALITY AUDIT REPORT 2021 ROUND II SPRAY CAMPAIGN SUBMISSION: MARCH 2022 Contract/Agreement Number: AID-OAA-I-15-00024/AID-621-TO-17-00005 Period of Performance: August 3, 2017, to February 3, 2023 (Extended) Contracting Officer's Representative (COR): Michael McBroom Developed by Data for Development performance monitoring staff with assistance from Kepler Consultant. Submitted by: David Hughes, Chief of Party ME&A 4350 East-West Highway, Suite 210 Bethesda, MD 20814-4410 Tel: (301) 652-4334 Email: dhughes@engl.com Cover Photograph: The wife of homeowner Masenilo Faresi sharing her IRS spraying cards with PSDQA data collector Ndamiye Cheyo in Ilalwe Village, Bukombe Ward, Bukombe DC, Geita Region, February 2022 (D. Hughes). DISCLAIMER The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. TABLE OF CONTENTS Executive Summary .......................................................................................................................................... 1 Findings .................................................................................................................................................. 1 Recommendations .............................................................................................................................. 3 1.0 Introduction.......................................................................................................................................... 4 1.1 PSDQA Objectives................................................................................................................ 4 2.0 PSDQA METHODOLOGY.............................................................................................................. 5 2.1 Data Collection Method ...................................................................................................... 5 2.2 Inclusion Criteria ................................................................................................................... 5 2.3 Sampling strategy ................................................................................................................... 5 2.4 Recruitment and Training .................................................................................................... 6 2.5 Data Collection process...................................................................................................... 6 2.6 Study Limitations................................................................................................................... 7 2.7 Data Quality Assurance ....................................................................................................... 7 2.8 Data Analysis.......................................................................................................................... 8 3.0 PSDQA RESULTS ............................................................................................................................... 8 3.1 Mobilization, Spray Coverage, and IRS Card Retention ............................................... 9 3.1.1 Mobilization Coverage.............................................................................................................. 9 3.1.2 Information Given During Mobilization ................................................................................ 9 3.1.3 Spray Coverage .......................................................................................................................... 9 3.1.4 IRS Card Retention................................................................................................................. 10 3.2 Spray Evidence .................................................................................................................... 11 3.3 Reasons Why structures were not sprayed................................................................. 11 3.4 Population Protected......................................................................................................... 12 3.5 Structure characteristics................................................................................................... 13 4.0 LESSON LEARNED AND RECOMMENDATION.................................................................. 14 4.1 Lessons Learned ................................................................................................................. 14 4.3 Recommendations............................................................................................................. 16 Annexes ........................................................................................................................................................... 17 Annex 1: PSDQA Survey Team.................................................................................................... 17 Annex 2: Selected Villages.............................................................................................................. 17 Annex 3: PSDQA Data Collection Tool..................................................................................... 17 Annex 4: Data for THE Development DQA Tool................................................................... 17 Annex 5: Documents Reviewed ................................................................................................... 17 Annex 6: Structure Definition Documents................................................................................ 17 LIST OF TABLES Table 1: Overall Spray Results Before and After the Spray (Percent).................................................. 2 Table 2: VectorLink Tanzania Phase II IRS Campaign Coverage, by District...................................... 4 Table 3: Number of Structures Found and Sampled ................................................................................ 6 Table 4: Sampled Wards and Village, Disaggregated by District............................................................ 6 Table 5: Level of Information During Mobilization ................................................................................... 9 Table 6: Mobilization Coverage, Spray Coverage, and IRS Card Retention..................................... 10 Table 7: Spray Evidence, Disaggregated by District Disaggregated.................................................... 11 Table 8: Population Protected in Target Areas, by District................................................................. 13 Table 9: Structure Characteristics, Disaggregated by District (in percent)..................................... 13 Table 10: Type of Roofing Materials for Eligible Structures, by District.......................................... 13 LIST OF FIGURES Figure 1: PSDQA Phase II Coverage ............................................................................................................ 5 Figure 2: Reasons Structures Were Not Sprayed.................................................................................. 12 Figure 3: Prevalence of Open Eaves, Disaggregated by District ......................................................... 14 ACRONYMS Acronym Description CI Confidence Interval DC District Council DIEC District Information Education Coordinator DMFP District Malaria Focal Person DQA Data Quality Assessment EOSR End of spraying report IRS Indoor Residual Spraying KII Key Informant Interview LGA Local Government Authority NMCP National Malaria Control Programme PMI President’s Malaria Initiative PO-RALG President’s Office-Regional Administration and Local Government PPS Probability Proportional to Size PSDQA Post-Spray Data Quality Audit SOP Spray Operator USAID United States Agency for International Development VEO Village Executive Officer WEO Ward Executive Officer ZAMEP Zanzibar Malaria Elimination Program EXECUTIVE SUMMARY In November 2021, the U.S. President’s Malaria Initiative (PMI) VectorLink Tanzania Project conducted an indoor residual spraying (IRS) campaign in three districts: Biharamulo District Council (D.C.) in Kagera Region, Bukombe DC in Geita Region, and Ukerewe DC in Mwanza Region. The spraying campaign was completed in December 2021. The United States Agency for International Development (USAID) requested Data for Development to conduct a Post-Spray Data Quality Audit (PSDQA) of the activity in collaboration with a sub-contractor, Kepler Consultant. The purpose of the PSDQA was to assess the accuracy of spray data reported to PMI during the 2021 spray campaign phase I11 in these districts. The PSDQA field work was undertaken between February 14, 2022, and February 25, 2022. The PSDQA team selected the wards, villages, and structures to be sampled using the probability proportional to size (PPS) approach. A total of 1,141 structures were identified for assessment, and reached 1,123 (98 percent) of the sampled structures. KoBo Toolbox, installed on tablets, captured the indicator data using pre-determined questions. The team included four Data for Development staff, five Kepler Consultant staff, two VectorLink staff, two District Malaria Focal Persons (DMFPs), two President’s Office-Regional Administration and Local Government (PO￾RALG) representatives, and 16 enumerators hired by Kepler Consultant. The PSDQA team conducted the survey and key informant interviews (KIIs) in Swahili. The survey questionnaire included questions to assess mobilization coverage, spray coverage, Indoor Residual Spraying (IRS) Card retention, and the proportion of people protected. Additionally, the team conducted KIIs with spray operators (SOPs), DMFPs, District Information Education Coordinators (DIECs), and VectorLink staff to solicit lessons learned and recommendations for future PSDQA assignments. The PSDQA team analyzed the data using Microsoft Excel and Stata Software. FINDINGS The PSDQA found the overall spray coverage2 to be 90.7 percent 3 (with a 95 percent confidence interval [CI] ranging between 92.6 percent and 97.2 percent) compared with 92.2 percent spray coverage reported by the PMI VectorLink Tanzania Project, as shown in Table 1 below. This indicates an overall good performance in the spraying exercise in these three districts. The main reason for non-spraying of structures was “Nobody was present/houses were locked at the time of spraying” (38.7 percent). The PSDQA team also noted instances where revisit spraying was not undertaken to ensure all reported structures were sprayed. Mobilization before spraying helped the successful implementation of this activity since household members understood the benefit of the IRS campaign and prepared their houses for IRS. The findings show that 92.3 percent of the household members were mobilized. However, not all people understood the importance of IRS to the community. PSDQA surveys indicated some misconceptions among community members, such as that IRS promotes the emergence of 1 Phase I of IRS campaign was conducted in October-November 2021 in three districts of Kigoma Region and the PSDQA was conducted in December 2021 by Data for Development in collaboration with the same subcontractor Kepler Consultant 2 Spray coverage is calculated by the number of structures sprayed divided by the number of structures eligible for spraying in sampled villages. 3 The spray coverage exceeded the World Health Organization (WHO) threshold of 85 percent. bedbugs. There was also a lower acceptance of the campaign in Ukewere DC compared to Biharamuro and Bukombe DCs. The project engaged the District Commissioner and District Executive Officer there to persuade the community members to accept the campaign in the areas with high refusal. Household occupants of sprayed structures were able to present IRS cards at 58.9 percent of sprayed structures, 95 percent had visible markings, and 54.3 percent had insecticide residue. Overall, 31.7 percent of the sprayed structures had all four pieces of evidence available: confirmation from the interview, IRS card, house marking, and insecticide residue presence. This is less than in the first phase (50.1%), largely because of the low level of noticing residual spray in Bukombe DC which was reported at 26.6 percent. Regarding population protection, 90.8 percent (with a 95 percent CI ranging between 89.9 percent and 91.6 percent) of the target population was protected by IRS. The population coverage of 90.9 percent reported that the End of Spray Report (EOSR) falls within the PSDQA estimate CI. Table 1: Overall Spray Results Before and After the Spray (Percent) Indicator IRS Campaign (Reported) PSDQA Estimate (Verified) Mobilization coverage (%) 92.3 Spray coverage (%) 92.2 90.7 Population protected (%) 90.9 90.8 IRS card retention (%) 58.9 % of sprayed structures with household marking 95.0 % of sprayed structures with insecticide residue 54.3 % of sprayed structures with all four evidences present45 31.7 There is a good engagement and collaboration with government officials at most levels in implementing the IRS campaign. Community mobilization conducted by mobilizers, SOPs, and hamlet leaders facilitated the acceptance of the IRS intervention in most villages. However, the DQA team observed some Village Executive Officers (VEOs) were not fully engaged in mobilization and spraying activities. PSDQA findings show that the overall mobilization coverage is 92.3 percent, indicating most community members (1,037 of 1,123) were involved in IRS mobilization, with 100 percent mobilization recorded in Nyamigongo ward in Biharamulo DC and Bukungu ward in Ukerewe DC. During mobilization, residents received information, with 30 percent indicating they received comprehensive information, 39.8 percent basic information, but 30.2 percent received inadequate information. In total 1,123 (90.7 percent) of sampled structures were sprayed, with 104 (9.3 percent) of eligible structures not sprayed. Findings show that IRS card retention in all three assessed districts is at 58.9 percent, while only 356 (31.7 percent) of 1,123 surveyed structures had all four pieces 4 Confirmation from an interview, IRS card, house marking and presence of insecticide residue. of verification evidence. Structures were mostly constructed using red, burnt bricks (43.3 percent) and roofing was usually corrugated iron sheets (91 percent). Only 12 percent of sampled structures had open eaves. Findings from the PSDQA indicate that most of the residents had positive feedback with regards to the IRS exercise, with 73.8 percent of residents reporting reduced numbers of mosquitoes, 43.1 percent saying the sprayed insecticides had eliminated other insects like bedbugs and fleas, 41.5 percent saying that the exercise had gone well, and 20 percent stating they had received sufficient IRS information. However, 17.7 percent of residents said mosquitoes were still at large, and 5.5 percent said mobilization was not well done with no IRS information being provided, and 2.3 percent said they had faced some health-related challenges (not necessarily attributable to the activity). RECOMMENDATIONS 1. PMI VectorLink Tanzania Project should ensure that, during the IRS exercise, the SOPs have adequate insecticide to spray the structures allocated. Team leaders should have extra insecticide to distribute to the sprayers who finish the sachets before completing their daily spraying targets. Spray teams should follow up on unsprayed structures and revisit planned and implemented. 2. Mobilization exercises are key to the implementation success. Not only should the messaging emphasize the importance of IRS, but targets should be well explained, so residents do not assume that all structures will be sprayed and all insects controlled. The project should continue to involve all village leaders and influential community members, including Ward Executive Officers [WEOs], VEOs, religious leaders, and the Local Government Authorities (LGAs) in the respective districts, recognizing that when remuneration is not provided, there may a reluctance to participate in the mobilization. Sufficient notice before the IRS campaign will also allow community members to plan and prepare for IRS intervention and ensure they will be available. The spraying exercise should be well planned so only residents of the structures reached on a particular day remove their belongings. 3. Even though even though this is the third year of the campaign, there was low voluntary acceptance of campaigns in some areas in Ukerewe DC. This should be investigated and messaging developed and disseminated to increase acceptance of the campaign in that specific context. 1.0 INTRODUCTION The President’s Malaria Initiative (PMI) VectorLink Tanzania Project, funded by the United States Government and implemented by Abt Associates, supports the implementation of indoor residual spraying (IRS) in select regions of the Tanzania Mainland and Zanzibar. The project’s objective is to further PMI’s goal to halve the burden of malaria in 70 percent of at-risk populations in sub￾Saharan Africa. The project fits into the National Malaria Control Programme (NMCP) mission for the period 2014–2020 as well as the Zanzibar Malaria Elimination Program’s (ZAMEP) mission, both of which aim to ensure that Tanzanians have access to quality, effective, safe, and affordable malaria interventions through timely and sustainable collaborative efforts with partners and stakeholders at all levels. The United States Agency for International Development (USAID) requested Data for Development to conduct a Post-Spray Data Quality Audit (PSDQA) of the activity in collaboration with a sub-contractor, Kepler Consultant. The overall purpose of the PSDQA was to assess the accuracy of spray data reported to PMI during the 2021 spray campaign phase I1 in Biharamulo District Council (DC) in Kagera Region, Bukombe DC in Geita Region, and Ukerewe DC in Mwanza Region between November and December 2021 The PSDQA of the spraying was undertaken between February 14, 2022, and February 25, 2022. Table 2 below summarizes spray progress in the districts in Kagera, Geita and Mwanza Regions. Table 2: VectorLink Tanzania Phase II IRS Campaign Coverage, by District Region District Structur es Found Structure s Sprayed Spray Progress (%) Population Found Population Protected Kagera Biharamulo 112,414 104,391 93% 403,406 377,193 Geita Bukombe 88,379 81,868 93% 283,460 262,132 Mwanza Ukerewe 122,557 108,406 88% 429,546 375,701 Total 323,350 294,665 91% 1,116,412 1,015,026 1.1 PSDQA OBJECTIVES The objectives of the PSDQA were to: 1. Validate the spray coverage reported by VectorLink Tanzania for the 2021 spray campaign phase II in Biharamuro DC, Bukombe DC, and Ukerewe DC. 2. Validate the proportion of people protected reported by the PMI Vectorlink Tanzania Project for the 2021 spray campaign in the targeted districts. 3. Assess mobilization coverage and IRS Card retention among beneficiaries. The PSDQA team measured mobilization coverage by the proportion of respondents who reported they were mobilized before the structure was sprayed. 4. Assess the ability of data management and reporting systems to document, transmit, and report quality data. 5. Identify lessons learned and incorporate improvements to data collection, entry, and supervision in future spray campaigns. 2.0 PSDQA METHODOLOGY 2.1 DATA COLLECTION METHOD The PSDQA team conducted a cross-sectional design to collect quantitative and qualitative data in Biharamulo, Bukombe, and Ukerewe districts. The team applied the probability proportional to size (PPS) approach to randomly sample wards, villages, and structures. The PSDQA team then designed a questionnaire using Kobo Toolbox. The questionnaire captured mobilization coverage, spray coverage, spray evidence, population protection, and the characteristics of structures in each sampled village. The PSDQA team trained all enumerators on definitions/identification of the structures and data collection in general on the first day. On the second day, the team pre￾tested the tool in two villages in Bukombe, DC, which were not sampled for the actual PSDQA. Based on the pre-testing results and lessons learned, the team rephrased some questions and added others. The skip pattern was corrected, and the final questionnaires were translated from English to Swahili. 2.2 INCLUSION CRITERIA All sampled structures eligible for spray during the phase 1I campaign were included. The PSDQA team did not exclude any areas in these three districts from the sampling selection. Figure 1: PSDQA Phase II Coverage 2.3 SAMPLING STRATEGY The PSDQA team determined the sample size using the PPS sampling strategy. Using the PPS approach, the study included three districts—Biharamulo, Bukombe, and Ukerewe—and sampled wards, villages, and structures. The calculated sample size (sampled structures) was divided by structures found to determine the sampling interval. Upon arrival in the village, surveyors used the sampling interval to select the structures to be assessed. Of 323,350 structures found in the sampled areas, the team sampled 1,141 structures (0.35 percent) (see Table 3). Table 3: Number of Structures Found and Sampled District Structures Found Structures Sampled Biharamulo DC 112,414 380 Bukombe DC 88,379 379 Ukerewe DC 122,557 382 Total 323,350 1,141 From each district, two strata were formed. The PSDQA team selected six wards each from Biharamulo DC and Bukombe DC, and ten wards from Ukerewe DC. The team determined the sampling in each district using the PPS approach, selecting 97 villages. The sampling size depended on the number of structures found and sprayed in the November-December 2021 IRS campaign. Table 4 provides more details on the distribution of villages. Table 4: Sampled Wards and Village, Disaggregated by District District Wards/Zones Villages Actual Number Sampled Actual Number Sampled Biharamulo DC 18 6 404 27 Bukombe DC 17 6 359 33 Ukerewe DC 25 10 513 37 Total 60 22 1,276 97 2.4 RECRUITMENT AND TRAINING A team of 20 surveyors (four field supervisors and 16 enumerators) attended a one-day training before data collection. The training covered the following topics: VectorLink Project Background, Spray Objectives, PSDQA Objectives, PSDQA Methodology, Research Ethics, Confidentiality Protocols & Quality Control Assurance, Data Collection Tools (Kobo Toolbox), challenges and lessons learned from PSDQA round I, and coronavirus disease 2019 (COVID-19) protocols. The Data for Development team (Co-Principal Investigators) and the Kepler Consultant team facilitated the training. The training was also attended by representatives from the President’s Office-Regional Administration and Local Government (PO-RALG), District Malaria Focal Persons (DMFPs), and District Information Education Coordinators (DIECs), and PMI VectorLink staff. 2.5 DATA COLLECTION PROCESS Upon arrival in a sampled village, the PSDQA team contacted the local authorities for introduction, solicited cooperation during the fieldwork, and helped each surveyor determine the entrance point (reference point) as their initial reference point before verifying the number of structures following the sampling interval. Each team member randomly selected the first structure as their initial reference point. Thereafter, team members walked to the left and counted the number of structures in the given sampling interval. When the survey teams arrived at a structure to be sampled, they determined eligibility for spraying by using the pre-established structure eligibility criteria before administering survey questions in the Kobo Toolbox.6 The 6 This is a free open-source tool for mobile data collection. survey questionnaire assessed mobilization coverage, spray coverage, evidence of spraying, the number of people protected and structure characteristics of the selected structures. If an ineligible structure was found, the teams selected a replacement structure by approaching the next structure to the left of the ineligible structure. If the surveyor found no adult present to complete the survey, s/he looked for a replacement structure heading to the left. The team followed the predetermined village sampling interval to achieve the required number of structures for the sampled village. The data collectors submitted their completed questionnaires each day. 2.6 STUDY LIMITATIONS Limitations of the PSDQA II survey included the following: ● In some villages, especially in Ukerewe DC (in Kakukuru Ward in the following villages, Kitanta, Masonga, and Mibungo) and in several villages in Bukombe DC (Nansuhukulu village in Bukombe Ward, Musasa village in Runzewe Western Ward and Nyampalahala Village in Busonzo Ward) there were insufficient structures to complete the sample size in the sampled villages. Overall the PSDQA covered 98 percent of the planned sampled structures, i.e., 1123 structures out of the proposed 1141 structures. Eighteen structures were missed. This coverage was within the 95% CI. The nature of the DCs and the intervals between structures, especially in rural areas, did not allow the team to replace the missed 18 structures within the planned period of the PSDQA. ● Although the team conducted the PSDQA within two months post-spray in the targeted districts, some spray evidence had already been lost or removed, such as IRS Cards, stickers, and chalk marks, resulting in increased reliance households’/respondents’ reports to determine spray status. ● Surveyors found it difficult to verify the spray residue as the insecticide used during the 2021 spraying was odorless and colorless. It was not easy to observe and verify the IRS residue. ● Heavy rains, especially in some villages of Biharamulo DC and some villages in Ukerewe DC, impaired the survey process despite the availability of protective gear (gumboots and umbrellas). In Biharamulo DC, in three villages in the Nyabusozi ward (Kaperanono, Mbindi, and Nyabusozi), the exercise took longer to complete than planned. ● Some areas had network problems, making it difficult to record the geocoordinates fast enough, which delayed some enumerators in completing the assessment. ● Structures in some villages, especially in Biharamulo DC and Bukombe DC, are scattered. In these circumstances, motorbikes were used to assist the team complete the allocated survey . 2.7 DATA QUALITY ASSURANCE The PSDQA team trained surveyors to review their completed forms before submitting them to the server. In some instances, surveyors sat with their supervisors to ensure all the questions were answered accurately and completely. Data for Development monitored the submissions online daily to ensure all completed forms were submitted. Further, Data for Development followed up with the supervisors and data collectors if the submitted forms did not match the targeted structures in the sampled village. The Data Manager also reviewed the accuracy, consistency, and completeness of the submitted data. In some instances, Data for Development, in collaboration with Supervisors, generated maps to visualize the submitted data to ensure data integrity. That data collector adhered to the pre￾determined interval. In areas where the targeted structures were not met, the surveyors were advised to add comments which ere the Supervisors, and Data also noted for the Development team. During the fieldwork, the Data for Development team and Kepler Consultant team supervisors implemented the following quality control procedures/protocols: 1. Supervisor Accompaniments: Supervisors randomly moved into the field with the surveyors to ensure quality and adherence to the audit protocol. Any challenges encountered were shared during the evening debrief meetings to ensure standardization. 2. Back checks: During data collection, the Data for Development supervisors and Kepler team supervisors revisited some structures in some villages to re-record the structure's information and reinterview the households to ensure the field team followed all protocols and sampling procedures. 3. Fieldwork meetings: PSDQA team conducted daily meetings to review filled-in data before uploading them to the KoBo Toolbox database. The team meetings were also used to discuss the daily progress, challenges encountered, how to address challenges, and lessons learned. 4. Use of tablets during data collection: The team used a mobile-based data collection system (KoBo Toolbox) during the data collection. This approach was easier to administer, saved time, reduced human errors during data entry, and reduced the possibility of data loss. This also enabled Data for Development and Kepler team to monitor the daily submissions in real-time and make corrections while the team was still in the field. 2.8 DATA ANALYSIS After consultation with supervisors and enumerators; the PSDQA team cleaned the data rectified any errors. The team then analyzed the data using Excel and Stata software. Descriptive statistics and confidence interval (CI) levels for structures sprayed and population protected were calculated and presented in frequency tables and graphs. Using content and thematic analysis, the PSDQA team analyzed data collected through key informants, where quotes were presented to support the findings. 3.0 PSDQA RESULTS The PSDQA results are presented and discussed in the following order as indicated in the study objectives: mobilization coverage, spray coverage, IRS Card retention, the proportion of people protected, lessons learned, and recommendations for future PSDQA. 3.1 MOBILIZATION, SPRAY COVERAGE, AND IRS CARD RETENTION 3.1.1 Mobilization Coverage The overall mobilization coverage is 92.3 percent indicating most community members (1,037 of 1,123) were involved in IRS mobilization. Biharamulo DC had the highest mobilization coverage of 97.6 percent, followed by Ukerewe DC at 93.3 percent and Bukombe DC with mobilization coverage of 86 percent.Two wards had 100 percent mobilization coverage. (Nyamigongo ward in Biharamulo DC and Bukungu ward in Ukerewe DC).The lowest mobilization coverage percentages were 76.2 percent in Bwisya ward, Ukerewe DC and 79.2 percent in Lyambamgongo ward, Bukombe DC. (Table 6). 3.1.2 Information Given During Mobilization The PSDQA team gathered information on the level of information given during mobilization. Three levels were considered; inadequate where residents were told to prepare for the IRS activity and given the spray dates; basic where residents were additionally told the benefits of the IRS activity and comprehensive where residents, were also informed about the types of insects to be eliminated, side effects, and what needed to be done after spraying. The results are shown in Table 5 below. Table 5: Level of Information During Mobilization NO Level of Information Category Number of Responses Percenta ge 1 Inadequate information (preparation before spraying and spray dates given) 313 30.2% 2 Basic information (preparation before spraying, spray dates given, benefits of IRS provided) 413 39.8% 3 Comprehensive information (preparation before spraying, spray dates given, benefits of IRS provided, type of insects to be eliminated, and side effects) 311 30% Total 1,037 100% 3.1.3 Spray Coverage The overall PSDQA spray coverage is 90.7 percent (with a 95 percent CI of 92.6 and 97.2) compared with a reported IRS campaign coverage of 92.2 percent. The reported IRS coverage falls within the 95 percent CI. Table 6 below shows that Biharamulo DC had the highest spraying coverage of 95.3 percent of the three assessed areas, followed by Ukerewe DC at 91.4 percent. Bukombe DC, at 85.2 percent, had a lower spray coverage during PSDQA than that reported to PMI. The reported spray coverage in all three districts falls within the 95 percent CI of estimated PSDQA coverage. In 22 wards where villages were sampled, nine wards7 (41 percent) had higher spray coverage during PSDQA than those recorded during the IRS campaign. The other thirteen wards (59 percent) recorded lower spray coverage compared to the figures reported to PMI. The 7 Nyabusozi, Katahoka, and Nyarubungo (Biharamulo DC), and Mukituntu, Bwisya, Bukiko, Kakukuru, Kagunguli, and Bukungu (Ukerewe DC). reported data for most wards (81 percent) fell within the acceptable 95 percent CI. Four wards (Bisibo ward in Biharamulo DC, Runzewe Magharibi ward in Bukombe DC, and Murutunguru and Nduruma wards in Ukerewe DC were not within the 95 percent CI. 3.1.4 IRS Card Retention The overall IRS Card retention is 58.9 percent. IRS Card retention varied across districts and wards: Biharamulo DC had the highest IRS Card retention rate of 63.9 percent, followed by Ukerewe DC with 57 percent and finally Bukombe DC with 55.5 percent. No ward visited had an IRS Card retention rate of 100 percent. Surveyors recorded the highest IRS Card retention of 70.2 percent in Mukituntu Ward, Ukerewe DC and the lowest in Bukombe ward in Bukombe DC at 44.4 percent (see Table 6). Out of the 464 structures (41.1 percent) where IRS cards were not available, 47 percent indicated that the IRS Cards were either lost or had been misplaced, 29 percent indicated that the persons keeping the IRS Cards were not present, and 22 percent responded that they were not given IRS Cards during the exercise. The remainder were either destroyed by children, left out in the rain, could not be located during the PSDQA or there were new tenants. Table 6: Mobilization Coverage, Spray Coverage, and IRS Card Retention Location Number of Villages Sampled Mobilization Coverage (PSDQA) Spray Coverage IRS Card Retention (PSDQA) IRS Campaign Reported PSDQA Estimates 95% CI Data 95% CI Biharamulo DC 27 97.6% 94.9% 95.3% 92.6%- 97.2% Yes 63.9% Bisibo Ward 4 96.0% 97.0% 88.0% No 54.0% Kalenge Ward 5 98.6% 94.8% 90.3% No 56.9% Katahoka Ward 3 95.2% 90.4% 90.5% No 64.3% Nyabusozi Ward 5 96.9% 96.4% 100.0% Yes 69.8% Nyamigogo Ward 4 100.0% 98.8% 98.6% Yes 69.6% Nyarubungo Ward 6 98.0% 89.0% 100.0% Yes 64.7% Bukombe DC 33 86.0% 92.7% 85.2% 81.1%- 88.6% Yes 55.5% Bukombe Ward 7 87.7% 92.6% 86.4% Yes 44.4% Busonzo Ward 5 89.7% 97.2% 91.2% Yes 67.6% Katente Ward 7 84.9% 83.2% 82.2% Yes 50.7% Lyambamgongo Ward 4 79.2% 92.7% 87.5% Yes 54.2% Runzewe Magharibi Ward 5 85.5% 96.9% 78.3% No 55.1% Ushirombo Ward 5 83.9% 94.2% 87.5% Yes 64.3% Ukerewe DC 37 93.3% 90.7% 91.4% 88.1%- 92.3% Yes 57.0% Bukiko Ward 2 90.0% 89.0% 95.0% Yes 55.0% Bukungu Ward 2 100.0% 79.0% 90.9% Yes 45.5% Bwisya Ward 2 76.2% 79.0% 90.5% Yes 61.9% Location Number of Villages Sampled Mobilization Coverage (PSDQA) Spray Coverage IRS Card Retention (PSDQA) IRS Campaign Reported PSDQA Estimates 95% CI Data 95% CI Igalla Ward 5 93.2% 98.0% 93.2% Yes 66.1% Kagunguli Ward 6 98.1% 90.5% 90.6% Yes 47.2% Kakukuru Ward 4 96.6% 90.9% 91.4% Yes 53.4% Mukituntu Ward 5 97.9% 90.9% 97.9% Yes 70.2% Murutunguru Ward 4 89.2% 89.5% 83.8% No 54.1% Namagondo Ward 3 92.3% 98.8% 92.3% Yes 53.8% Nduruma Ward 4 90.0% 93.5% 87.5% No 52.5% Total 97 92.3% 92.2% 90.7% 88.8%- 92.3% Yes 58.9% 3.2 SPRAY EVIDENCE The PSDQA team assessed spray evidence in sampled structures based on four criteria: 1) presence of insecticide residue, either wall staining or smell; 2) household self-report as confirmation of spray status; 3) presence and documentation of spray status on IRS Card and 4) house markings by chalk or IRS sticker. Overall data analysis indicated that 356 (31.7 percent) of 1,123 surveyed structures had all four pieces of evidence for verification. As indicated in Table 7 below, the most available evidence was a household report with 100 percent availability, followed by house marking at 95 percent, IRS card presentation at 59 percent and insect residue present at 54 percent. Surveyors indicated difficulty to identify residue in many structures since the chemical used was colorless and odorless. In these circumstances, the surveyors relied on household testimony. Disaggregated by District Table 7: Spray Evidence, Disaggregated by District Disaggregated District Sprayed Structures Surveyed (N) IRS Card Present (%) House Marking Present (%) Insecticide Residue Present (%) Household Report (%) Biharamulo District 380 63.9% 97.5% 79.8% 100% Bukombe District 371 55.5% 89.6% 26.6% 100% Ukerewe District 372 57.0% 97.4% 52.9% 100% Total 1123 58.9% 95.0% 54.3% 100% 3.3 REASONS WHY STRUCTURES WERE NOT SPRAYED Of 1,123 structures assessed, the PSDQA found 106 unsprayed structures after excluding ineligible structures8 . PSDQA team identified five main categories of reasons why the structures were not sprayed after (see Figure 1): 8 Ineligible structures include kitchens, food stores, and toilets that are separate, standalone units from living/sleeping structures as well as buildings where people do not live/sleep, for example, churches, mosques, shops, schools (classrooms and administration block), stores, and warehouses, and animal sheds and granaries. 1. Forty-one (41) structures (38.7 percent) were not sprayed because no one was at home/houses were locked when spraying. 2. Twenty-six (26) structures were not sprayed (24.5 percent) because of sick or vulnerable populations in the house, including seven structures that had infants. 3. Five structures (4.7 percent) were not sprayed due to refusal from the households to spray their houses. 4. Three structures (2.8 percent) were not sprayed because there were community ceremonies/funerals at spraying. 5. Thirty-one (31) other structures (29.2 percent) were not sprayed due to 1) spray operators (SOPs) did not turn up at 20 structures (19 percent); 2) SOPs did not have insecticides to complete the assignment at two structures (2 percent); 3) heavy rain interrupted the spraying at two structures (2 percent); 4) four structures (4 percent) were eligible structures with several rooms but regarded as ineligible by sprayers because they had single rooms used as shops/storage rooms; 5) one structure (1 percent) was not sprayed because it was ineligible for spraying at time of spraying and two structures (2 percent) were not sprayed because SOPs did not return as promised.9 Figure 2: Reasons Structures Were Not Sprayed 3.4 POPULATION PROTECTED Overall, the PSDQA results indicate that the IRS intervention protected 90.8 percent of the population, a 95 percent CI of 89.9 and 91.6 percent. The population protected during the PSDQA is compared with the population estimated during the IRS campaign. Only at Bukombe DC were fewer people protected during the PSDQA than the reported IRS campaign results (see Table 8). Out of the total projected population of 4,494 recorded during PSDQA, 956 were children under five years of age (21 percent), and 90 were pregnant women (2 percent). 9 At one structure the insecticides ran out and the SOP promised to return after getting the insecticides. The other structure was mistakenly identified to be in another hamlet by the SOP, and the SOP promised to return but did not do so. Ceremonies, 2.8% Nobody was present, 38.7% Others, 29.2% Refusal, 4.7% Vunerable Person in the house, 24.6% Table 8: Population Protected in Target Areas, by District District IRS Campaign Results (%) PSDQA Estimate (%) 95% CI Biharamulo District 93.5% 95.1% 93.9%, 96.1% Bukombe District 92.5% 85.7% 83.9%, 87.3% Ukerewe District 87.5% 91.9% 90.5%, 93.2% Total 90.9% 90.8% 89.9%, 91.6% 3.5 STRUCTURE CHARACTERISTICS10 Overall, most of the structures surveyed were constructed with red, burnt brick (43.3 percent), mud (34.5 percent), or cement (16.6 percent). The remaining structures (3.5 percent) were constructed with other materials including concrete or wood. Red, burnt brick was the principal mode of construction in Bukombe DC (53 percent of structures) and Ukerewe DC (37.9 percent. In Biharamulo DC, structures were mostly constructed with mud (52.8 percent). Table 9: Structure Characteristics, Disaggregated by District (in percent) District Cement 11 Mud Red Brick Other12 Biharamulo DC 6.5% 52.8% 39.1% 1.6% Bukombe DC 22% 19.6% 53% 5.4% Ukerewe DC 27.7% 30.6% 37.9% 3.6% Total 18.6% 34.5% 43.3% 3.5% Corrugated iron sheets and grass were the roofing materials used in the audited areas; corrugated iron sheets were more popular in all the three districts with 91 percent of roofs being covered with corrugated iron sheets. Table 10 provides a breakdown of the materials used in roofing for the surveyed structures in the three districts assessed. Table 10: Type of Roofing Materials for Eligible Structures, by District District Corrugated Iron Sheet (%) Grass Palm/Thatch (%) Biharamulo District 87.4% 12.6% Bukombe District 93.3% 6.7% Ukerewe District 92.5% 7.5% Total 91.0% 9.0% The survey team also assessed the prevalence of open eaves in the structures visited. Eaves are the edges of the roof that overhang the face of a wall and, normally, project beyond the side of 10 The roof and the open eaves are more important characters than the type of material used for the walls. Thatch roof will harbor water longer than a tin roof and has more surface area for mosquitos to hide within the roof area. The open eaves allow the mosquitos to enter the house more easily. Those houses that are painted with a water based paint may result in the paint washing off but it does not necessarily affect the impact of the IRS program on mosquito control. 11 Includes concrete 12 Includes whitewash, water paint and oil paint among others., a building. Some are open, thus have space between the eave and the wall, while others are closed. Out of the 1,123 structures that were sampled, 139 (12 percent) had open eaves. Of the structures with open eaves, 57 (5.1 percent) were found in Ukerewe DC, 55 (4.9 percent) in Biharamulo DC and 27 (2.4 percent) and Bukombe DC. Figure 3: Prevalence of Open Eaves, Disaggregated by District 4.0 LESSON LEARNED AND RECOMMENDATION 4.1 LESSONS LEARNED ● VectorLink engaged District Leaders and Ward Executive (WEOs) in planning and implementing the IRS interventions. It was strategic for the VectorLink project to get their buy-in and support during the mobilization, IRS campaign, and PSDQA exercise. This mobilization resulted in a high spray coverage of 90.7 percent. That said, the Village Executive Officers (VEOs) were less engaged in the IRS activity, which made it difficult for some mobilizers and SOPs to implement the activity in their areas. Where mobilization was less effective, as in Ukerewe DC,, Local Government Authorities (LGAs) issued orders, including legal reinforcement, to ensure mobilization and spraying. ● Mobilization needs to emphasize the benefits of the IRS campaign as well as reducing misconceptions such as IRS increases bed bug populations or reduces male fertility. ● The PSDQA team gathered the following information from the interviewed householders: o 466 respondents (41.5 percent) indicated mobilization and spraying went well. o 829 respondents (73.8 percent) indicated the IRS assisted in eliminating mosquitoes. o 484 respondents (43.1 percent) indicated the insecticide eliminated other insects like fleas, 4.9% 2.4% 5.1% Biharamulo DC Bukombe DC Ukerewe DC Presence of Open Eaves o 225 respondents (20 percent) said the IRS education was sufficient. o 6 respondents (0.5 percent) indicated that the insecticide did not have a strong smell. ● 200 respondents (17.8 percent) indicated that mosquitoes were not eliminated even after the IRS exercise. ● 62 respondents (5.5 percent) said that mobilization was not done properly and no knowledge was passed on; ● 20 respondents (1.8 percent) indicated their houses were not sprayed. The most frequent reason cited was the SOP did not reach their homes. Also during the PSDQA, the team came across ten structures in Kidete village, (Bukombe DC), that were not sprayed. ● 37 respondents (3.3 percent) informed the PSDQA team that they found it cumbersome to move their belongings for the spraying. Some indicated their belongings were damaged by rain. ● 13 respondents (1.2 percent) believe that the sprayed insecticides increase various other organisms like fleas. ● 9 respondents (0.8 percent) indicated that the insecticide smell was too strong for them.13 ● 19 structures (1.7 percent) were excluded as they had sick/vulnerable people living in the structure. ● 8 structures (0.7 percent) were excluded as they had young children living in the structure. ● Although IRS acceptance exceeded 90 percent in the three districts, surveyors reported misconceptions related to IRS refusal. One noted case at Bukombe DC, Runzewe ward, in Musasa Village at Imalampaka hamlet, where 19 structures were not sprayed because hamlet leaders did not allow SOPs to carry out the exercise because they had false beliefs (such as the insecticide reduces male fertility, and it increases the presence of bedbugs). Some respondents in this hamlet indicated that structures were sprayed only after LGA persuasion. The interviews with seventeen SOPs from Biharamuro and Ukerewe DCs, indicated there was a high acceptance of IRS in rural areas compared ourban areas. The structures in urban areas have many belonging (i.e. T.V., beds, tables) compared to rural areaswhich are difficult to move, the households fear their belonging will be stolen and most of the structures in urban are closed during daytime when people go-to work. ● The interviewed SOPs mentioned several operational improvements that would improve the campaign. 1) The speakers mobilizers use should have improved sound; 2) Recruit mobilizers and sprayers from the villages to increase trust; 3)Sprayer personal protection equipment should fit to increase efficiency. 4) Ensure SOPs have good communication skills to convince householders to spray their structures. ● The use of mobile applications simplified the collection and reporting of data. 13 Some residents indicated that the insecticide had a smell, hence the comments about the strong smell. 4.3 RECOMMENDATIONS 1. PMI VectorLink Tanzania Project should ensure that, during the IRS exercise, the SOPs have adequate insecticide to spray the structures allocated. Team leaders should have extra insecticide to distribute to the sprayers who finish the sachets before completing their daily spraying targets. Spray teams should follow up on unsprayed structures and revisit planned and implemented. 2. Mobilization exercises are key to the implementation success. Not only should the messaging emphasize the importance of IRS, but targets should be well explained, so residents do not assume that all structures will be sprayed and all insects controlled. The project should continue to involve all village leaders and influential community members, including Ward Executive Officers [WEOs], VEOs, religious leaders, and the Local Government Authorities (LGAs) in the respective districts, recognizing that when remuneration is not provided, there may a reluctance to participate in the mobilization. Sufficient notice before the IRS campaign will also allow community members to plan and prepare for IRS intervention and ensure they will be available. The spraying exercise should be well planned so only residents of the structures reached on a particular day remove their belongings. 3. Even though even though this is the third year of the campaign, there was low voluntary acceptance of campaigns in some areas in Ukerewe DC. This should be investigated and messaging developed and disseminated to increase acceptance of the campaign in that specific context. ANNEXES ANNEX A: PSDQA SURVEY TEAM ANNEX B: SELECTED VILLAGES ANNEX C: PSDQA DATA COLLECTION TOOL ANNEX D: DATA FOR THE DEVELOPMENT DQA TOOL ANNEX E: DOCUMENTS REVIEWED ANNEX G: STRUCTURE DEFINITION DOCUMENTS