SOLUCION TB Strengthening Observed therapy Linking Up Community-based Integrated Outreach Networks for TB control Midterm Evaluation Submission Date: November 30, 2006 External Evaluation Conducted: August – October 2006 Cooperative Agreement Number: GHS-A-00-04-00013-00 October 2004 – September 2008 Project Concern International 5151 Murphy Canyon Road, Suite 320, San Diego, CA 92123 Tel: (858) 279-9690 Fax: (858) 694-0294 Contact: Blanca Lomeli blomeli@projectconcern.org or Clara Eder, ceder@projectconcern.org Project Concern International Tijuana and Mexicali, Baja California, Mexico Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 _________________________________________________________________________________________________________________________________________________________ i TABLE OF CONTENTS ACRONYMS…………………………………………………………………………………..ii A. Summary…………………………………………………………..………..………….1 B. Assessment of Progress……………………….…………………...…....……...….….14 1. Technical Approach…………………………………...……….……………..14 a. Progress by Intervention Area………………….……..…………….......…14 2. Cross-cutting Approaches……………………………….………..…………..26 a. Behavior Change and Communication ……………………….….…..……26 b. Capacity Building……………………………………...………………..…27 c. Community Mobilization………………………..…………...………….…28 d. Sustainability Strategies………………………….………………………...29 3. TB Project Indicators…………………………………………………………...29 C. Project Management………………………………………………….……..…….……32 1. Planning………………………………………………………………….....…..32 2. Staff Training…………………………………………….…....………....……..33 3. Supervision of Project Staff……………………………………..………..…….33 4. Human Resources and Staff Management…………………..….….……...……34 5. Financial Management………………………………………….….….…....…..35 6. Logistics………………………………………………….……..………....……35 7. Information Management………………………………………………...……..35 8. Technical and Administrative Support…………………………………...……..36 9. Mission Collaboration…………………………………………………....……..36 D. Stories of Success………………………………………………………….…...………36 E. Conclusions and Recommendations………………………………………...……….…38 F. Results Highlights……………………………………………….…………….….….…41 Attachments A. Action Plan/Revised Timeline of Activities FY 07 B. Evaluation Team Members C. Evaluation Assessment Methodology D. List of Persons Interviewed E. Project Data Sheet F. CD Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ ACRONYMS ACSM Advocacy, Communication and Social Mobilization AIDS Acquired Immunodeficiency Syndrome BC Baja California CBO Community-based Organization CSHGP Child Survival and Health Grants Program DIP Detailed Implementation Plan DOTS Directly Observed Therapy Short course EpiTB National TB Registry in Mexico FY Fiscal Year HIV Human Immunodeficiency Virus ISESALUD Instituto de Servicios de Salud Publica del Estado de Baja California (Public Health Service Institute for Baja California) LLR Lower Level Result M&E Monitoring and Evaluation MPH Master’s Degree of Public Health MSC Medicina Social Comunitaria (Social Community Medicine) MTE Midterm Evaluation NTP National Tuberculosis Program PAHO Pan American Health Organization PCI Project Concern International PLWHA Persons Living With HIV/AIDS PMP Performance Monitoring Plan QIVC Quality Improvement Verification Checklists RSEM Registro Semanal de Entrega de Medicamento de TB SA Substance Abuse SAT Self-administered Treatment SOLUCION TB Strengthening Observed therapy Linking Up Community-based Integrated Outreach Networks for TB control TB Tuberculosis UABC Universidad Autónoma de Baja California (Autonomous University of Baja California) USAID United States Agency for International Development USMBHC COO US Mexico Border Health Commission’s California Outreach Office WHO World Health Organization ii Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 1 A. Summary Description of SOLUCION TB Project and its Objectives In 2004, the United States Agency for International Development’s (USAID) Child Survival and Health Grants Program (CSHGP) provided PCI with US $1,495,725 for the period October 2004 through September 2008 for the SOLUCION TB project. The goal of the project is to influence national efforts in Mexico to improve TB treatment success by expanding the Directly Observed Therapy – Short Course (DOTS) model in the jurisdictions of Mexicali and Tijuana, Baja California. The project objectives include: • Expansion of the community-based health worker system to increase directly observed therapy (DOT) for TB patients. • Development of a sustainable DOTS model by increasing community participation. • Development of effective strategies for reaching marginalized, high-risk target groups (PLWHA, substance abusers). • Improvement in the use of monitoring and surveillance data for TB program management. • Improvement of provider practices by increasing TB knowledge and participation in DOTS. • Increasing DOTS components of existing local medical school curricula. • Improvement of information and communication systems for DOTS including effective documentation and sharing of tools and results. • Mobilizing political commitment for state and local TB control programs. Main Accomplishments: October 2004 through September 2006 The SOLUCION TB strategy has been able to expand the DOT workforce for jurisdictional TB control efforts and, by doing so, has demonstrated the feasibility and appropriateness of community-based activities for increasing TB treatment success. According to midterm evaluation findings, the SOLUCION TB strategy has been successful in developing motivated and knowledgeable outreach staff, involving health center teams in case management, and addressing educational needs of providers and staff. Tijuana and Mexicali have integrated DOT activities into ongoing operations, improved communication systems, and created a data system for project monitoring. Involvement of drug rehabilitation centers is a promising model for expansion of DOT care to the community for specific high-risk populations. The progress achieved in both participating cities is quite satisfactory. In particular, jurisdictional, state, and federal-level officials have been engaged and supportive of DOTS expansion; demonstrating feasibility and strength of this private-public partnership model. Main Constraints and Areas in Need of Further Attention • Absence of written policies and procedures to standardize project operations; including criteria and method of patient assignment, treatment documentation guidelines, and supervisory schedules. • Lack of progress on web-based logic map for broader communication. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 2 • Survival of long-standing practices, which interfere with acceptance of strict DOT (e.g. weekly allocation of medication supplies) and lack of analysis of these practices on outcomes. • Resistance of providers to adopt more patient-centered TB control strategies; based on limited knowledge of outcomes, concern about personal risk, and limited resources. Lack of involvement of patients in strategies to improve outcomes. • Lack of definitive plans from state and federal level decision makers to maintain and expand community-based DOT and limited progress on engagement of community partners to support and sustain DOT capacity. • Insufficient use of available information on treatment outcomes to monitor program effectiveness and to take corrective actions. Conclusions and General Recommendations Resulting from the Evaluation Conclusions • The SOLUCION TB project has increased DOT capacity in Tijuana and Mexicali. The model of specialized community-level TB outreach staff is on target to improve treatment success. DOT staff is committed and motivated. Involvement of health centers is important for success. • Sustainability will depend up on allocation of public and private resources and political commitment even beyond increased resources. This is a program that does require changes in attitudes/disposition/commitment. Government and private funding organizations have not been adequately engaged; on the other hand, the project has been successful at involving the National TB program and negotiated its expansion to an additional 12 States. Key findings and data have not been fully communicated to decision-makers. Resources for DOT could be maximized through partnerships between ISESALUD and existing community organizations. • The training and education provided within the SOLUCION TB model has been effective. Longstanding practices, issues, and attitudes that impede progress have started to be identified and acknowledged. Health center staff and private physicians have emerged as critical partners in implementing patient-centered approaches. • The co-morbidities of HIV-AIDS, substance abuse, and mental health pose a substantial risk for rising TB morbidity and mortality in Baja California and additional resources should be allocated to support these populations. • Systems to collect data have been developed and are in place, yet have not been used effectively in planning or advocacy. Outcomes should be available in a timely manner and analyzed to measure the effectiveness of the SOLUCION TB model. • Systems of communication, data collection, and supervision are in place. Refinements in several areas could promote efficiency and clarity of effort. Recommendations • There should be more flexible enrollment criteria to reflect the mix and transmission risk of patients under treatment in the community. Maintain a blend of home and clinic DOT and assess the relative effectiveness. Develop written criteria to standardize patient assignment to staff. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 3 • ISESALUD should allocate resources for DOT staff. Review options for sustaining and expanding the DOT workforce. Consider costs and benefits of having workers absorbed by ISESALUD versus partnerships with community organizations. • Increase engagement of HIV programs and drug rehabilitation centers. Identify opportunities to develop partnerships to expand access to vulnerable populations and to support DOT through existing outreach activities and new activities. • Develop strategies to strengthen health center teams. Involve the centers, and other partners, early in planning for operations that affect their clients, staff, or workload. Enhancing electronic connectivity of the health centers should be considered. • Share models and ideas with other DOTS/DOTS-Plus expansion initiatives to identify workable local solutions for sustainability and expansion. Share international standards of TB control with decision-makers to demonstrate alignment of SOLUCION TB with worldwide standards. Develop a specific strategy for engaging decision-makers and social organizations. • Expand training in co-morbidities, such as HIV, mental health, and drug abuse. Sessions for private physicians should be expanded through work with medical societies. Eliminate efforts to educate doctors-in-training. Written materials should be created to keep partners informed and expand awareness. Staff training should be streamlined and patient-focused. • Develop one specific strategy for engaging patients in community awareness and mobilization activities. Patient satisfaction assessment should be a standard element of project evaluation. • Review data at regular intervals with relevant partners to ensure realistic strategies for quality improvement are designed and implemented. Agree upon and analyze data in ways that will provide improved clarity for planning and advocacy purposes. Review data to ensure all collected information is used for well-defined purposes. • Create a policy and procedure manual, which includes forms and instructions, workflow charts, patient selection and assignment criteria, supervision standards and data management schedules. Include policies to standardize outcome analysis of patients on self-administered doses. Conclusions and Specific Recommendations and PCI’s Responses Conclusions and specific actionable recommendations based on the general recommendations presented above are grouped under five broad areas. These areas address ongoing SOLUCION TB activities, as well as suggested scale-up considerations. PCI’s responses to each set of conclusions and recommendations are included below. DOT (Directly Observed Therapy) Model Implementation Conclusion: The SOLUCION TB project has increased DOT capacity in Tijuana and Mexicali. The model of specialized community-level TB outreach staff appears on target to improve treatment success. DOT staff are committed and motivated to provide service to their patients. Two levels of DOT staff have been created; DOT workers and promotoras. Involvement of the ISESALUD health centers is important for long-term success. SOLUCION TB should have more flexible enrollment criteria to reflect the mix and transmission risk of patients under treatment in the community. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 4 Response: The project is pleased with the advances in the area of model creation and its potential for replication/adaptation to other sites. The project has succeeded in developing a specialized team of workers who share a common goal and are committed to TB treatment and control. Project staff believe that part of the success in terms of treatment control rates continuing to improve is due to the increased level of support that exists now within the ISESALUD system, as a response to having dedicated, committed and trained DOT workers available to support health centers’ efforts, either by having them provide home visit services exclusively, or by having them dedicated at the health centers participating in the project. Improved clarity regarding patient enrollment, information flow issues, and the DOT model specifics are needed and will be a focus of Years 3 and 4 of project implementation. Project core management staff will work together to further refine the DOT model that SOLUCION TB proposes, as a strategy to strengthen already existing services. Recommendation: Maintain a blend of home-based and clinic DOT. Develop methods to assess the relative effectiveness of DOT delivery setting, such as adding “location” variables to the database or conducting patient focus groups. Develop criteria to standardize assignment to home￾based vs. clinic DOT. Response: The blend of home-based and clinic-based DOT will continue. Patient selection criteria will be further defined and incorporated into the Operations Manual to be produced. The database will be revised and revamped to more clearly orient physicians to the best DOT modalities for their patients within SOLUCION TB. PCI’s database specialist will work together with State-level data management staff on this topic in particular. New variables, including location and treatment observation schedule options, will be added to the SOLUCION database. Recommendation: Formulate treatment supervision algorithms, which would provide rigorous case management and documentation, while building in flexibility to maximize patient adherence and resource use (e.g., provide strict DOT for all smear positive, pulmonary cases for the intensive phase, then risk-based determinations on strict DOT vs. weekly allocations for self￾administered treatment (SAT) in continuation phase). Response: Following this recommendation, the team will work together to identify a treatment algorithm to better inform the selection of patients to be enrolled into SOLUCION TB (other than classification as a ‘new pulmonary smear positive case; plus patients’ address and proximity to a SOLUCION TB participant health center or DOT worker); and their treatment ‘modality’ (e.g. strict DOT for all smear positive pulmonary cases and subsequent determination of three weekly vs. daily DOT (either at the clinic or home-based). In addition to this being an existing reality the project faces, identifying successful yet less intense observation options might work well with some patients, and help to avoid abandonment at the time treatment changes from intensive to maintenance. Recommendation: Develop written guidance to expand the formal reach of the model to patients with specific, high risk profiles, such as relapse patients, those with TB meningitis, or HIV co￾infected regardless of site of disease. Develop a system to ensure adherence and outcome data on these groups by risk-cohort. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 5 Response: Written guidelines to reach patients with specific high risk profiles will be completed by February 2007. Mechanisms for the tracking of adherence and outcome data will be detailed and incorporated in the Operations Manual. Analysis of data will take place during the data￾analysis management-team sessions on a bi-yearly basis. The team will identify appropriate data to be collected, analysis, report and sharing strategies on groups by high risk-cohort. Recommendation: Review the various options for maintenance of the current number of DOT staff. Include the management team and health center partners in the review. Consider costs and benefits of having DOT workers absorbed by ISESALUD versus maintaining promotoras through community-based networks. Response: ISESALUD has confirmed their commitment and intention to absorb 4 DOT health workers beginning January 2007, ahead of time according to ISESALUD’s commitment per the DIP process. At the moment, SOLUCION TB staff do not anticipate an alternative approach of additional community-based networks and health workers that would contribute to the sustainability of the project. We believe that as long as these health workers remain dedicated to TB control, the plan will work out well. Recommendation: Develop methods for reliably monitoring DOT by existing staff at rehabilitation centers and at other community-based facilities. Work with SOLUCION TB staff, management, and health centers to identify community partners with existing infrastructure to provide DOT; such as HIV-centers and schools. Response: Rehabilitation centers’ TB patients do receive services through ISESALUD and/or SOLUCION TB DOT workers. Because of the existing services and limited availability of health workers, and because of the importance of having well-trained and committed individuals carrying out DOT, the SOLUCION TB core management team believes that it is best to continue the model as it is and not add another layer of complexity that might fall outside existing supervision, training and support systems for DOT workers (from SOLUCION TB or from ISESALUD). Currently, rehabilitation and HIV centers do not have sufficient health professionals or health workers trained in TB control on staff. Having to train and provide oversight to additional health workers would pose an additional burden on ISESALUD staff. Recommendation: Engage the ISESALUD HIV programs. Identify opportunities to provide or support DOT through existing outreach activities. Response: Although the project made an effort initially to coordinate activities with HIV/AIDS providers from ISESALUD (having them participate in the DIP workshop for example), no concrete coordination/collaboration activities are taking place as part of SOLUCION TB. The project will make an effort to re-engage in dialogue and seek opportunities for coordination and collaboration in Years 3 and 4. Recommendation: Develop strategies to strengthen health center teams. Importantly, involve health centers, and other partners, early in planning for operations that affect their clients, staff, or workload. This includes patient management, staff training, and community mobilization activities. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 6 Response: The project did seek to involve and inform all levels of staff, beginning with an initial presentation to the heads of both jurisdictions; having TB jurisdiction managers participate as part of the core management team from the beginning; making presentations to health center directors; and developing an informational brochure on SOLUCION TB. In addition, all health directors were provided with printed information, the SOLUCION TB brochure, and a CD containing information on the project as well as different TB technical topics including National TB norms, International Standards and presentations from Stop TB, WHO and others. However, the project acknowledges that existing longstanding barriers and attitudes towards communication, and very busy schedules of those involved, do pose a barrier to prompt and timely communication. The project has recently produced an informational flyer on the DOT promoters’ role and the flow of information in terms of interaction with the health centers. The project will distribute these fliers at the next convenient opportunity to all relevant health center staff. The project will complete an Operations Manual that describes in detail all aspects of the project and provides more information regarding its various components. The manual will be available at all health centers for consultation. The project will continue to make an effort to more strategically recruit health center staff to participate in key aspects of project planning. For example: asking selected health center staff to review key aspects of the manual (patient management, etc.), assessing training needs when developing upcoming training schedules and participation on strategic ACSM activities. Sustainability Strategies Conclusion: Sustainability will depend up on allocation of public and private resources. Government and private funding organizations have not been strongly engaged thus far, but should be a primary focus in the next 12 months. Key findings and outcome data have not been compiled and communicated to decision-makers. The reach of resources for DOT could be maximized by partnerships between ISESALUD and existing community organizations with common missions. Response: Sustainability is a priority of the SOLUCION TB project and of PCI and ISESALUD. Finding the most appropriate balance between external resources and resulting projects which help in the short term, and institutionalized efforts through Mexican structures and systems which are needed for the long term, is a goal the project seeks to achieve. As the MTE results show, the injection of human resources to address DOT tasks, motivate health service providers, and respond to the patients’ demands, has resulted in improved treatment success rates and decreased treatment abandonment. As the project begins its 3rd year and approaches its final year in FY08, management staff are committed to finding the best mechanism to sustain results, commitment and motivation for improved TB control. Recommendation: ISESALUD should allocate resources for DOT staff. Different models for hiring and managing the provision of DOT should be reviewed and discussed with ISESALUD administration. Factors including skills required, flexibility, cost, oversight, and community support should be considered. Response: ISESALUD management has confirmed their commitment to ‘absorb’ four of the DOT workers currently participating in the SOLUCION TB project. This is scheduled to take place in January 2007. The MOU signed between ISESALUD and PCI, following the DIP, states that 2 health workers were to be absorbed in FY07, and 4 more in FY08. Although moving Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 7 forward the absorption of 2 of the scheduled health workers is an indication of commitment, the project will seek to mutually agree upon a schedule for the incorporation of the remaining 2 workers (of a total of 6). ISESALUD state and jurisdiction staff will select these four candidates out of the existing pool of DOT workers. Based on interest and performance, two DOT workers will be hired in Tijuana and two in Mexicali. These highly trained and committed DOT workers will continue to be dedicated to TB management and supporting ISESALUD’s TB control program, under ISESALUD’s supervision. Recommendation: Identify and use existing materials that clarify the level of international commitment to DOT and sound TB control practices. Materials, such as the International Standards for TB Control and the Patient Bill of Rights, should be used to review practices and demonstrate to decision-makers the alignment of SOLUCION TB to worldwide standards. Response: Project staff will address this issue and produce/adapt/translate the above mentioned materials by January 2007. The Patient Bill of Rights information is included in the soon to be produced ‘Promotora Manual.’ It will also be included in an upcoming issue of the Newsletter, and also in the Operations Manual to be produced soon. Informational materials currently distributed to physicians (ISESALUD and private) in different training opportunities will continue to include information from PAHO, WHO, STOP TB’s ‘the human face of TB,’ and other relevant information. Recommendation: Share experiences and information across USAID funded projects and other avenues for international networking. Seek successful models in other locales to identify workable local solutions to sustainability and expansion. Sharing models and ideas with other DOTS/DOTS-Plus expansion initiatives will be critical for scale-up in Baja California and other Mexican states. Response: PCI project staff are part of the CORE group’s TB listserve and access and exchange information with other USAID funded projects on relevant TB topics. PCI staff share selected information via email with all members of the SOLUCION TB management team. PCI is very open to participating in additional exchange and networking opportunities. Recommendation: Develop a specific, written strategy for engaging decision-makers and social organizations. Response: As the project begins its 3rd year and continues on to its 4th and final year, the ‘Voices and Images’ (Photovoice) project will begin to be implemented (January 2007). Technical assistance and training will be provided by the USMBHA, as part of its successful ‘Voices and Images’ project currently implemented in other areas of Mexico, the US/Mexico Border and other countries. The project includes the development of a concrete strategy for engaging decision-makers and social organizations. The project will produce and make available relevant materials and information that clearly and effectively tell the story about the current situation of TB in this region and in the Tijuana and Mexicali jurisdictions. Project staff will make sure that an advocacy agenda is drafted along the development and implementation of the ‘Voices and Images’ component of the SOLUCION TB project. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 8 Training and Education of Staff, Health Professionals and Community Health Workers Conclusion: The training and education provided within the SOLUCION TB model has been effective. Long standing practices, issues, and attitudes that impede progress have started to be identified and acknowledged. Health center staff and private physicians have emerged as interested and critical partners in supporting a comprehensive, patient-centered approach to TB control, which embraces DOT as a core strategy. The co-morbidities of HIV-AIDS, substance abuse, and mental health pose a substantial risk for rising TB morbidity and mortality in Baja California and efforts must be strengthened to address these populations. Response: A strong education and training program that seeks to expand up on the current system, and has a patient-centered, case management approach, has been the priority for SOLUCION TB. The project has trained a number of health workers, including health promoters, public and private physicians as well as medical students. Training activities reflect partners’ strengths, as ISESALUD has concentrated on technical training such as TB control and infection control issues, and PCI has led the social, cultural and communications-based training, including inter-personal communication, gender, stigma-reduction, prevention and appreciative inquiry￾oriented trainings. Recommendation: SOLUCION TB staff should have targeted training in conditions that complicate the management of TB, such as HIV, mental health, and drug abuse. The training should provide clinical information, but also be used as networking opportunities between individuals and agencies engaged with these populations. Response: Training efforts have resulted in a well trained group of DOT health workers interacting with clients, as well as with health center staff (mainly nurses). These trained groups of people have clarity of purpose and are concentrating on treatment completion efforts. On a smaller scale, training opportunities for private physicians were offered in Tijuana and Mexicali. Patients have mentioned that some of their first medical interactions were with private physicians without good results in general (in terms of prompt diagnosis). Additional training for SOLUCION TB and DOT staff will include infection control and HIV/AIDS and substance abuse interactions with TB, both in terms of treatment for patients as well as service providers’ own protection. Recommendation: Private medical providers are a key audience for education about TB treatment and management principles. Educational sessions for private physicians should be expanded through work with medical societies including the pulmonary, infectious disease, and pediatrics. Education for pharmacists should be initiated using similar presentations and training venues as for medical providers. Response: Physicians have expressed their gratitude for having these training opportunities made available, and seem to have found them useful and informative. The project will continue to implement these trainings as planned. Training opportunities do represent an opportunity for awareness raising and increased sensitization regarding the topics of TB control and existing needs in the larger community. Co-infection issues and TB and substance abuse will also be addressed in upcoming training sessions for private physicians, which will also be offered to specialist societies, including pediatrics, pulmonary and infectious disease specialists. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 9 SOLUCION TB will seek to identify the most appropriate training mechanisms/providers for these specialty topics. Recommendation: A newsletter should be created to keep partners informed and expand awareness. A written plan for audience, content, and distribution should be prepared. Response: The first edition of a newsletter is in the process of being finalized. The target audience includes health center staff and DOT health workers. This was a need identified during the proposal preparation and DIP workshop. The newsletter will also be shared at private physician trainings and other training opportunities. It will be produced quarterly. The first edition will describe the project in general, the DOT strategy, the importance of treatment adherence, and the green-light committee. It will also include a success story from two of the DOT promotoras and the official Mexican TB treatment schedule. PCI’s SOLUCION TB staff will share content plans of future editions with ISESALUD partners and seek their participation for upcoming editions. Recommendation: Longstanding attitudes and concerns, which impact on the provision of patient-centered care, should be the focus of specific training efforts for providers, community agencies, decision-makers and staff. Topics should include infection control, stigma, and community mobilization. Response: The first stigma/adherence trainings in Tijuana and Mexicali were well received and began to create awareness about the topic. Approximately 100 individuals attended 4 different training sessions. The audience was composed of different health center workers (nurses, social workers, physicians and administrative staff). Most of the staff was covered, but it is evident that the need for additional training continues, emphasizing the topic of infection control. Misinformation regarding infection and prevention techniques was also identified. The project will therefore plan additional trainings emphasizing infection control and prevention techniques. Recommendation: Reduce the number of centralized staff meetings and increase health center￾based meetings. Introduce practices to identify and track issues needing follow-up to provide additional information for planning and resource allocation. Response: Centralized staff meetings only take place for core-management team meetings and interactions. Quarterly ‘quality circles’ take place at the health center level and include participation from jurisdiction-level and core-management team staff. Quarterly quality circles include identification of successes as well as opportunities and challenges. Follow-up is provided as appropriate. Project staff will include the description of this process and its follow-up activities in the upcoming Operations Manual. Recommendation: Develop at least one specific, written strategy for engaging patients in community awareness and mobilization activities. Response: The project has signed an agreement with the USMBHA (US-Mexico Border Health Association) for technical assistance to be provided to SOLUCION TB in the incorporation and implementation of the ‘Photo-Voice’ project. This project (‘Voices and Images’) is an ACSM methodology that gives patients a voice and empowers them to tell their story to decision makers and other stakeholders. The first training will take place in January 2007. An advisory group will Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 10 be formed and will play a key role advancing the message about TB control needs and its realities. Recommendation: Eliminate efforts to educate doctors-in-training. While a worthwhile endeavor, other SOLUCION TB activities are of higher immediate priority. Response: The project will concentrate on the above-mentioned training efforts and put aside the training of physicians-in-training for the moment. The project acknowledges that TB control and DOT are already part of the existing curricula in local universities. Data Management and Utilization Conclusion: Systems to collect data have been developed and are in place. However, data has not been used effectively in planning or advocacy. Cohort information has not been generated in a timely manner, nor has this information been compared to non-SOLUCION TB cohorts. A more efficient data collection, analysis, and feedback workflow is needed. This core element of the project must be addressed early in the remaining two years. Response: SOLUCION TB staff acknowledge that the timeliness and adequate use of information is an area that requires improvement within the project. Specific MTE recommendations will be addressed as follows. Recommendation: Six month cohort analyses need to integrate key EpiTB variables, risk factor variables for HIV and substance abuse, and SOLUCION TB adherence indicators. Develop a written plan to set the schedule, format, and parameters of the analyses. Produce twice yearly reports based on the agreed upon elements. Response: Staff has agreed that the above-mentioned information needs to be mutually agreed upon by the management team. The PCI’s database expert will visit the ISESALUD TB state office and dialogue with EpiTB and other data management staff to make recommendations to improve the existing database. The core management team will make this a priority of its next management meeting. The team will agree upon the following: type of data (indicators to be shared), type of reports to be produced and shared, frequency of the reports, and sharing mechanisms. The appropriateness of the SOLUCION TB web page as a mechanism for prompt data sharing will be explored and decisions will be made (including appropriateness of the database being web-based). Dr. Alberto Delgado (Director of Prevention Services at ISESALUD’s state office) has agreed that the ‘institutionalization’ of this process within ISESALUD is a priority for the project. The project will clearly identify the person responsible for report producing and sharing, within the existing management team. Recommendation: Develop a standard schedule for face to face reviews of outcome and process data. Data must be made available to management and line staff at regular intervals to gauge performance and make meaningful improvement. Response: The project will utilize every other management team meeting (bi-yearly) for data￾analysis and decision making. The team will identify appropriate data to be shared with ISESALUD health center staff and DOT workers currently participating in the project, to increase their awareness about accomplishments to date, and to identify future challenges. This Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 11 will serve as a mechanism to reinforce and motivate their continued participation, as well as to improve TB management in general. Key data, mutually agreed upon with ISESALUD, presented in a meaningful and appropriate manner, will also serve to educate and raise awareness within selected ACSM audiences, for an expanded advocacy portfolio in Years 3 and 4. Recommendation: Involve all relevant partners in data reviews to ensure realistic strategies for quality improvement are designed and implemented. Health center staff, collaborating community agencies, and field DOT staff should be involved in decision-making. Response: Once the most appropriate and relevant data is identified at the next management meeting, SOLUCION TB staff will incorporate this information a) within the SOLUCION TB newsletters, b) at promotoras’ ‘Encuentros’, and c) through the quarterly quality improvement circles. As new partners and other external audiences are incorporated and contacted (CBOs, social service groups, decision-makers etc.), data will be shared, as previously agreed upon by ISESALUD and PCI staff working in SOLUCION TB. We believe that it is crucial for health center staff treating TB patients and for promotoras to understand the impact of their TB management activities. In the past, the project has seen that a lack of awareness about the challenges that TB poses, a lack of understanding about the implications of not strengthening these efforts, and/or a lack of interest on the part of service providers, are all issues that might directly and indirectly affect patients’ treatment adherence. Recommendation: Agree upon and analyze data in ways that will provide improved clarity for planning purposes, such as city-level outcomes, clinic versus home-based DOT, outcomes by co￾morbid conditions. Response: All the above-mentioned purposes will be discussed during the data-analysis management-team sessions on a bi-yearly basis. Recommendation: Compare outcomes of SOLUCION TB patients with outcomes of comparable non-SOLUCION TB patients. This type of analysis is important to communicate success to policy-makers and potential funders. Response: Comparison between SOLUCION TB and non-SOLUCION TB treatment completion, treatment success, and abandonment rates will be carried out at data analysis meetings undertaken by the management team. Recommendation: Review the current SOLUCION TB database and data collection system to ensure key variables are captured (e.g. drug abuse) with standard definitions. Review data collection practices to ensure that the information being recorded is used for a specific and well￾defined purpose. Response: This will be part of the first decisions jointly made by the ISESALUD and the SOLUCION TB management team at the next team meeting. Since only limited information has been shared with the management team to date, there is currently not enough clarity as to the usefulness of the information produced by the database. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 12 Recommendation: Collect accurate data on observed versus SAT. Openly address the practice of providing self-administered doses by developing policies to standardize selection and approval of patients for SAT doses, and by developing a method to collect that information. Response: This was an evaluation finding that was not part of the original SOLUCION TB plan. Changing attitudes and practices that have been in existence for a long time, is a challenging and difficult task, even when resources are provided for this purpose (daily observation/DOT). The team is carefully considering these elements: the existing practice of weekly SAT and/or three weekly observations for patients in the intensive phase (even within the SOLUCION TB project); the increased likelihood of treatment abandonment for patients changing from intensive to maintenance phase (wrongly believing they are cured and have completed treatment); and sustainability challenges as the project approaches its 4th and final year. The management team will carefully review its protocols to include the alternative of both daily DOT and three weekly DOT for patients, following strict criteria yet to be developed. While the project is hesitant to include a third option of weekly DOT/majority SAT, it will likely need to determine the viability and validity of this option within the existing context. As with the other two options, strict criteria, yet to be developed, will need to be followed. Once a final decision is reached, information/reporting tools will be adapted to reflect these various approaches to DOT. The project will closely monitor these approaches and link them to the database and appropriate reporting mechanisms to ensure prompt and adequate analysis and sharing of outcomes and results. Operational Systems Conclusion: Systems of communication, data collection, and supervision are in place. Refinements in several areas could promote efficiency and clarity of effort. Responses: Ensuring good, smooth operations is critical to the implementation and replication of the SOLUCION TB model. The project recognizes that clarity in some areas is still needed, particularly when it refers to the interactions between different areas and the communication and expectations of staff. As much as the management team has a greater understanding of the project, meeting the challenges of communicating and sharing that understanding can only be done with the appropriate tools, mechanisms and processes. Clarity of key issues such as patient enrollment, DOT definitions and others is still a need that the project must address. A ‘user friendly’ Operations Manual that is distributed and utilized appropriately will help ensure proper implementation and will promote and ensure the integrity of the SOLUCION TB model. Recommendation: Create a comprehensive policies and procedures manual. This should include project forms, definitions and instructions, workflow charts, patient selection and assignment criteria, and data management policies. Response: The project will designate a team leader to carry out this process while obtaining input from all management staff. Opportunities for face-to face meetings will be utilized for reviewing drafts and obtaining feedback from management staff. Selected health center staff and DOT promoters will also be incorporated into the review/planning process. All revised and existing tools, including, information flow-charts, will be incorporated into the manual. The revised patient enrollment criteria as well as recommendations for daily DOT and SAT for the Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 13 three weekly approach will also be incorporated. There is a simple promotora manual that was recently completed by staff, but has not been printed, which will also be revised to incorporate key formats/information pieces and reflected in the finalized policies and procedures (operations) manual. Recommendation: Develop the web-based logic map in accordance with the DIP. Response: Following previous web-based models, PCI has begun development of the web-page for SOLUCION TB. Formats, reports, presentations, technical information and links will be available over the internet. There will be a password protected section exclusively for SOLUCION TB staff, and a mechanism to update and upload information and reports, as appropriate. The development of a public section has been initiated and will be made available to promote education about TB. Once a more finalized version is available, the management team will have another opportunity to provide further input and recommendations. Recommendation: Develop a real time monitoring system to ensure DOT staff is being given patient assignments that maintain minimum caseloads. Response: This is a challenge for the project. Manual analysis of information is not conducive to the timeliest responses regarding caseload. Recent transitions from home-based to clinic-based assignments for some of the community DOT promotoras in Tijuana has resulted in early lessons being learned in terms of timeliness and appropriateness of assignment. A small grant approved last fiscal year by the USMBHC’s COO (US Mexico Border Health Commission’s California Outreach Office) will be used to produce a computer-based mapping system that will assist project staff in SOLUCION TB, and ISESALUD in general, in mapping the location of both the needs and the resources available. The system will identify all patients’ location, as well as that of the health centers and the DOT promotoras, regardless of who is funding the services. The jurisdiction manager will then be able to identify gaps and potential overlaps in service provision, ensuring maximum optimization of limited resources. Recommendation: Joint supervision of DOT staff should continue to involve health center, PCI and ISESALUD supervisors to maximize both inclusion and oversight. Supervision schedules and methods should be specified in the manual. Response: Project staff agree that this is the best possible mechanism. This procedure and its purpose will be clearly specified in the upcoming Operations Manual. Recommendation: Patient satisfaction assessment should be a standard element of evaluation and should be done confidentially. This practice can inform management about direct concerns of patients that may be missed otherwise. Response: Patient satisfaction studies on adherence barriers and facilitators have been used in the past within SOLUCION TB. For example, results of such studies have been shared at the recent stigma trainings for health clinic staff. The project will make an effort to incorporate patient satisfaction as a regular evaluation/quality assurance activity in Years 3 and 4. Recommendation: Responsibility for ancillary tasks by DOT staff, such as transport of sputum specimens to the labs, should be reviewed. Limits on such assignments or creative solutions to Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 14 accomplish necessary tasks should be sought. Enhancing electronic connectivity of the health centers should be considered (e.g. fax, computer). Response: These activities are not part of the DOT worker and DOT promotora position descriptions. It does occur, however, when DOT promotoras and workers identify such ancillary tasks as a strong need and in support of their patients. The project will work with the DOT workers/promotoras and their supervisors to revise this practice and find creative alternatives, including those suggested above. Recommendation: The purpose and design of the RSEM form should be reviewed. If retained, the form should be redesigned to ensure patient confidentiality. Response: As the monitoring indicators and the database are revised, the weekly reporting format will also be revised based on a reassessment of its usefulness and purpose. If retained, it will be modified to ensure confidentiality. PCI’s Action Plan and Revised Timeline for FY 07 can be found in Attachment A. B. Assessment of the Progress Made Toward Achievement of Project Objectives 1. Technical Approach The goal of the SOLUCION TB project is to support DOTS-strategy expansion by implementing a range of activities to increase TB treatment success rates. The primary focus is to influence national TB efforts by expanding the DOTS model using community-based health workers. The project is operating in the jurisdictions of Mexicali and Tijuana, Baja California. This is a 100 percent TB project. The general program strategy seeks to build in sustainability by close collaboration with jurisdictional and national level stakeholders, as well as community groups involved in community-based care. Strategies focus on expansion of observed therapy success through training, capacity-building, patient involvement, and community partnership development at the local level. Progress by Intervention Area Intermediate Result 1: The SOLUCION TB (Strengthening Observed-therapy Linking Up Community-based Integrated Outreach Networks) model implemented in the departments of Mexicali and Tijuana, Baja California The SOLUCION TB model is based on strengthening the scope and number of community-linked workers to provide supportive observed therapy to TB patients. While Baja California has a variety of health outreach projects in communities, these have not previously been focused on the specific challenges presented by the chronicity, treatment requirements, and marginalizing quality of TB disease. This result area is the core of the project, taking the majority of time and resources of the project thus far. Focused efforts have been undertaken as described under specific lower level result (LLR) areas: Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 15 LLR 1.1. Community-based promotores system to increase direct observation of treatment (DOT) for TB patients improved and expanded Development of Community DOTS Providers SOLUCION TB has increased the number of community workers providing DOT in both project cities. There are currently 22 additional DOT community workers in Tijuana and 12 in Mexicali. Of the 22 in Tijuana, 19 are DOT promotores and three are DOT workers. In Mexicali, the split is eight and four, respectively. The total amount of workers (including DOT community workers and DOT workers) supported by the SOLUCION TB Project is thirty-nine (or 38.5). At the State level, 1.5 additional administrative positions are paid by the SOLUCION TB project. These workers take care of data input and data-base management. Administrative positions resulted from the increased workload the project posed onto the state TB manager. In Tijuana, there is one administrative/data-input position recently hired to carry out added administratively workload (paid out of the unused lab-technician position). Two additional positions are at the jurisdiction level to assist the TB manager with supervision and administrative tasks. These administrative positions do not get assigned patients for DOT. The project goal is for each DOT staff to have a minimum of five patients. The distinction between these two types of staff is important to understand in terms of project operations and mechanisms for sustainability. DOT promotores receive a monthly stipend of up to $3600 pesos (~$360 dollars). Half of the amount is a base stipend, while the other half varies depending on patient load. If the promotore does not have at least five new pulmonary cases assigned in six months, the variable stipend can be reduced. DOT workers receive a flat monthly salary of $4600 pesos ($460) and health benefits, which does not fluctuate based on patient load. This makes the DOTS workers comparable, in compensation, to staff already doing DOT work within ISESALUD, while the promotores are more akin to incentivized volunteers. In Mexicali, both types of staff are assigned to health centers where five or more patients per staff have been achievable. In Tijuana, DOT workers have been assigned to health centers, while promotores were frequently given community assignments linked to geography versus health center. As a result, caseloads of some promotores were less then five new pulmonary cases per semester. Personnel were recruited at the outset of the project, and decisions about which positions they assumed (promotore vs. DOT worker) was jointly made by PCI, state, and jurisdictional partners. The staff is predominantly female (31 female, 3 male). In general, DOT workers have a higher level of education and may be credentialed healthcare professionals (e.g. nurses, social workers). The project plan anticipated that ISESALUD would absorb DOT workers as operations proceed; 2 in year three, and 4 in the final year. From the compensation they receive, DOT staff pay for transportation to visit patients, to deliver sputum specimens to central labs, and to retrieve specimen results. In addition, they pay for incentives, such as food, for their patients. Providing incentives for patients is not part of the DOT workers’ and promotoras’ position description. Many promotores feel that the compensation received is insufficient to cover the expenses they incur on behalf of their patients. In addition, stipend amounts to promotores in Tijuana have recently been reduced for those having less than five patients: a 50% reduction in the variable portion of their stipend if their patient census is less than three. Employment and collaboration (in the case of promotoras) Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 16 contracts did outline how stipends would be paid and that transportation expenses would be paid from the stipend. However, many staff still believe that the actual costs of motivating and supporting a patient is not realized, and that having fewer patients does not always mean that their costs are proportionally reduced. The DOTS providers received core training at the outset of their employment and continue to receive periodic formal and informal group trainings. A recent session addressed the stigma of tuberculosis, which was particularly well reviewed in Mexicali. This training focused on an area of TB management that was seen as critical in achieving treatment success, yet had limited attention in the past. Assignment of Patients The assignment of patients to the DOT providers has been done differently between Tijuana and Mexicali, although both cities prioritize newly diagnosed, pulmonary smear positive cases who live in specific geographical areas, as noted in the DIP. A patient’s risk for default was not mentioned as a reason for assignment to SOLUCION TB. In Mexicali, 11 of the 25 health centers have assigned DOT staff. All TB patients that attend these centers are placed on DOT with the health center worker, regardless of site or category of disease (new, relapse, etc). However, only patients who are new pulmonary, smear positive cases are reported as SOLUCION TB patients. The majority of patients come to the health center to receive their observed doses. (The intensive phase of treatment is six days per week. Monday through Friday doses are taken via DOT). If patients fail to attend the clinic, the health worker visits them at their home within two-three days. Barriers to attendance are addressed, as possible. Some patients may be provided DOT in the home if they are chronically unable to come to the center. Common reasons for home DOT included physical inability to travel, lack of transport, and drug abuse. Some centers have two shifts of DOT staff to provide access for patients with work conflicts; with hours as long as 7am to 7pm. In Tijuana, initially only three health centers (of 37 total) had assigned DOT staff and the emphasis was on home-based DOT. Promotoras were assigned patients within a limited geographic area, usually based on the area where the promotora lived and/or had familiarity with the neighborhood. As noted, this system of patient allocation is being reorganized due to low numbers of eligible patients assigned to some of the promotoras. Promotores are being assigned to an additional five health centers and some promotores are covering a wider or different geographic area. The assignment of cases within health centers is largely made by health center administrators. Assignment of cases to promotores who are not based in health centers is made by Tijuana’s TB Control Director. Medication is received from the State, and is generally supplied as DOT-BAL (intensive), a four￾drug combination for two months and DOT-BAL (continuation), a two-drug formulation for four months. Recently, interruption in the supply of the intensive phase DOT-BAL has required substitution with Rifater and Ethambutol. Health care staff had differing opinions on the desirability of clinic versus home-based DOT. Some felt the emphasis on clinic DOT was a barrier to full adherence for many patients, while others felt it was an important method to ensure patient responsibility for their own recovery. In Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 17 addition, there was some lack of clarity as to whether patients who were other than new, smear positive pulmonary cases were eligible for formal SOLUCION TB enrollment. Workers understood that other classifications of patients (e.g. extrapulmonary, relapse) could be provided DOT if capacity permitted, but not all reported them on the SOLUCION TB adherence forms. Some pointed out that there was little difference in the potential consequences for the patient or the community between certain classifications; for example, a smear positive case that transferred from another jurisdiction was not officially a “new” case, but posed the same treatment issues. Written criteria for enrollment, prioritization, and documentation procedures should be developed. Ensuring SOLUCION TB reports for all patients provided with DOT by project staff will provide a fuller picture of workload and areas of success. For staff, it would reinforce standard practices and acknowledge the importance of their work with a range of TB cases. In fact, concentrating on TB treatment among the new smear positive, is only one element of a comprehensive approach to limit morbidity and one that has been acknowledged as inadequate to control TB. Consideration should be given to submitting SOLUCION TB reports for all patients provided with DOT by project staff. This would provide a fuller picture of workload and areas of success. For staff, it would reinforce standard practices and acknowledge the importance of their work with a range of TB cases. In fact, concentrating on TB treatment among the new smear positive, is only one element of a comprehensive approach to limit morbidity and one that has been acknowledged as inadequate to control TB. A change in approach should be considered to focus on a broader range of patients, especially during the intensive phase. Recruitment and hiring of future DOT staff should be carried out with input from medical staff of the health centers with which they will interact. When assignments in duties overlap substantially, as with DOT workers and promotores, there should be clarity as to the requirements for selection. Conditions of compensation and expectations for out-of-pocket expenses should be identified, in writing, prior to hiring, and continue to be clarified when misconceptions persist. In this project, promotores signed an acknowledgement about stipend variability, but adjustments were not made until the end of the second year. This, understandably, allowed promotores to develop customary practices and incur customary expenses on behalf of their patients. Therefore, the change in compensation was unexpected and seen as unfair by some project staff. Compensation and patient assignment practices to achieve equity and fairness for staff are critical. The number and location of assigned patients is not in the control of the promotores. In some centers, promotores have been assigned patients who live at great distance from the health center and from each other. In some centers, this occurs because there is concern about having a DOT person from an outside agency in their unit. In others, it is based on true patient need. In either case, promotores can be burdened with extreme transportation costs and be limited in the number of patients they can reach in a day. A standard method of patient assignment needs to be clarified, documented and communicated to all partners. There should be a process to reimburse for unavoidable, but above-standard expenses incurred by DOT staff, as well as a mechanism for rapid review of circumstances when caseload goals are not met. These steps will help to build and maintain cohesion and trust between all parties. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 18 Number of Patients Enrollment of patients began in April 2005. A total of 197 patients were enrolled and had outcomes captured in the SOLUCION TB database during the year. Although SOLUCION TB community staff provided DOT for additional TB patients, only those eligible under the original project criteria were enrolled as official SOLUCION TB patients. These 197 enrollees represent 23% of the 873 cases reported in the State of Baja California for 2005. As of October 30, 2006, only 10 patients have been entered into the SOLUCION TB database for the 2006 calendar year. This represents a backlog in data entry, not a decrease in enrollment. Treatment Outcomes The project goal is to increase treatment success of new, pulmonary, smear positive cases from the baseline 58.4% to 85%. Treatment success is defined, for purposes of the project, as Number cured + number completed/Number starting treatment. The January-June 2005 semester represents the first cohort of SOLUCION TB patients. Data for all enrolled patients are entered at the State level into the SOLUCION TB database, as well as the national EpiTB database. For patients who started treatment in the initial SOLUCION TB cohort (January-June 2005), there was an 82% success rate. The number of enrolled patients, who have been entered into the SOLUCION TB database, by semester, is noted on Table 1. Table 1 Semester Number Enrolled Treatment Success Jan-June 2005 100 82% (of 97 with final outcomes) July-Dec 2005 97 74% (of 60 with final outcomes) Jan-June 2006 10* Available Oct ’07 July-Dec 2006 0* Available April ‘08 *As entered in database through Oct.30-06 Outcome data are disaggregated for each city on Table 2, revealing higher success rates for Mexicali. These results should be cross-matched with adherence data for the patients represented by the cohorts, as well as comparable non-SOLUCION patients. In Tijuana, outcomes should be reviewed for development of strategies for improvement. Causes of death, destinations of patients that moved, and resistance likelihood for patients that failed should be ascertained. The extremely high success rate in Mexicali may partially be attributed to weekly allocations of medication for some patients. This strategy may be valuable for specific patients, and selection criteria should be reviewed and considered for replication or modification. Consideration should be given to following longer term outcomes (2 years disease free, post-treatment) as an indicator of treatment success. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 19 Table 2 Semester Number Enrolled Cured # (%) Completed Failed Died Abandoned Moved Continues Jan-June 05 Mexicali 42 42 (100%) 0 0 0 0 0 0 Jan-June 05 Tijuana 58 40 0 2 1 (2%) 10 (69%) (3%) (17%) 2 (3%) 3 (5%) July-Dec 05 Mexicali 46 45 (98%) 0 0 0 The SOLUCION TB database also categorizes patient adherence in terms of the number of observed doses that are taken relative to the overall treatment length for that phase of therapy. The intensive phase (10 weeks) and the continuation phase (15 weeks) are evaluated separately. The level of adherence is divided into 4 categories: Excellent = 100% adherence (All required doses taken within the standard number of weeks +/- 2 weeks); Satisfactory = 90-99% adherence (All required doses taken within the standard number of weeks +/- 3 weeks), Adequate= 80-89% adherence (All required doses taken within the standard number of weeks +/- 4 weeks), and Precarious <80% adherence (Required doses have been taken in a timeframe > 4 weeks past scheduled treatment length). Also collected in the SOLUCION TB database are some patients receiving DOT from project staff, but who do not fit eligibility criteria; recorded as “Special Cases”. 1 (2%) 0 0 July-Dec 05 Tijuana 51 27 0 1 (2%) 2 7 2 12 (53%) (4%) (14%) (4%) (23%) For the initial cohort of 100 patients (Jan-June 2005), adherence results for 46 were received during the evaluation. 86% of eligible patients completed scheduled intensive phase doses within 4 weeks of the norm, and 91% completed continuation phase doses with 4 weeks of the goal. Special cases comprised 25% and 28% of the cases in each phase, respectively (Table 3). Table 3 Level of Adherence Intensive Phase Continuation Phase Number Percent Number Percent Excellent 100% 24 52% 22 50% Satisfactory 90-99% 11 24% 12 27% Adequate 80-89% 5 11% 6 14% Precarious <80% 6 13% 4 9% Total SOLUCION eligible 46 44 Special Cases 15 25% 17 28% Daily documentation of doses taken or missed is kept, for every patient, on the standardized national form known as the “tarjeton” and additionally, for SOLUCION TB patients, on the form entitled “Registro Semanal de Entrega de Medicamento de TB (RSEM)”. The RSEM form is a method to send timely adherence information to supervisors and for data entry in Mexicali. However, neither the RSEM, nor the tarjeton are structured to capture the common practice of providing weekly (or longer) supplies of medication to patients. The covert, but apparently widespread nature of this practice is a major obstacle to understanding the impact on DOT in this project. A primary recommendation of this evaluation is to develop opportunities for open discussion and systems of data collection that acknowledge and address this situation. It was Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 20 identified that staff might provide different levels of DOT for different patients, sometimes requiring daily attendance for SOLUCION TB patients, but allowing weekly attendance by non￾SOLUCION TB patients. Practices seem to vary widely and occurred for different reasons (e.g. limited resources, lack of belief in daily DOT). Ultimately, weekly allocation of medication may be reasonably incorporated into treatment designs for selected patients, if appropriate evaluation systems are in place. Medical Teams In the health centers where DOT personnel are assigned, key relationships exist between the medical staff and the DOT workers. In the centers visited, the relationships appeared supportive. Medical staff, physicians and nurses, appreciated the support of the outreach staff in providing special focus on treatment adherence for TB patients. In all centers, there was mention of the positive effect that the DOT staff made between the rates of success of their patients before and after the SOLUCION TB project. DOT staff at the centers relied on back up during times of absence from other health center staff. A supportive attitude of the medical director was noted as important to the overall quality of patient care. The DOT personnel seemed well integrated into the operations of the centers. In some centers, DOT staff is being assigned “non-SOLUCION TB” tasks such as administrative duties or delivery of sputum specimens. This may be a positive indicator of acceptance of the DOT worker into the health center team, but is also an indicator of staffing shortages mentioned by many project participants. Mention was also made of the importance of including health centers, and other key project partners, early in project planning. This step was not built into initial SOLUCION TB planning and likely caused misinformation and confusion to occur as the design was rolled out. Expansion or re-design decisions should include existing and planned partners from the outset. DOT staff, in both cities, is clearly enthusiastic about the project. In particular, they displayed remarkable caring and ownership for the health of their assigned patients. Most shared stories about the needs of their patients and how they sought to assist them; buying food, holiday baskets, or small necessities. Many, especially in Tijuana, felt that home-based DOT was easier for their patients. Particular mention was made, in both cities and by staff and patients alike, about barriers posed in giving sputum samples. Patients noted logistical problems in getting to centers to provide samples or confusion about how long the specimens could be kept in their homes before delivery. DOT staff was generally responsible for delivery of specimens to the laboratory, but could often share this task with other health center professionals. A repeated concern from staff is the lack of separate space within the health centers for their TB patients. Reasons given included confidentiality, privacy for giving injectables, and control of transmission. On the other hand, there was substantial progress noted in providing timely service to patients that came to the center. The previous long waits to receive a daily DOT dose, which had discouraged patients from continuing therapy, were minimized by having specific and well￾trained staff attends to TB patients. In Tijuana, the reorganization of the health workers had created some discontent. There are concerns about being assigned to unfamiliar geographic areas, where they did not know the neighborhood, and having their stipends reduced at the same time. Mexicali workers were Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 21 generally content with operations and supported the clinic-based DOT. However, the ability to make home visits for those patients that could not reliably attend clinic was considered essential for maximum effectiveness. An important aspect of the home-based service is the opportunity it affords workers to see the circumstances of their patients’ lives. This increased rapport with the entire family and provided a venue for confidential and open discussions. It also provides opportunity to identify vulnerable contacts and symptomatic household members. DOT personnel often become very attached and involved with their patients well-being. This is a strong motivator for their work, yet can also be a stressor when patients have needs that are immense or succumb to their illnesses. Outreach staff may benefit from opportunities to express their emotional reactions and frustrations in supportive, structured venues. Bi-annual ‘Encuentros’ (meetings) of promotoras do provide an opportunity for sharing this type of feelings and emotions. Also, monthly promotora meetings in each jurisdiction are often used to allow for sharing of this information. Promotoras are encouraged to utilize a daily ‘log’ to write down relevant aspects, incidents and tasks as a way for them to process and address these feelings. Part of the supportive supervision of the promotoras carried out by SOLUCION TB staff involves direct access to Community and City Coordinators over the phone. This is a practice that occurs fairly often and it’s encouraged by PCI SOLUCION TB staff as a mechanism to promptly address incidents, as well as part of the supportive supervisory environment. Patients Patients in both project locations were grateful for the attention and care from their health workers. They acknowledged responsibility for their own care, but felt services needed to be convenient to enable them to attend to their TB and the other elements of their lives. Problems attending clinic-based DOT was mentioned frequently in Tijuana, especially among those who worked or who lived a distance from a center. Some patients admitted they might default, as some had in the past, if treatment was not so convenient. A common theme was the poor knowledge in the community about TB; what the symptoms are, how it is treated, and how it is spread. Several patients had been to multiple medical visits before having a correct diagnosis made. The education they received from their current health care team had been very helpful and they felt they understood the disease fairly well. They supported a public information campaign to alert the population about TB. Nearly half of the patients in Tijuana had seen the US-based Project Accesso infomercial on television, and felt it was the right type of message. Consideration may be given to developing similar campaigns in project cities, though the cost and benefit must be weighed against competing priorities. There remain misconceptions about the way TB was spread. Some patients mentioned separation of utensils or bedding as useful for their family’s protection. Stigma was noted; although the opinion that education could help in dispelling fears was widely held. Patients were also quite emotional about their experiences with TB and engaged in sharing their stories and listening to fellow patients. Having patient feedback in group settings appears to be a viable and valuable method to understand how different outreach and treatment practices affect the patient. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 22 LLR 1.2. Effective strategies for reaching marginalized, high-risk target groups (PLWHA, substance abusers) developed and implemented Persons Living with HIV/AIDS Testing for HIV infection is recommended for all patients diagnosed with active TB. HIV testing in Baja California is based on a blood EIA test (venous sample or rapid testing) with Western Blot confirmation. Patients pay for the tests when they can afford to do so, but may be subsidized based on need. Testing coverage for patients in SOLUCION TB was not verified. A new national TB registry system, the “unified platform”, is currently being pilot tested at the state-level and is scheduled to replace EpiTB in January 2007. This registry adds new fields to national data collection, including whether the patient was offered HIV testing, whether they accepted, and the result. This information will be valuable in future SOLUCION TB project years. The exact percentage of SOLUCION TB enrollees who are known HIV positive was not provided. This information could be determined through a match between the SOLUCION TB and State (EpiTB) databases. Administrative staff report that approximately 5% of SOLUCION TB patients have HIV infection. This would suggest that approximately 10 patients were HIV infected in the 2005 cohort of 206 patients. The status of HIV treatment is not always documented in the TB chart at the centers, although current practice in most cases is to delay HIV treatment until TB therapy is completed. The intersection between TB and HIV is becoming increasingly worrisome among marginalized populations. Recent data from Tijuana reveals HIV infection rates of 5% and TB infection rates above 70% among subpopulations of drug users. Increased coordination with HIV programs within ISESALUD should be a priority. There may be opportunities in both cities to mobilize HIV staff to assist in a variety of TB control activities, including DOT, early case identification, or locating patients that abandon treatment. HIV programs could also assist with training of SOLUCION staff and partners, or benefit by attending TB trainings. Substance Abusers Administrative staff at ISESALUD estimate that 30-40% of patients with TB have substance abuse problems. The exact percentage of SOLUCION TB enrollees who admit to substance abuse was not available. Alcoholism is the only drug use variable captured in the federal reporting system. Baja California has chosen to collect other types of drug abuse categories in EpiTB user￾defined text fields. This field must be hand-tallied and thus are time-consuming to analyze. Health center and DOT staff have developed unofficial relationships with drug rehabilitation centers in their areas. Most of these centers are small and private, and some may come and go over time. Yet, several health centers have incorporated services with these facilities as part of their outreach and DOT work. The personal relationships forged by the DOT workers and promotoras with their local centers are an important way to reach these high-risk populations. The role of SOLUCION TB has been to strengthen existing ISESALUD outreach to rehabilitation centers. Incentives for patients, like Ensure nutritional supplements, have been bought with project funds. DOT staff interacts with the centers in several ways. They may provide DOT at Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 23 the centers directly or may deliver weekly supplies of TB medication for rehabilitation staff to give DOT themselves. In the latter situation, apparently, patients are not enrolled as a SOLUCION TB patient. Rehabilitation centers are asked to transport TB patients to their monthly medical and sputum collection appointments, and are given gasoline vouchers to cover the expense. In Tijuana, a formal relationship with a private, non-profit drug treatment organization, CIRAD, is being implemented. CIRAD operates several centers with a daily census of over 700 clients. Currently, a SOLUCION TB promotora is providing DOT at one CIRAD center. CIRAD also operates Las Memorias, a residential center for HIV-infected clients. ISESALUD is providing DOT at that facility. SOLUCION TB incentives are used to support that relationship. Training CIRAD staff to provide DOT to their own clients should be a core capacity building and sustainability strategy. LLR 1.3. Capacity of laboratories to conduct procedures improved This was initially envisioned as a key component of the project to ensure patients have access to high quality sputum microscopy services. However, ISESALUD was able to hire adequate staff early in the life of the project. Therefore, this result area is not being pursued as a SOLUCION TB activity. LLR 1.4. Organizational information and communication systems for DOTS improved and expanded Connectivity Computer equipment and internet access has been adequate for project activities. Email has become the routine method for communication between the state, local directors, project coordinators, and PCI management. SOLUCION TB paid for three laptops, and three printers for the State office and the TB jurisdictional directors. Maintenance of the phone lines and email access has been absorbed by ISESALUD. Reports on DOT administration for each patient are maintained by the DOT personnel on the Registro Semanal de Entrega de Medicamento de Tuberculosis. At two-week intervals, the forms are delivered to the jurisdictional offices and copies are sent to the State. At the State level, the information is entered into the SOLUCION TB database. At the State, 1.5 positions assist the TB Director with data entry and administrative tasks. Data entry requires four hours per day. There is no integration between the SOLUCION TB database and the EpiTB systems, thus double entry of patient information is occurring. The SOLUCION TB database captures details of daily dose observation, which is not part of the EpiTB system. At the jurisdictional level, each TB unit has part time SOLUCION TB assistants assisting with a variety of tasks, from data management to DOT personnel supervision. As noted, a new federal disease reporting system is being instituted. The impact of this change on the personnel resources required at the local level is unclear. Connectivity remains a problem at the local health center level. Computers are not available in the majority of centers, leaving face-to-face information sharing the main avenue of Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 24 communication. Access to faxes at the center level would improve efficiency for transmitting clinical data (e.g. sputum results) and submission of required forms. A policy and procedure manual has not been developed for SOLUCION TB. This would be helpful to ensure participants understand criteria for patient enrollment and data flow, including timelines, responsible parties, and directions for use of forms. Staff can describe their routine practices and felt that training was adequate about required documentation and processes. Standardization would be maximized, however, if a manual were available for reference and review. Logic Map The planned logic map web-based site has not yet been developed. This would be valuable for efficient distribution of training materials, meeting dates and outcomes, and project updates. It could have particular utility in providing more timely access at the jurisdictional level to cohort demographics and outcomes. Information about patient characteristics, status of enrollment, and outcomes is not widely distributed. The ability to review such data should provide jurisdictions useful information for improvement in operations and approaches. Intermediate Result 2: Political commitment for national and state TB control program improved and sustained Achievement in this result area is critical for the long-term sustainability of successful SOLUCION TB strategies. Although, treatment outcome data are not yet available for the majority of enrolled patients, all participants felt confident that adherence to treatment has been strengthened by committed DOT staff being placed in communities. Assuring these gains are incorporated into existing jurisdictional and community systems is a concern for project partners. LLR 2.1. SOLUCION TB model strategy approaches, tools, and results documented and effectively shared Prior to the SOLUCION TB project, DOT was already practiced on a more limited scale within the two project sites. In 1999, an initial expansion project, funded in part by the Border Health Commission began to develop the partnership between ISESALUD, PCI, and community-based providers of DOT. The current project has successfully trained and deployed additional DOT personnel. As an avenue to sustainability, project partners hope to increase the number of community-based groups with trained DOT staff. Drug rehabilitation centers are current partners. The voices for a political dialogue about TB control should be strengthened as more care organizations become aware of the TB situation and begin to have a stake in preserving the newly strengthened infrastructure. The planned launching of the internet-based logic map should proceed in years 3 and 4. Strategies and materials that have been successful should be documented in clear and simple formats. Consideration should also be given to sharing barriers have been addressed and describing strategies that have been unsuccessful, in order to assist others in understanding the full scope of the project’s intentions, implementation, and results. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 25 Regular monthly meetings are held with outreach and administrative staff from PCI and the jurisdictional level. These meeting are mostly information sharing from management to frontline staff. DOT personnel expressed interest in more opportunities to discuss specific patient problems and develop workable solutions. Many staff felt that health center-based meetings were very productive in bringing together all levels of providers, as well as SOLUCION TB administrative liaisons. Largely, the utility of the health center meetings (quarterly quality circles) rely on the engagement of their medical staff. Fostering health center involvement to develop sustainable partnerships and advocates within the jurisdictional hierarchy is important. Reducing the number of all-staff gatherings to every other month may allow more time to strengthen health center relationships. Quality Circle meetings take place intermittently. Not all personnel could distinguish a Quality Circle meeting from the other types of reunions they attended. There was, however, agreement that quality improvement and sharing issues of mutual concern is extremely valuable. The key elements for lasting and meaningful progress, regardless of the various opportunities for sharing information, was seen as involvement and support of the health center medical director and adequate resources for required work. Larger encuentros occur two times per year. These sessions bring DOT workers and the SOLUCION TB management teams from both cities together. These have been helpful in expanding opportunities to share day-to-day experiences, as well as to establish connections with colleagues across jurisdictional boundaries. These encounters provide support for DOT staff who can feel isolated from colleagues and not officially part of the ISESALUD organization. Future encuentros could be expanded to include DOT workers from other jurisdictions including San Diego and Imperial counties, or other northern Mexican jurisdictions. Project partners should be able to present the elements and outcomes of SOLUCION at a variety of important meetings in Years 3 and 4. All of the venues described in the DIP, including medical school administrative committees, the local TB-HIV coalition, and the State StopTB working group should have the SOLUCION TB DOT expansion project on their agendas. Innovative messages about DOT and its basic role in every community’s fight against TB should be considered for events such as World TB Day and high profile public health and policy sessions. LLR 2.2. Dialogue about improving TB policy increased A variety of activities to increase the local, state, and national dialogue about TB control is outlined in the DIP. One on one discussion about the design and role of SOLUCION TB in strengthening DOT has occurred at meetings such as the 2005 US-Mexico Border Health Association. ISESALUD management, from national to local levels, remains supportive and optimistic about maintenance and integration of community-based TB control activities. The power of engaged medical providers should not be underestimated. For example, the influence that medical directors have on patient care at the health center level is pivotal. Assuring their interest in maintaining and expanding a strong infrastructure for DOT in communities should be a focus of activities to influence the policy dialogue. Health centers that have a supportive medical staff are able to develop solutions to difficult patient management issues. As Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 26 success for patients is realized, commitment takes hold. Opportunities to engage medical providers and nurses at a variety of levels should be sought. In the first two years, it was important to lay the groundwork for DOT expansion strategies. In years 3 and 4, it will be vital to implement a variety of methods to improve and inform the dialogue about TB policy. Strategies to educate and persuade opinion leaders about the value of DOT expansion must be undertaken. Intermediate Result 3: Quality utilization of TB DOTS components of existing local medical school curriculum increased LLR 3.1. Increased participation of medical school faculty in SOLUCION TB events In Mexicali, participation of the UABC (Autonomous University of Baja California) medical school has been targeted. Dra. Morales is on the faculty and provides sessions to medical students on general TB control and the DOT strategy. Sessions are given each semester reaching approximately 55 students in 2006. Seventy-eight medical students from Xochicalco Medical School, in Tijuana, received training in March and August 2006 from Dr. Cerecer. Selected participants were “pasantes”, recently graduated physicians who are completing a year of required government service in underserved areas. General TB training had been part of their curriculum prior to the SOLUCION TB project, but specific information regarding adherence and DOTS have been added as a result of the project. Training medical students is a vital long-term strategy for strengthening TB control. Students should have appreciation for the role of strict treatment supervision in an overall TB program. The content of the previous trainings could be posted on the SOLUCION TB website. However, these activities should not be priorities for the SOLUCION TB project in the two remaining years, as other trainings, such as for private providers, would have more immediate impact. LLR 3.2. Number of students participating in hands-on DOTS fieldwork increased Students have not participated in DOTS fieldwork. If possible, hands-on opportunities should be built into the learning curriculum at medical schools in the project areas. Ideally, partnerships between the medical schools and project participants should be enhanced in order to create sustainable opportunities for students and doctors-in-training to become engaged with the community-based aspects of TB control. However, this strategy may be limited by the time remaining in the project, liability concerns, and higher priority project activities. 2. Cross-cutting Approaches a. Behavior Change and Communication Communication has been enhanced in several key areas. First, face-to-face meetings over the first two years have provided forums for deeper understanding between the jurisdictions, PCI, and partners. This has facilitated the partners’ abilities to identify the roles they can play in TB control and to align their vision. Secondly, ensuring connectivity through the internet has improved efficiency and real time information access between federal, state, local, and Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 27 community partners. Planning, sharing problems, and reviewing agreements are among the elements that now have become routinely handled on-line between partners. Peer communication between DOT workers has been very successful. There appears to be great cohesion among staff in each city. They look to each other for support and mutual understanding of successes and barriers they face. They seem empowered to advocate for changes to enhance patient success, such as more training and assistance with transportation. Positive communication within health centers, and between centers and the jurisdictional administration enhances worker satisfaction and optimism about maintenance of the DOT infrastructure. Staff in the centers appreciated visits by supervisors and felt that the more their center directors were involved with planning and operations, the better the support for their activities would be. A key SOLUCION TB focus has been on stigma. Stigma is a barrier to TB control; both as it affects patients’ willingness to be treated and providers’ willingness to care for patients. The 2006 trainings on this topic were acknowledged as effective and necessary. Several line staff observed positive changes in the attitude of medical providers and enhanced support of DOT efforts. The training was adapted from HIV-specific materials. Future trainings could be even more successful if they address core factors that limit change by medical professionals. For example, the stigma training did not adequately relieve anxiety about personal risk of exposure to TB. This realistic fear is a powerful determinant of provider attitude and one that, in turn, creates situations where patients sense they are unwelcome. Additional trainings on this topic should be planned to reach more providers and to explore the topic more deeply. Patients also mentioned that the caring attitude and education provided by DOT staff helped many to understand their previous attitudes about TB had not been well informed. Fact-based messages to the public could provide essential knowledge to combat stigma and influence public attitude on a wider scale. Not fully explored, is the subtle sensibility on the part of some medical staff that patients may be partly responsible for their illness and, at a minimum, they should be held responsible for their treatment. While community mobilization strategies may assist patients in taking more active roles, the patient-centered approach to TB control acknowledges that providers have the primary role in assuring patients are successful. Ultimately, successful incorporation of DOT into TB programs occurs when there is a cultural shift to accepting the strategy as an essential element of treatment. This should be an area for exploration and training in upcoming project years. Another way that the value of DOT can be established is through use of data. There should be a focus on bringing outcome data back to the health center and provider levels. The level of DOT adherence and its effect on treatment length and success, and cost, may be helpful in bolstering interest in maintenance and expansion of the strategy. b. Capacity Building Approach One element of capacity building has been enhanced training of ISESALUD and community￾based staff in TB diagnosis and treatment standards. The project has also leveraged trainings offered at related conferences such as the 2005 Curry Center TB conference focusing on multi￾drug resistant TB. In the first two years of the project, a cadre of committed, well-trained, experienced DOT personnel has been developed. Importantly, reaching out to health center medical staff continues to be essential in strengthening their appreciation for the key role they Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 28 play in the overall TB control efforts of their community and their state. Training topics in upcoming years should include more information about prevention of MDR TB through sound therapy practices, infection control strategies for health center and facility-based staff (such as rehabilitation centers), and the triad of HIV, substance abuse and TB. Private providers should be increasingly engaged in the next two years. Patients noted that private providers missed the diagnosis in many of their cases. Lack of linkage of private providers to ISESALUD resources is well established. Training physicians about diagnosis, treatment strategies, reporting advantages, and how to create observed therapy options for their patients should be considered. Providers at major hospitals may be specifically targeted to receive training on discharge planning for TB patients. Anecdotes were shared, which revealed unnecessary gaps in treatment continuity when patients left the hospital. TB patients were generally referred to their local health centers, but reliable and comprehensive notification was not routinely received by the health center. Three educational sessions were provided for 200 physicians mainly from the Associations of General Practitioners in Tijuana and Mexicali. Goals of the training were to increase knowledge of the national TB treatment norms, increase knowledge of DOT, and improve communication between ISESALUD and the private community. Trainings to general practitioners should continue, and consideration should be given to including pulmonary, infectious disease, and pediatric associations in future sessions. Inclusion of disease rates, HIV-coinfection trends, outcomes with DOT, and other evidence-based information will be especially important in engaging providers as TB advocates and believers in the value of DOT and patient-centered approaches. Another area that may provide sustainable improvements in treatment outcome is incorporating the CureTB/Binational Card strategy where appropriate. The initial outcomes of SOLUCION patients reveal that four moved and 18 abandoned treatment. Some may have moved or traveled to the US or another jurisdiction in Mexico. Targeted use of the CureTB referral system may improve successful outcomes. Experiences in US states has shown that early discussion with potentially mobile patients about the Binational Card purpose and process can foster an open discussion about travel plans and how to secure care after a move. c. Community Mobilization The main community mobilization strategy in SOLUCION is to increase collaboration between natural partners in TB control: rehabilitation centers, HIV programs, and organizations with existing outreach worker programs. Work with rehabilitation centers was taking place prior to the SOLUCION project, but linking the centers as partners has been enhanced by providing resources to give selected rehabilitation centers incentives for providing DOT to their residents. This strategy should be broadened and formalized, and should seek to create partnerships based on mutual interest in client success. Opportunities to provide education on topics such as TB prevention and infection control may be useful. Government and community-based HIV programs will be important colleagues to mobilize in remaining project years. The recently USAID-funded PreveTB project combines approaches to case-finding among high-risk populations, with testing for TB infection among drug users. Groups that have long-standing links to HIV and drug using populations, could be invaluable in Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 29 disseminating information about adherence, mobilizing resources for incentives, and advocacy with funders and decision-makers in their spheres of work. Many existing community-based groups have competing priorities, which limits their ability to take on new initiatives. Resources are invaluable in keeping these groups engaged in specific issues, yet the level of resources needed to maintain assistance in TB education, and even DOT service, should be explored. d. Sustainability Strategies Improved communication is an area where lasting change has already been accomplished. Assuring the key TB control administrators are connected by phone and email to each other, community partners, and national level personnel is firmly established. The hiring of ten DOT workers will be a significant accomplishment in sustainability. Several obstacles exist. First, resource allocation must occur at the State level. Second, positions must fit into existing job categories within ISESALUD. Finally, staff must meet the job qualifications established by ISESALUD. Qualifications include at least a nurse assistant certification. It is not clear that all current DOT workers can be hired by ISESALUD, despite their training and experience. The absorption of 4 staff is scheduled for January 2007. Another major strategy for sustainability is to work with relevant community institutions to incorporate DOT into the duties of their existing staff. These groups could include drug rehabilitation centers, the Las Memorias facility, and organizations like migrant residential projects, home for the elderly, dialysis centers, and schools. This effort goes hand in hand with capacity building. As these organizations are identified, educated, and have staff mobilized to reliably deliver DOT, the capacity to provide observed therapy will be expanded and sustained across multiple community partners. Esperanza is a community outreach organization that has been in existence for many years. Three Esperanza promotores collaborate with and are funded by SOLUCION and take part in all training and capacity building activities. This organization should be assisted in building capacity to continue DOT services beyond the LOP. The HIV programs within ISESALUD are critical partners. In Tijuana, the HIV program already has outreach activities among marginalized groups. Cross-training HIV staff to provide DOT and DOT staff to provide HIV educational messages may yield significant opportunities for leveraging scarce resources. A stated strategy in the DIP is making social service clubs aware of SOLUCION activities. Groups, like Rotary International, are specifically interested in community-based projects and have been engaged in TB-specific initiatives elsewhere along the US-Mexico border. Establishing relationships with similar groups is a mechanism toward sustainability through increased advocacy partnerships, as well as broader opportunities for funding. 3. TB Project Indicators Under each result area, the key indicators are reviewed. Not every project indicator and benchmark listed in the DIP are detailed below. For example, the pre- and post-test results for each training are not included, but the results are available from project coordinators. This Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 30 evaluation concentrated on the areas deemed most meaningful in terms of the core goals and strategies of the SOLUCION TB model. I.R.1. The SOLUCION TB model implemented in the departments of Mexicali and Tijuana, Baja California Treatment completion + cure = 85%: Not met, in progress. The initial cohort shows an overall success rate of 82%. Project operations were in start-up. The second cohort is not scheduled to have final outcomes until January 2007. The rates for each cohort should be compared with the comparable non-SOLUCION cohort. Semester Number enrolled HIV/AIDS Drug Abuse Cured* # (%) Completed Failed Died Abandoned Moved Continues Jan-June 05 100 82 (82%) 0 2 (2%) 1 (1%) 10 (10%) 2 (2%) 3 (3%) July-Dec 05 97 72 (74%) 0 1 (1%) 2 (2%) 8 (8%) 2 (2%) 12 (12%) Jan-June 06 10 1 1 - - - - - - - July-Dec 06 0 - - - - - - - - - *Of those with a final outcome: Number enrolled – number who continue = Number with final outcome Decrease to 3% the proportion of patients who abandon treatment: Not met, in progress. The initial cohort showed an abandonment rate of 10%. As above, the second cohort results will be available in January 2007. The abandonment rate for each cohort should be compared with the comparable non-SOLUCION cohort. L.L.R.1.1. Community-based promotores system to increase direct observation of treatment (DOT) for TB patients improved and expanded All DOT staff hired: Met Monthly meetings held: Met Patients assigned to DOT staff = 1190 over life of project: In progress. In the start-up year, 197 patients were enrolled. Promotore and supervisor staff trained: Met Quarterly quality circles accomplished: Met 10% of promotores supervised monthly: Substantively met. Percentage supervised varies each month, but supervision takes place regularly. Encuentros take place twice per year: Met L.L.R.1.2. Effective strategies for reaching marginalized, high-risk target groups (PLWHA, substance abusers) developed and implemented Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 31 Two rehabilitation centers will be chosen for participation: Partially met. One large drug rehabilitation center in Tijuana is being targeted for formal inclusion in Year 3. Several small centers are being served by DOT promotoras (incentive system for participation is for the large rehab center). 100% of enrolled patients will receive medical follow-up: Not evaluated. The specific indicator for medical follow-up is unclear. The indicator may be collection of monthly sputums or monthly physician visits or both. This should be reviewed for Years 3 and 4. L.L.R.1.3. Capacity of laboratories to conduct procedures improved No longer needed and therefore included in the project. L.L.R.1.4. Organizational information and communication systems for DOTS improved and expanded Utilization of internet-based system for information sharing: Met I.R.2. Political commitment for national and state TB control program improved and sustained ISESALUD budget allocations for National Tuberculosis Program (NTP) activities increased: Not met. Expected movement of staff to ISESALUD has not occurred on schedule. This will be a focus in Years 3 and 4. One MD seminar held annually in each jurisdiction: Met. HIV-TB-SA committee established, Stop-TB committee established, 10 meetings with decision￾makers: Not met. The first two project years were appropriately focused on operational implementation. Advocacy activities will be reviewed for initiation in Years 3 and 4. L.L.R.2.1./ L.L.R.2.2. SOLUCION TB model strategy approaches, tools, and results documented and effectively shared/ Dialogue about improving TB policy increased Benchmarks were not met. As above, the first two project years were appropriately focused on operational implementation. Expansion and advocacy activities will be reviewed for initiation in Years 3 and 4. I.R.3. Quality utilization of TB DOTS components of existing local medical school curriculum increased Training plans for medical schools will incorporate a DOTS component: Partially met. Education sessions were held for doctors in training in each city. Broader participation by medical school faculty was not achieved. Depending on project priorities for Years 3 and 4, this element may be deferred. L.L.R.3.1. Increased participation of medical school faculty in SOLUCION TB events Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 32 One seminar per year with 25 students/faculty: Partially met. Faculty has not been engaged. Depending on project priorities for Years 3 and 4, this element should be omitted. L.L.R.3.2. Number of students participating in hands-on DOTS fieldwork increased Field placements of students in two clinics per jurisdiction: Not met. This activity needs approval by faculty and implies liability for the jurisdictions. Depending on project priorities for Years 3 and 4, this element may be omitted. C Project Management 1. Planning In the initial planning efforts, input was solicited from federal, state and jurisdictional level HIV and TB personnel. The project plan is well thought out, especially in the area of staff training and communication. The work plan was ambitious and targets in Result Areas 2 and 3 are not yet met. However, Result Area 1 is the key part of the SOLUCION TB project and needed to be implemented before concentration on most other areas could logically proceed. Result Area 1 targets for treatment success were ambitious, but many of the core elements have been realized. Health center directors and the medical staff who see TB patients at the centers were not part of project planning. Even jurisdictional staff felt they had been only somewhat involved in the actual design. This lack of inclusion created varying levels of resentment, confusion, and resistance during the rollout of activities. Expansion or replication of the model should involve planning partnerships at the service delivery level. Strategies to ensure that community level staff is able to contribute and to have their input incorporated in a meaningful manner must be established. Objectives are well understood by state, jurisdictional, and PCI staff. It is not clear that health center staff or the DOT personnel are aware of the breadth and full scope of the SOLUCION TB project. It does appear that all participants understand that new pulmonary, smear positive patients are targeted for enrollment and priority DOT services. Community partners and line staff are focused on their part of the activities. Various participants expressed hope that advocacy will happen at the higher levels to assure that outreach and DOT support can continue and expand. Patient enrollment information has been used to redirect program design in some areas. For example, as noted previously, many promotoras in Tijuana were not being assigned the target level of five patients. Low patient assignment per staff caused the recent shift of some promotoras to health centers. Target enrollment levels have been achieved in Mexicali, where center-based workers are able to provide DOT for more than five patients and could also attend to non-pulmonary and relapse cases as well. Unfortunately, Tijuana promotoras did not feel they had been part of the planning for design change and because of the reduction in income caused by the changes, some feel unappreciated and discouraged. Data on the number of drug users and HIV-infected patients enrolled in SOLUCION TB have not been available to date, nor have data on treatment outcomes. It will be important to use this data to revise activities in the upcoming months. Moreover, data reports have not been made available to participating health units. This would be useful to build trust and communication between the Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 33 partners throughout the life of the project and to create the capacity to continue open data sharing for continuous TB control improvement. 2. Staff Training Staff training has been a priority activity and has been discussed in other parts of this evaluation. A variety of venues has been used and clinical as well social and quality improvement concepts have been taught. Pre- and post-tests are used to assess concept acquisition. Supervisors review DOT personnel work at regular intervals to assure protocols are being followed. Staff feels they would benefit from more training. Information on topics such as infection control, HIV testing, and mental illness would be valuable to incorporate into training during the remaining project years. Including community partners and medical staff in all trainings would be helpful to increase collaboration and inclusion. Thought should be given to increasing the use of case conferencing in which medical staff and DOT workers are brought together to brainstorm solutions to difficult patient management situations. An additional training need is the development of a policies and procedures manual. This manual should provide clear instruction on the method and responsibility for patient assignments, how to complete project forms, timelines of submission, chains of command, workflows, etc. Such a document would standardize understanding and practice, as well as provide clear expectations for participants and staff. A SOLUCION TB newsletter is being developed. The newsletter will be provided to staff, health centers and other project partners. It will include articles on TB treatment norms, outcome and disease trend data, and DOT activities. Ideas for articles and features and other points of distribution will be reviewed in an ongoing manner. This is an excellent strategy for continuous education, partnership building, and information sharing. Challenges will include creating and distributing the newsletter on a regular basis. 3. Supervision of Project Staff Broadly, administrative supervision is done by PCI and medical supervision by the jurisdiction. However, the pattern of this spilt supervision varies within the project. In Mexicali, supervision is handled primarily by the jurisdictional administration. In Tijuana, PCI administration is more involved in the health centers and the jurisdiction oversees the field promotoras more directly. Both systems appear to work well within their context. There is a form that all DOT staff are required to complete; the Registro Semanal de Entrega de Medicamento de TB (RSEM). However, the purpose of this form is understood differently by different staff. Some believe it is to have patients confirm that they were given the doses noted on the form. Others believe it is to motivate patients to conform to treatment requirements. Still others think it is a supervision tool, to assure they have done their assigned work. In the limited review, it was evident that the RSEM is completed differently by different staff; some having patients sign at the beginning of the week, some at the end, and some allowing proxies to sign on behalf of the patient. Moreover, some forms have daily doses indicated when the patient was given a weekly supply. The information is entered into the SOLUCION TB database and linked to the specific staff member responsible for the DOT. This information could be used to identify Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 34 workload and outcomes for each worker. However, the data entry is time-consuming and results are not available to supervisors in a timely manner. A potentially effective supervision tool is the Quality Improvement Verification Checklist. This form is used by ISESALUD and PCI staff to evaluate performance of the promotores and DOT workers. The goal is to review 10% of staff each month, but the level of actual reviews per month varies. The form lists competencies in patient treatment and education that the employee should have mastered. Supervisors mainly observe interaction between the worker and patient, but may ask the patient questions to augment their observations. Patients are generally provided an opportunity to give verbal consent to have a supervisor visit their home. In some cases, the ISESALUD and PCI supervisor do the evaluations jointly. This is a good practice to ensure transparency in the quality of information collected. Sometimes patients are asked questions with their DOT provider present, which may introduce some bias. The value of the RSEM is unclear. Having patients sign the form is not a reliable indicator of the number or timing of the doses given, nor is it necessarily a motivator for personal responsibility. Patients already sign an ISESALUD-required declaration of responsibility at the outset of treatment. It is not clear that patients receive standard information about the form. Patients may feel required to sign the form to receive service. Another concern is the layout of the form, which includes space for five patients on each RSEM sheet, thus compromising confidentiality. A comprehensive review of the design and utility of the RSEM should be undertaken. To fully assess workload per worker, a broad picture of all assigned cases, SOLUCION TB plus non-SOLUCION TB cases, is needed. To accurately assess the number of DOT doses provided, strategies such as unannounced, spot observations and private patient interviews should be considered. The value of data entry of each dose should be reviewed. It is labor intensive and may not provide more information than the standard tarjeton. A more useful endeavor would be to develop methods to quantify weekly allocations of medication, a practice that appears to be widespread but not openly reported. Neither the tarjeton nor the RSEM measure this practice. 4. Human Resources and Staff Management All SOLUCION TB positions are filled; 27 promotores, 7 DOT workers, and 5 administrative staff. DOT workers may be assigned to assist with some administrative tasks based on their higher salary and skill requirements. There has been no staff turnover in the project. A policy and procedure manual has not been developed. Staff learns expectations and operations through training, on the job experience, and through meetings and direct instruction. At project end most administrative staff will remain with their respective organizations and either work fewer hours or be switched to other funded projects if resources for DOT expansion are not maintained. A total of 6 DOT workers are scheduled to be absorbed by ISESALUD. This remains contingent on resources and individual qualifications. Promotores will remain in place if resources are secured, otherwise most will be released or moved to other funded projects. Moreover, because current staff is not employed directly by ISESALUD they have substantial flexibility in work hours and pay schedules. This flexibility would not be maintained if they were employed by ISESALUD directly. Need for control. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 35 5. Financial Management No significant problems were identified. Current financial practices are sufficient for the needs of the project. Funds come to the PCI International Office and are provided to the PCI Mexico SOLUCION TB cost center based on needs projected for the month. Funds are transferred to the MSC (Social Community Medicine) affiliate in Tijuana to pay staff and other costs. A close working relationship between the offices assures adequate controls and flexibility is in place for smooth project operations. For example, ISESALUD has encountered delays in their ability to absorb DOT workers. Funds saved in other project areas were reallocated to maintain these staff until ISESALUD was able to take them on. The match target for the SOLUCION TB project is $531,400. Through September 2006, PCI had booked a total of $363,351 in cash and in-kind match representing 68% of the target. Cash and in-kind contributions include: personnel expenses such as the ISESALUD State TB Director, health center physicians and nurses, and health promoters; ISESALUD medical supplies, fuel expenses covered by ISESALUD; and meeting and office space provided by the University of Baja California. 6. Logistics All planned equipment and installations have taken place. Decisions about the type of equipment to purchase were made based on the requirements of the users. No significant problems were encountered. Computers and printers purchased for ISESALUD will remain in their possession at the project’s end. Early in the project, a vehicle was donated to the State TB program. This has been used by the Mexicali jurisdiction primarily for supervision visits and meeting attendance. 7. Information Management There have been effective systems put in place to collect project data. The RSEM forms are completed and filled out weekly and are collected and delivered in a timely manner. The SOLUCION TB database can be used in conjunction with the existing national EpiTB database to collect patient adherence, outcomes, and demographics. The number of attendees and outcomes (pre- and post-tests) from trainings and educational sessions are collected by SOLUCION TB coordinators in each city, as are the Quality Improvement Verification Checklists (QIVC) supervision forms. Refer to Item 3 for comments on the utility of the RSEM and QIVC forms. It is not clear that the expected use of the RSEM to track effectiveness of each DOT staff person has been realized. Data entry is lagging and reports from the initial cohort have not yet been distributed to supervisors. As noted before, the quality of the information is variable and may give false impressions about relative effectiveness of staff. Overall treatment outcomes as collected in EpiTB may be a superior measure. Analysis of all SOLUCION TB treatment data are done at the State level. The data analysis has been delayed, in part by a 3-month absence of the State TB Director. To date, there has been no management review of adherence or treatment outcomes. Ideally, data on adherence and outcomes should be reviewed by the partners every six months. SOLUCION TB outcomes by Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 36 cohort should be compared to non-SOLUCION patients from the same semester. Data from each city should also be reviewed separately. Over time, it might be useful to compare results between different centers. Information from trainings has been used to identify success and interest in the sessions, as well as to plan additional educational activities. QIVC information has been used to assess staff performance. Patient outcome data is now becoming available and should increasingly be useful to gauge effectiveness of SOLUCION interventions and to refine operations, 8. Technical and Administrative Support The SOLUCION TB project receives administrative and technical support from the PCI International Office in areas such as planning, meeting facilitation, and report generation. There has been minimal collaboration between SOLUCION TB and other USAID-funded TB DOT projects. USAID should create or support venues for sharing experiences between agencies involved in DOT development and expansion models. 9. Mission Collaboration The Mexican Mission office has been involved and supportive throughout the project. Molly Lindner has attended several key meetings and has reviewed operations. The Mission has been available for technical assistance, including insight into national level plans and priorities. Recently the Mission has acknowledged the success of the SOLUCION TB model of private￾public partnership by promoting an expansion of the program into 12 other Mexican states. D. Stories of Success V’s Story V was 19 years old when we met. She lived with her mother and two brothers in a poor neighborhood in Mexicali. She had a lot of family problems and had started using drugs. When she first was diagnosed with TB she was hesitant to take the medicine, but after much education about the disease and DOT, she began treatment. She came to the health center for her medication, but would miss a dose here and there. I would go to her home and she had a hundred reasons to justify not showing up; “my mom can’t bring me to the clinic”, “the medications don’t work and make me feel depressed”, “they make me nauseous, I’m afraid to take them”, etc. I encouraged her to “feel the success” of continuing treatment and sharing that success with me. I told her it was important to me for her to get cured, that we were working together towards that goal. She wasn’t alone and could count on me to support her in her struggle --that the SOLUCION TB team was a part of her family. It wasn’t an easy process, because we held these dialogues for the entire 6-month treatment period. Little by little, however, she came out of her depression. She began looking at things differently. She stopped seeing people who were not a good influence in her life. She met a young man who eventually became her husband. She came to the health center more regularly and finished her treatment! Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 37 Right now she is a healthy, young mother of a precious baby. We are thankful she had a chance to change her life. She keeps bringing sputum samples to make sure she remains cured. Her relationship with her mother has improved. Her son is her pride and joy. R’s Story I used to bring medicine to a patient and her child. One day her husband told me about his friend, R, who was very sick. I told him that I’d like to visit his friend to see what was wrong. He took me to his house. I found a very thin and upset young man. He asked me for medicine. I used my knowledge to explain that we needed to get proper specimens before starting treatment, to be sure of the diagnosis. He said he was unable to bring specimens to the clinic, and I told him I could bring them for him if he wanted me to. I also took the time to explain how TB was transmitted; how he may have contracted the disease and how he could spread it to others. Three weeks later I spoke to him again. He had not brought a specimen because he thought he was dying. I brought a specimen cup to him and delivered it to the lab. It confirmed that he had TB. I became his DOT worker. His mother started to confide in me and told me that R used drugs. As my rapport with R grew, I asked him if he would like to enter drug rehabilitation. He told me he was afraid to go to such places. Through much work and support, he completed his treatment. Two months after he completed his therapy, I talked to the patient about a rehabilitation center I had learned about through my church. I had spoken to the director who was interested in having him in the program. This time, my patient accepted. Today he is still at the rehabilitation center receiving help, and is cured of his TB. M’s Story M had learned about TB seven years earlier, when her brother was diagnosed. M took care of him with little support from family or friends. Her brother was cured and M thought TB was a thing of the past. M was diagnosed with diabetes at the age of fifty, and a year later, started losing weight. She was initially happy about the weight loss and, at first, did not pay attention to the cough, sweats or malaise she was experiencing. After a few months, M saw several private physicians and was prescribed costly medications without improvement. She talked to her brother and realized it was probably TB. Her brother took her to a health center in Tijuana. She felt some rejection by staff at the health center and was not sure she could visit the center every day to receive DOT. Her brother was working six days a week and could not bring her to the visits. Soon after the initial diagnosis, she developed a collapsed lung and was referred to a hospital. When she returned home, she decided she did not want to continue the difficulties of treatment for this disease. She was told about SOLUCION TB and cautiously agreed to have a promotora visit her in her home. At first M felt uncomfortable at the thought of having a stranger come to her house, and worried that the person would make her feel bad about having TB. However, as soon as she met her promotora, she changed her mind. Her promotora was more than just someone who gave her pills; she would take care of setting medical appointments, she would remind M when her sputums were due, and she provided moral support. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 38 M is finishing her treatment this week. Looking back, she feels she would have abandoned treatment after the first few months without her promotora’s help and support. M has become active in her community, promoting TB awareness and slowly helping change some of the negative attitudes and stigma about TB prevalent in her neighborhood. E. Conclusions and Recommendations Conclusions and specific recommendations are grouped under five broad areas. These areas address ongoing SOLUCION TB activities, as well as suggested scale-up considerations. DOT (Directly Observed Therapy) Model The SOLUCION TB project has increased DOT capacity in Tijuana and Mexicali. The model of specialized community-level TB outreach staff appears on target to improve treatment success. DOT staff are committed and motivated to provide service to their patients. Two levels of DOT staff have been created; DOT workers and promotoras. Involvement of the ISESALUD health centers is important for long-term success. SOLUCION TB should have more flexible enrollment criteria to reflect the mix and transmission risk of patients under treatment in the community. • Maintain a blend of home-based and clinic DOT. Develop methods to assess the relative effectiveness of DOT delivery setting, such as adding “location” variables to the database or conducting patient focus groups. Develop criteria to standardize assignment to home-based vs. clinic DOT. • Formulate treatment supervision algorithms, which would provide rigorous case management and documentation, while building in flexibility to maximize patient adherence and resource use (e.g., provide strict DOT for all smear positive, pulmonary cases for the intensive phase, then risk-based determinations on strict DOT vs. weekly allocations for SAT in continuation phase). • Develop written guidance to expand the formal reach of the model to patients with specific, high risk profiles, such as relapse patients, those with TB meningitis, or HIV co-infected regardless of site of disease. Develop a system to ensure adherence and outcome data on these groups by risk-cohort. • Review the various options for maintenance of the current number of DOT staff. Include the management team and health center partners in the review. Consider costs and benefits of having DOT workers absorbed by ISESALUD versus maintaining promotoras through community-based networks. • Develop methods for reliably monitoring DOT by existing staff at rehabilitation centers and at other community-based facilities. Work with SOLUCION TB staff, management, and health centers to identify community partners with existing infrastructure to provide DOT; such as HIV-centers and schools. • Engage the ISESALUD HIV programs. Identify opportunities to provide or support DOT through existing outreach activities. • Develop strategies to strengthen health center teams. Importantly, involve health centers, and other partners, early in planning for operations that affect their clients, staff, or workload. This includes patient management, staff training, and community mobilization activities. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 39 Sustainability Strategies Sustainability will depend up on allocation of public and private resources. Government and private funding organizations have not been strongly engaged thus far, but should be a primary focus in the next 12 months. Key findings and outcome data have not been compiled and communicated to decision-makers. The reach of resources for DOT could be maximized by partnerships between ISESALUD and existing community organizations with common missions. • ISESALUD should allocate resources for DOT staff. Different models for hiring and managing the provision of DOT should be reviewed and discussed with ISESALUD administration. Factors including skills required, flexibility, cost, oversight, and community support should be considered. • Identify and use existing materials that clarify the level of international commitment to DOT and sound TB control practices. Materials, such as the International Standards for TB Control and the Patient Bill of Rights, should be used to review practices and demonstrate to decision-makers the alignment of SOLUCION TB to worldwide standards. • Share experiences and information across USAID funded projects and other avenues for international networking. Seek successful models in other locales to identify workable local solutions to sustainability and expansion. Sharing models and ideas with other DOTS/DOTS￾Plus expansion initiatives will be critical for scale-up in Baja California and other Mexican states. • Develop a specific, written strategy for engaging decision-makers and social organizations. Training and Education of Staff, Health Professional and Community Health Workers The training and education provided within the SOLUCION TB model has been effective. Longstanding practices, issues, and attitudes that impede progress have started to be identified and acknowledged. Health center staff and private physicians have emerged as interested and critical partners in supporting a comprehensive, patient-centered approach to TB control, which embraces DOT as a core strategy. The co-morbidities of HIV-AIDS, substance abuse, and mental health pose a substantial risk for rising TB morbidity and mortality in Baja California and efforts must be strengthened to address these populations. • SOLUCION TB staff should have targeted training in conditions that complicate the management of TB, such as HIV, mental health, and drug abuse. The training should provide clinical information, but also be used as networking opportunities between individuals and agencies engaged with these populations. • Private medical providers are a key audience for education about TB treatment and management principles. Educational sessions for private physicians should be expanded through work with medical societies including the pulmonary, infectious disease, and pediatrics. Education for pharmacists should be initiated using similar presentations and training venues as for medical providers. • A newsletter should be created to keep partners informed and expand awareness. A written plan for audience, content, and distribution should be prepared. • Long-standing attitudes and concerns, which impact on the provision of patient-centered care, should be the focus of specific training efforts for providers, community agencies, Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 40 decision-makers and staff. Topics should include infection control, stigma, and community mobilization. • Reduce the number of centralized staff meetings and increase health center-based meetings. Introduce practices to identify and track issues needing follow-up to provide additional information for planning and resource allocation. • Develop at least one specific, written strategy for engaging patients in community awareness and mobilization activities. • Eliminate efforts to educate doctors-in-training. While a worthwhile endeavor, other SOLUCION TB activities are of higher immediate priority. Data Management and Utilization Systems to collect data have been developed and are in place. However, data has not been used effectively in planning or advocacy. Cohort information has not been generated in a timely manner, nor has this information been compared to non-SOLUCION cohorts. A more efficient data collection, analysis, and feedback workflow is needed. This core element of the project must be addressed early in the remaining two years. • Six month cohort analyses need to integrate key EpiTB variables, risk factor variables for HIV and substance abuse, and SOLUCION TB adherence indicators. Develop a written plan to set the schedule, format, and parameters of the analyses. Produce twice yearly reports based on the agreed upon elements. • Develop a standard schedule for face to face reviews of outcome and process data. Data must be made available to management and line staff at regular intervals to gauge performance and make meaningful improvement. • Involve all relevant partners in data reviews to ensure realistic strategies for quality improvement are designed and implemented. Health center staff, collaborating community agencies, and field DOT staff should be involved in decision-making. • Agree upon and analyze data in ways that will provide improved clarity for planning purposes, such as city-level outcomes, clinic versus home-based DOT, outcomes by co￾morbid conditions. • Compare outcomes of SOLUCION TB patients with outcomes of comparable non￾SOLUCION TB patients. This type of analysis is important to communicate success to policy-makers and potential funders. • Review the current SOLUCION TB database and data collection system to ensure key variables are captured (e.g. drug abuse) with standard definitions. Review data collection practices to ensure that the information being recorded is used for a specific and well-defined purpose. • Collect accurate data on observed versus self-administered treatment (SAT). Openly address the practice of providing self-administered doses by developing policies to standardize selection and approval of patients for SAT doses, and by developing a method to collect that information. Operational Systems Systems of communication, data collection, and supervision are in place. Refinements in several areas could promote efficiency and clarity of effort. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 41 • Create a comprehensive policies and procedures manual. This should include project forms, definitions and instructions, workflow charts, patient selection and assignment criteria, and data management policies. • Develop the web-based logic map in accordance with the DIP. • Develop a real time monitoring system to ensure DOT staff is being given patient assignments that maintain minimum caseloads. • Joint supervision of DOT staff should continue to involve health center, PCI and ISESALUD supervisors to maximize both inclusion and oversight. Supervision schedules and methods should be specified in the manual. • Patient satisfaction assessment should be a standard element of evaluation and should be done confidentially. This practice can inform management about direct concerns of patients that may be missed otherwise. • Responsibility for ancillary tasks by DOT staff, such as transport of sputum specimens to the labs, should be reviewed. Limits on such assignments or creative solutions to accomplish necessary tasks should be sought. Enhancing electronic connectivity of the health centers should be considered (e.g. fax, computer). • The purpose and design of the RSEM form should be reviewed. If retained, the form should be redesigned to ensure patient confidentiality. F. Results Highlights An innovation from SOLUCION TB is the assignment of TB-specific staff to monitor treatment. In the past, personnel attending TB patients at the community level divided their time between activities of many programs. The ability to focus on the needs and adherence of TB patients varied depending on competing priorities of the center and jurisdiction. As a result of having personnel specifically assigned to attend to TB patients and their families, the treatment of this illness has moved closer to a patient-centered, case management model rather than a disease￾centered, medical model. In both cities, patients commented repeatedly that when faced with situations that put them at risk of treatment default, their DOT worker would take actions to keep them adherent. Some patients had past experiences with long waits for DOT doses and inattentive health personnel that had interfered with successful TB treatment for themselves or family members. Patients expressed feelings of isolation when faced with barriers to continuing their treatment (e.g. conflicting work schedules to receive DOT, side effects, stigma). They felt that the attention of DOT workers, to listen and assist with problems, made a great difference in their success with treatment and provided needed emotional support. As part of this case management approach to the TB patient, DOT personnel have identified many basic barriers to successful treatment, as well as the need to address co-morbidities associated with TB, such as HIV/AIDS, substance abuse, mental health and diabetes. DOT staff have found ways of communicating about these issues to their peers and co-workers, leading to case management discussions to find solutions to keep patients adherent. The focal point provided by the DOT staff has fostered collaboration on behalf of the patients within the community health centers with physicians, nurses and social workers. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 42 In turn, health center medical staff viewed the DOT workers as advocates for TB patients’ needs. They acknowledged that the outreach staff was well-positioned to identify and provide critical information to medical providers about emerging social or clinical problems that patients were experiencing. The DOT personnel have become integral members of the TB health care team and consensus was that, without them, treatment of TB patients would revert back to the less successful medical-model approaches of the past. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ Attachment A Action Plan/Revised Timeline of Activities for FY07 Three IRs and nine LLRs have been identified to achieve the strategic objective. Activities for each IR and LLR are described in the table below. Please note, activities responding to the MTE’s conclusions and recommendations are marked in bold and italics. IR1: The SOLUCION TB Model Implemented in the Departments of Mexicali and Tijuana in Baja California Level Activity Time Frame Personnel Community Daily or three weekly visits for new TB cases Ongoing Promotora/es Health Facility Daily or three weekly visits for new TB cases Monthly reports to jurisdiction Ongoing Same as above DOTS workers, clinic directors Jurisdiction Patient selection and assignment Model will be revised to accommodate two treatment modalities: daily DOT and weekly DOT plus self￾administered treatment (SAT). Definitions, indicators and tools to be identified/revised by management team. Ongoing Articulation of new model completed by February 2007 Jurisdiction-level managers SOLUCION TB management team LLR 1.1: Community-Based Promotor/a System to Increase Direct Observation of Treatment (DOT) for TB Patients Improved and Expanded Level Activity Time Frame Personnel Community TB therapy observed for all participating patients Ongoing Promotora/es; community coordinator Health Facility Facilitation regularized supportive supervision and TB control Began Q1 Year 2 SOLUCION TB Coordinators, Health directors, physicians in charge Jurisdiction Patient selection and assignment Revision and finalization of patient selection criteria Training of staff and promotoras on revised selection criteria, and refresher training on communication flow Ongoing Jan/Feb 2007 February 2007 Jurisdiction managers SOLUCION TB management team w/ input from selected health clinic staff SOLUCION TB management team 1 Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ Definition of patients’ satisfaction measurement tools and mechanisms to implement them Quality circles and coordination between PCI and ISESALUD Revision of weekly reports format Revision and re-definition of SOLUCION TB database Monitoring/supervision visits to 20% of promotoras Forums for promotoras from both jurisdictions/training and motivation March 2007 Quarterly Completed by February 2007 Completed by February 2007 Monthly Bi-annually SOLUCION management team and IO staff Jurisdiction coordinators and TJ and Mexicali SOLUCION TB coordinators SOLUCION TB management team; PCI IO M&E staff Same as above Jurisdiction managers (or designees) and PCI coordinators for TJ and Mexicali SOLUCION TB management team LLR1.2: Effective Strategies for Reaching Marginalized, High-Risk Target Groups (PLWHAs, Substance Abusers) Developed and Implemented Level Activity Time Frame Personnel Community Implementation of collaboration project with 2 rehabilitation centers in Tijuana Start up: January 2007 SOLUCION TB management team Health Facility Coordination of medical follow￾up as required Ongoing, after January 2007 Jurisdiction TB managers; Health clinic physicians Jurisdiction NGO patient selection and assignment of 1 of 3 groups (2 in Tijuana and 1 in Mexicali) Meeting with HIV/AIDS state and jurisdiction level coordinators to design a plan to strengthen collaboration between SOLUCION TB and HIV/AIDS programs at ISESALUD Training of SOLUCION TB staff on HIV/AIDS and substance abuse issues Training of private physicians on TB issues related to Ongoing April/May 2007 Completed by March 2007 First trainings completed by SOLUCION TB management team SOLUCION TB management team SOLUCION TB management team Same as above 2 Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ HIV/AIDS and substance abuse Written guidelines to reach patients with specific high risk profiles and mechanisms for the tracking of adherence and outcome data incorporated in the Operations Manual. Sept. 2007 Feb 2007 SOLUCION TB management team LLR1.3: Capacity of laboratories to conduct Procedures Improved N/A LLR1.4: Organizational Information and Communication Systems for DOTS Improved and Expanded Level Activity Time Frame Personnel Community No additional activities planned Health Facility No additional activities planned Jurisdiction Completion of SOLUCION TB web-page Utilization of web-based system for information and report sharing Adequate and timely communication via email Development of Operations and Procedures Manual February 2007 Ongoing, after Feb 2007 Ongoing Completed by May 2007 SOLUCION TB management team SOLUCION TB management team, IO and ISESALUD key staff SOLUCION TB management team SOLUCION TB management team IR2: Political Commitment for National and State TB Control Program Improved and Sustained Level Activity Time Frame Personnel Community Start up of ‘Voices and Images’ ACSM project Beginning January 1st 2007, and ongoing SOLUCION TB management team; TA from USMBHA as per MOU Health Facility Increase physicians’ awareness of TB control activities via quarterly newsletter December 07, March 07, June 07 and September 07 ISESALUD and PCI staff Jurisdiction Advisory group for ‘Voices and Images’ organized Advocacy plan drafted in March 2007 March-July SOLUCION TB management team SOLUCION TB 3 Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ alignment with ‘Voices and Images’ Meetings and contacts with legislators and decision-makers 2007 Q4 and ongoing management team SOLUCION TB management team and ISESALUD staff LLR2.1: SOLUCION TB Model Strategy, Approaches, Tools and Results Documented and Effectively Shared Level Activity Time Frame Personnel Community Medical associations to host and/or support infection control and TB/HIV seminars Q2 and 4 SOLUCION TB management team Health Facility Selected health clinic staff to participate in seminars on stigma, and infection control Qs 2 and 4 SOLUCION TB management team and ISESALUD staff Jurisdiction Plan, coordinate and implement TB seminars Operations and Procedures Manual completed and shared Webpage utilized for model sharing Coordination with SOLUCION TB expansion activities Q2 and Q4 Q3 and Q4 Q2 and ongoing Q3 and ongoing SOLUCION TB management team Same as above Same as above Same as above LLR2.2: Dialogue about Improving TB Policy Increased Level Activity Time Frame Personnel Community Selection and recruitment of group of patients to participate in ‘Voices and Images’ Q2 and 3 SOLUCION TB management team Health Facility Update key clinic staff on SOLUCION TB progress, TB status and control Q2 and ongoing SOLUCION TB management team Jurisdiction Presentation of ‘Voices and Images’ project results to decision makers and stakeholders Q3 and 4 SOLUCION TB management team 4 Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ IR3: Quality Utilization of TB DOTS Components of Existing Local Medical School Curriculum Increased; No activities will be implemented for this IR, as per MTE recommendation. LLR3.1: Increased Participation of medical school faculty in SOLUCION TB Events; No activities will be implemented for this LLR, as per MTE recommendation. LLR3.2: Number of Students Participating in Hands-On DOTS Field Work Increased; No activities will be implemented for this LLR, as per MTE recommendation. 5 Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ Attachment B Evaluation Team Members Kathleen Moser, MD External Evaluation Leader Jose Luis Burgos, MD External Evaluation Assistant Elizabeth Ferreira, MD Director, National TB Program – México City Molly Lindner, MPH USAID México Mission Liaison – México City Alberto Delgado, MD Director, Preventive Medicine – Baja California Guadalupe Felix, MD TB Program Director – Baja California Ofelia Morales, MD TB Program Chief – Mexicali, BC Paris Cerecer, MD TB Program Chief – Tijuana, BC Janine Schooley, MPH Vice President for Technical Services and Program Development – PCI San Diego Clara Eder, EdD, MPH, RD Senior Technical Officer for M&E – PCI San Diego Blanca Lomeli, MD Director SOLUCION TB – PCI San Diego Enrique Gomez, MD SOLUCION TB Coordinator – Mexicali, BC Jesus Madrigal, MD SOLUCION TB Coordinator – Tijuana, BC Eva Mendoza, MD SOLUCION TB Community Coordinator – Tijuana, BC Liliana Andrade, MD MSC Consultant – Tijuana, BC Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ Attachment C Evaluation Assessment Methodology The evaluation consisted of qualitative and quantitative elements. In order to develop an overall assessment of the project, both key informant interviews and focus groups were conducted. In addition, available data, as well as project databases and forms were reviewed. 1) Focus groups: The evaluators asked for the project staff to convene two focus groups in each city; one with promotores and the other with current patients. Lunch and transportation to the event were provided as incentives for patient participation. Prepared open-ended questions were used as prompts for discussion when necessary. The groups were held in the MSC office in Tijuana and the Universidad Autonoma de Baja California in Mexicali. Project supervisors were not present during the promotore sessions. In Tijuana, patients from drug rehabilitation centers were among the patients, but the presence of center attendants may have limited comment from those few patients. Groups were 1 ½ - 2 hours in length. 2) Key Informant Interviews: In order to gain a comprehensive overview of the project, a targeted group of knowledgeable project staff were interviewed. These interviews were based largely on open-ended questions, but included directed inquiry when specific clarification was needed. Informants included staff at all levels, from front line to management. Interviews were 1-2 hours in length. Some interviews were conducted in person, but due to scheduling and travel limitations, others were over the phone. 3) Site Visits: Visits were made to one health center in Tijuana, as well as to the jurisdictional office. Two health centers and the jurisdictional office were visited in Mexicali. At the health centers, the attending DOT staff person and the medical lead were interviewed. Charts and forms were reviewed at all centers and patient interaction was observed in one center. A visit to the state health office provided an opportunity to interview the State TB Director and to see the SOLUCION and EpiTB databases. 4) Data Review: Aggregate outcome data was received from the State TB Program for the first two project semesters. A printout of the first semester adherence data from the SOLUCION database was also provided. A convenience sample of project forms were reviewed for the type and completeness of data collected; supervision checklist of DOT workers and promotores, the national TB tarjeton, and the RSEM. Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ Attachment D Persons Interviewed Individuals Guadalupe Felix, MD TB Program Director – Baja California Ofelia Morales, MD TB Program Chief – Mexicali, BC Paris Cerecer, MD TB Program Chief – Tijuana, BC Blanca Lomeli, MD Director SOLUCION TB – PCI San Diego Enrique Gomez, MD SOLUCION TB Coordinator – Mexicali, BC Jesus Madrigal, MD SOLUCION TB Coordinator – Tijuana, BC Diana Carmen Herrera SOLUCION TB Liaison – Tijuana, BC Remedios Lozada AIDS Program Chief – Tijuana, BC Lorenzo Alvarado Director, ISESALUD – Tijuana, BC Antonio Granillo Medical Director, Las Memorias Focus Groups DOT staff Tijuana – 12 staff Mexicali – 14 staff Patient Focus Group Tijuana – 20 patients Mexicali – 14 patients Health Center Visits La Mesa – Tijuana Hidalgo – Mexicali Vicente Guerrero – Mexicali Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ Attachment E 1 Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 2 Project Concern International (PCI) Mexico SOLUCION TB MTE November 30, 2006 USAID Coop. Agreement No. GHS-A-00-04-00013-00 ____________________________________________________________________________________________________________________________________________________________ 3