STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 1 Prepared by: DISCLAIMER This is an external evaluation. The views expressed in this document are the authors' and do not necessarily reflect the views of the United States Agency for International Development or the United States Government APRIL 2014 This publication was produced for review by the United States Agency for International Development. It was prepared by Democracy International, Inc. USAID/TANZANIA STRENGTHENING TRANSPARENCY, ACCOUNTABILITY, RESPONSIBILITY (STAR) PROGRAM EVALUATION FINAL REPORT STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 2 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Submitted to: USAID/Tanzania Contractor: Democracy International, Inc. 7600 Wisconsin Ave. Suite 1010 Bethesda, MD 20814 Tel: 301-961-1660 www.democracyinternational.com STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 3 STAR PROGRAM PERFOR￾MANCE EVALUATION FINAL REPORT APRIL 2014 STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 4 ACRONYMS AFNET Anti-FGM Network BONGA Building Organizational Networks for Good Governance and Advocacy CELG Centre for Environmental law and Governance CETA Civic Education Teachers Association CCT Christian Council of Tanzania CSOs Civil Society Organization DI Democracy International, Inc. DED District Executive Director DFID Department for International Development DRG Democracy, Rights, and Governance GIZ Gesellschaft für Internationale Zusammenarbeit GoT Government of Tanzania IoAs Institutions of Accountability LGA Local Government Authorities M&E Monitoring and Evaluation PETS Public Expenditure Tracking PMO-RALG Office of Regional Administration and Local Government SAHRINGO South Africa Human Rights NGO Network SATC Supporting Accountability to Tanzania Citizens SOW Statement of Work TACOSODE Tanzania Council for Social Development TAPP Tanzania Advocacy Partnership Program UNGO Union of Non-Governmental Organizations USAID United States Agency for International Development ZCGF Zanzibar Current Generation Forum STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 5 OVERVIEW AND PURPOSE At the request of the United States Agency for International Development (USAID) in Tanzania, Democracy International, Inc. (DI) conducted a post-program performance evaluation of the Strengthening, Transparency, Accountability, and Responsibility (STAR) program implemented by Pact between September 2009 and January 2012. STAR was the last of three programs im￾plemented by Pact to support Tanzanian advocacy organizations to demand transparency and accountability in public allocation systems and government institutions. Pact implemented the first program, Tanzania Advocacy Partnership Program (TAPP), from 2002 to 2005, and the second program, Building Organizational Networks for Good Governance and Advocacy (BONGA), from 2006 to 2008. DI’s evaluation team assessed STAR in the context of these two previous programs. As outlined in the Statement of Work (SOW), the evaluation had two purposes: (1) provide a post-program independent performance evaluation; and (2) use lessons learned from those programs in inform USAID/Tanzania’s current and future programming. The DI team assessed how the program’s performance has influenced Tanzanian civil society as well as Pact’s partners and other stakeholders. The evaluation provides a detailed examination of the progress STAR has achieved toward meeting their established five expected results: 1. Build the capacity of the citizens in selected districts to enable them to participate in the development plans, monitoring of local budgets and service delivery, and to strengthen their voice in demanding a transparent, accountable and responsive government; 2. Enhance capacity of civil society organizations (CSOs) and citizen groups to engage in participatory policy making and planning process; 3. Enhance the capacity of CSOs and citizen groups in monitoring and evaluation of public services and goods; 4. Increase the ability of citizen groups to understand and demand their rights for infor￾mation by working with local government authorities to ensure that lower local gov￾ernment representatives make information publically available; and 5. Strengthen the technical capacity of CSOs to undertake advocacy activities that grow out of the public expenditure tracking (PETS) process and capacity building initiatives. To the greatest extent possible, the evaluation sought to analyze where the program was suc￾cessful and where it was less successful, identify the factors affecting program implementation, identify obstacles to program success, and examine the effect of the program on USAID/Tanzania’s strategic objectives. The team also formulated lessons learned regarding the role of civil society organizations in increasing governmental accountability and transparency, program management and implementation, including whether resources were sufficient for pro￾gram objectives, identifying unanticipated factors that were not envisaged in the original design, how the program responded and adapted to these factors, and the effect of these factors on the program’s goals, activities, and results. The evaluation team interviewed 14 of the 21 Pact partners under STAR, including all of the organizations in Dar es Salaam, Zanzibar, Morogoro and Dodoma. The three-week time period STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 6 of the fieldwork did not allow the team to travel to areas far from Dar es Salaam. Moreover, as is typical of such evaluations, the reports of activities by the Pact partners were not very differ￾ent from one another, so the benefits of one or two additional meetings with one or two Pact partners in outlying areas were not cost-effective given the significant amount of time it would have taken to reach them. Unfortunately, the team was able to meet only half a dozen PETS committees, which were hard to contact, and only a few local government officials. As antici￾pated in the Evaluation and Design Methodology, memories had faded between the end of the STAR program in January 2012 and the October/November 2013 field trip. After the funding ceased, the Pact partners had moved to other projects and many of their staff had moved to other jobs and NGOs, many PETS committees had atrophied or disappeared altogether (only between 10 to 20 percent of the total PETS committees supported by STAR are still in opera￾tion), and many government officials had moved. Moreover, almost all of the CSOs ended their connections with, and certainly their mentorship of, the PETS committees once their STAR funding ended. Those that continued contact were typically the small, locally-based CSOs, such as Wopata (Women and Poverty Alleviation in Tanzania).1 The Mission has designed and is currently beginning implementation of a new program, Sup￾porting Accountability to Tanzania Citizens (SATC), which has similarities to STAR. Under SATC, however, there will be direct grants by the Mission to six CSOs. Unlike STAR, however, there will also be grants to four Government of Tanzania (GoT) “Institutions of Accountability” (IoAs). A contract for capacity building with Chemonics is already in place, but Chemonics will not act as a grantor like Pact did. The six CSO grantees will be working with subgrantees on public expenditure tracking, just like they did under STAR. SATC will be a four- to six-year program, however, and will be focused in five regions, not throughout Tanzania. Still, there are important lessons from STAR which should inform SATC during the implementation process. 1 Wapota, for example, has a very small office in a string of five to six stalls along the main rural road in the Kilosa district near Morogoro. It continues its work on other projects and keeps touch with “its” PETS committees by phone and in person while it is working on other projects. But Kilosa district is small, Wapota works locally, and so the PETS committees it mentored are in its “neighborhood.” The larger, more urban, more distant CSOs had little or no contact once STAR funding ended because they moved on to other donor-funded projects which took their time and attention and through which they did not necessarily naturally encounter the PETS committees or their members. That said, these larger CSOs typically did not make any systematic effort to keep in touch even by phone. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 7 BACKGROUND In 2002, USAID/Tanzania ‘s Democracy and Governance office began what turned out to be a 10-year initiative to indirectly fund CSOs to track government budgets and expenditures to en￾sure that appropriated funds were actually delivered and ultimately used for their intended pur￾poses. Funding constraints, however, led USAID to divide the program into three sequential and technically independent programs implemented by Pact: TAPP, implemented from 2002 to 2005; BONGA, implemented from 2006 to 2008; and STAR, implemented from 2009 to Janu￾ary 1, 2012.2 STAR aimed to empower Tanzanian citizens at community level and engage with CSOs at the national level to participate and demand transparency and accountability in the devolved sys￾tems for allocation and use of public resources, and in planning and delivery of public services for socio-economic development and poverty reduction. Under STAR, Pact trained and funded 21 local CSOs that worked in 40 districts across Tanzania. Also referred to as Pact partners, these CSOs in turn trained and funded local PETS committees at the village, ward, and/or dis￾trict levels, who were the ultimate implementers and beneficiaries of the program. During STAR, Pact partnered with 21 CSOs: three in Arusha, seven in Dar es Salaam, two in Dodoma, one in Kigoma-Kasulu, one in Mbeya-Ileje, one in Mbeya-Ilomba, one in Morogoro, one in Mwanza, one in the Mwanza-Ukerewe islands, and one in Shinyanga, and two in Zanzibar. Each Pact partner created or supported between two and six PETS committees, most of which were spread out in different parts of the country and located far from the Pact partner. Each PETS committee chose to track local budget and expenditures in one or two sectors, usually health, education, agriculture, or the environment. Most committees chose—or were encouraged to choose—to track sectors similar to those in which its grantor Pact partner specialized. Pact trained its CSO partners in several areas, including: budget monitoring and tracking, advo￾cacy, organizational strengthening, project and grants management, and financial management. These were the same skills that the Pact partner in turn trained its PETS committee partners although the emphasis with the PETS committees was on tracking and advocacy. The PETS committees were designed to demand their rights to monitor public expenditures in their cho￾sen sector, to confront government officials about discrepancies between what had been budg￾eted and what had actually been delivered and expended, to hold the public officials accountable for the funds, and to pursue remedies for misallocations, corruption, and mismanagement. One Pact partner encapsulated this, saying, “PETS means following the money from the central gov￾ernment through the local governments to the end users and beneficiaries.” The authority under which PETS committees tracked, monitored, reported, and held public of￾ficials accountable—at least during STAR—derived from the prime minister and the Office of Regional Administration and Local Government (PMO-RALG). In 2007, PMO-RALG issued guidance to public officials and the public in general (see “Prime Minister’s reflected authority,” below) that reinforced the bureaucratic right (established under law), and, indirectly, the politi- 2 BONGA was funded by the Millennium Challenge Corporation (MCC) as part of the USAID￾implemented threshold program for Tanzania that had failed the MCC’s corruption criterion to be eligi￾ble for an MCC compact. Consequently, BONGA had a more pronounced corruption component, alt￾hough the misallocation and diversion of funds, including for personal benefit, was a primary dimension of all three programs. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 8 cal right of the public, for PETS activities and required public officials to cooperate with them in doing so. In the six years since the guidance was issued, CSOs have proposed modifications to strengthen their legal rights but none have yet been promulgated and the original guidance is still in effect. One purpose of STAR was to give the guidance concrete effect by supporting the PETS committees to demand their rights and provide them with the capacity to realize those rights. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 9 EVALUATION METHODOLOGY At the request of USAID/Tanzania, Democracy International conducted a performance evalua￾tion of STAR in October–November 2013. The DI evaluation team consisted of Gerald Hyman as the Evaluation Team Leader, Vivien Marles as the Evaluation Specialist, Rose Aiko as the Lo￾cal Technical Advisor, and Dorica Andrew as the Local Logistics Coordinator. The Team Lead￾er, the Technical Advisor, and the Local Logistics Coordinator spent three weeks in country, while the Evaluation Specialist joined the team in country for the first week of fieldwork. Two USAID officials from Washington, DC, jointed the team for the first two weeks of fieldwork as part of their training and professional augmentation opportunities. According to the SOW, found below in Annex A, the purpose of the evaluation was to: (1) provide a post-program independent performance evaluation; and (2) use lessons learned from those programs to inform USAID/Tanzania’s current and future programming. The DI team as￾sessed how the program’s performance influenced Tanzanian civil society as well as Pact’s part￾ners and other stakeholders. The evaluation responded to the following questions:  To what extent was/is STAR’s capacity building sustainable? How might current or new USAID programs use, leverage, or build upon that capacity?  What tools and mechanisms—PETS committees, citizen report cards, community score cards, PIMA cards, notice boards—proved most effective in helping ordinary citizens monitor public budgetary decisions and service delivery, under what circumstances? Was this appropriately measured by the program, and if not, what are methods to measure such a result?  To what extent did the (1) CSO capacity building, (2) subgrants, and (3) Strengthening Networking Information Sharing among Tanzania CSOs each contribute to the achieve￾ment of the program objective? The team also examined target communities’ awareness and attitudes toward the STAR pro￾gram, Public Expenditure Tracking, and oversight of local government. The evaluation analyzed where the program was most successful and where it was less successful, examined the factors affecting program implementation, identified obstacles to program success, and analyzed the effect of the program on USAID/Tanzania’s strategic objectives. Findings are structured around two key areas: (1) Impact and (2) Sustainability. The team formulated lessons learned regarding the role of civil society organizations in increasing governmental accountability and transparency program management and implementation, including whether resources were suf￾ficient for program objectives. Finally, the team also identified unanticipated factors that were not envisaged in the original design, examined how the program responded and adapted to these factors, and the effect of these factors on the program’s goals, activities, and results. As outlined in the DI’s Evaluation Design and Methodology report submitted to the Mission, the team relied on a comprehensive desk review and over 100 key informant interviews in Dar es Salaam, Dodoma, Morogoro, and Zanzibar. In addition to Dar es Salaam, the team visited Do￾doma, Morogoro, and Zanzibar to assess the program’s regional impacts. The team conducted interviews with: Mission DRG officers, several current and former Pact staff, 14 of the 21 Pact partners, several local PETS committees, approximately two dozen current Government of STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 10 Tanzania national and subnational officials, especially the PMO-RALG; several former GoT offi￾cials; major donors that had implemented similar programs, including Die Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) and Irish Aid; other implementers, such as the United Kingdom’s Department for International Development’s (DFID) implementer KPMG and the Dutch Ministry of Development Cooperation’s implementer SNV; and the Mission’s monitoring and evaluation (M&E) contractor, the Mitchell Group. The list of interviewees can be found be￾low in Annex B. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 11 METHODOLOGICAL LIMITATIONS Like all post-hoc project evaluations, the STAR evaluation is subject to several limitations. The evaluation team, however, has taken active steps to minimize bias and generate systematic, veri￾fiable, credible information. One of the greatest challenges the team faced is the fact that the program ended nearly two years ago, which made data collection challenging. In fact, none of the Pact staff that worked on STAR was still with Pact at the time of the evaluation, except for one staff member who worked on STAR at the very end of the program in a non-substantive capacity. The team, however, did locate two Tanzanian members of Pact’s STAR staff, now working for other organizations, who agreed to be interviewed and provided valuable insights into the difficulties Pact encountered (e.g., the gaps in funding between TAPP, BONGA, and STAR) and how they attempted to deal with them. Similarly, as the evaluation team asked stakeholders to recall events for a program that began over four years ago, the memories of some key informants may have faded. To mitigate the occurrence of recall bias, the team uti￾lized multiple data points to triangulate and substantiate participants’ responses. It also proved exceedingly difficult to contact and interview PETS committees, as they could not be identified without the help of Pact partners. Many partners, however, had lost track of these committees, and since most members were ordinary citizens, committee members were hard to identify, contact, and interview. As such, the team was only able to conduct interviews with four or five PETS committees, out of a total of 70–80 committees nationwide. Relevant gov￾ernment officials were even more difficult to reach, especially those who were no longer work￾ing in the same area. As such, due to the nonrandom nature of the sampling, the evaluation was subject to selection bias, with findings reflecting the feedback of stakeholders who may not be fully representative. The evaluation timeline and logistical considerations also constitute a key limitation. The timeframe of this evaluation required a narrow, focused approach and precluded more rigor￾ous, quantitative evaluation techniques, such as public opinion surveys. As noted above, CSOs and government officials that received Pact assistance are located throughout Tanzania. Except for two in Zanzibar, most partners were within a seven- or eight-hour drive from Dar es Sa￾laam, where 40 percent of Pact partners are located. Given the size of the team, the timeframe for the evaluation, and budget considerations, the evaluation team was not able to visit all of these locations, thereby leaving the more rural and distant Pact partners unrepresented. While interviewing every project stakeholder may be impractical and undesirable, the evaluation team worked to carefully select and cull the interviewee list to include individuals who not only could provide substantive feedback but also provide them with a balanced, unbiased assessment of project performance. As noted below, Pact and its staff had general ideas about what was meant by “strengthening,” “helping,” or “building” the capacity of its partners but did not have concrete or explicit opera￾tional definitions of these concepts, let alone metrics. Thus, in addition to the apparent absence of quantitative indicators of progress in these goals, the DI team explored as explicitly as possi￾ble with current Pact staff (who are currently working on other programs) and former Pact staff that worked on STAR what Pact intended by way of strengthening, helping, and building, how successful those programs were, what limitations they encountered, and what they would do differently. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 12 Despite these challenges, however, their cumulative effect was not as great as may have been predicted. The experiences of the various Pact partners and the PETS committees did not vary much. By the end of the interviews, for instance, the team was hearing variations on the same themes and findings converged. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 13 KEY FINDINGS Below, we present key findings in two key areas: (1) Impact, and (2) Sustainability. IMPACT TRANSPARENCY AND ACCOUNTABILITY Given the sporadic character of the three-phase program, the design and the duration of the program, the distance between the Pact partners and their PETS committee mentees, and a host of other reasons, STAR’s achievements are remarkable. Most Pact partners the team in￾terviewed provided detailed anecdotes of the successes of its PETS committees in identifying misallocations or misappropriations of funds, bringing them to the attention of officials, and suc￾cessfully achieving a reversal or restoration of funds or led to a suspension, removal, or reas￾signment of culpable officials (see “Causal relationship,” below). Examples of these successes include:  According to the Centre for Environmental Law and Governance (CELG), PETS com￾mittee members reported a case of unlawful land acquisition by a potential Jatropha Farming Investor who was embroiled in land conflict with villagers in Kilwa. CELG inter￾vened to assist villagers to recover their land, by showing that the appropriate proce￾dures for allocating village land to investors, including consultations with villagers, were not followed. Failing to prove that the land was rightfully acquired, the investor aban￾doned the planned project and the land was returned to villagers.  Attempts to share information, and transparency and accountability have increased. “The PETS committees sometimes turn themselves into auditors, not just monitors. They should serve both sides: government should present the accounts and villagers should be able to monitor their use and whether the quality and use is correct. People are not fools. They should be able to say whether value has between received,” one re￾spondent told the team.  A representative from the Anti-FGM Network (AFNET) reported that STAR has helped to increase transparency. The respondent told the team that before PETS activities, offi￾cials were unwilling to provide information, but now they are. A Bahi councilor similarly noted that “things are changing. Before the exercise, it was hard to know how re￾sources were allocated, but now it is displayed on the village notice board.”  Several respondents reported diminished resistance to PETS and a cultural shift within government toward greater public access and accountability to the public. “In the begin￾ning,” reported a representative from the Union of Non-Government Organizations (UNGO), “government officials were not open and did things the way they wanted, but because of PETS there is now public awareness and the government officials are more responsible to the public. After a while, government officials followed up on PETS re￾ports or complaints, including the DED [the District Executive Director appointed by STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 14 the central government], who had to act if only because the information was public and document and distributed.”  As a representative from UNGO reported, “When PETS began, LGA [local government authority] officials viewed PETS committees as inspectors, but at the end they under￾stood that the PETS committees were there to help. As a result, there were a lot of changes as a result of PETS; for example, government budgets were posted on notice boards.” UNGO also reported that PETS committees in other areas formed unrelated to UNGO when they saw what was happening.  In general, respondents from the Lawyers Environmental Action Team (LEAT) claimed that “government officials will behave differently if they know they are accountable and can be punished,” echoing both government guidance and the underlying hypothesis of the STAR program. Another representative from the Tanzanian Council for Social De￾velopment (TACOSODE) said, “The DEDS came to know that someone is watching. The documents used to be confidential but now they are public. Originally, they said ‘you are not competent,’ but we gave them competencies.”  In more subtle but perhaps more enduring ways, said an UNGO representative, “rela￾tions between officials and parents have changed because of PETS. The public is no long￾er scared. It now asks about funds received and used. PETS and social awareness cam￾paigns have brought awareness and understanding of their rights; they are no longer sleeping.” In an interview in Zanzibar, another respondent told the team, “villagers should have recourse for leaking roofs or cracked walls through village meetings and then to their political representatives [councilors].”  Because the officials know that people are watching, respondents reported less spoilage and deflection of funds. A PETS committee in Mtwara, for instance, discovered that some medications had been stolen. After reporting the theft to government officials, some hospital staff members were fired. Similarly, a village official was removed because of deflection of funds identified by a PETS committee. In another case, a village official shifted funds from program uses to his own per diem, but the PETS committee stopped it.  AFNET reported that a PETS committee focusing on water found that “funds had been diverted and chased out the leaders.” Another PETS committee focusing on healthcare did the same. Based on their legal “right to know” about finances, PETS committees questioned public authorities, insisted on having village council meetings called, and de￾manded financial reports.  In addition to greater awareness and greater access to public budgets and greater public awareness, UNGO reported direct effects on government personnel as a result of its PETS activities: “One person was trained, then went back to his village and found that village officials did not report back to villagers about spending and resources. He orga￾nized the PETS work in the village and they found many problems in village administra￾tion. They decided to organize and insist on public information. After two years, things have changed. Now there are regular assembly meetings twice a year, including reports about income and spending. Similarly a village health center was supposed to get materi- STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 15 al for a health care facility but it didn’t come. The government and the villagers were supposed to cooperate to build the center, so villagers cooperated, took initiative and brought stones, water, and sand and the government provided roofing and cement.” Although these examples are illustrations and anecdotes of funds misappropriation, they are too widespread and independent of one another to be simply dismissed as accidental, isolated, or idiosyncratic. Still, the evaluation team could not establish a causal link between PETS com￾mittee reports and the transfers, which are routine in the normal course, or whether the trans￾fers were promotions or the results of requests. A deeper and more systematic review at the village- and ward-level is required to conclusively establish cause and effect. Similarly, without adequate quantitative indicators to measure whether transparency of budgetary transfers in￾creased and public service delivery actually improved as a result of PETS committees’ activities, it is difficult to draw any conclusive results beyond the anecdotal reports described above. INCREASED AWARENESS There seems to be an increased awareness about the budget in communities that have or have had active PETS committees. “Before PETS, officials were unwilling to provide information, but now they are,” said a representative from AFNET. The response from a PETS committee mem￾ber in Bahi was similar but more qualified. “Before, it was hard to know how resources were allocated, but now the budget is displayed on the village notice board. There is now increased awareness in all three villages where we worked, but no sanctions or accountability,” a PETS committee secretary in Bahi told the team. “Accountability,” he said, “does not end with awareness.” Nevertheless, he went on to say that “because the officials know that people are watching, spoilage and deflection of funds are less. Information travels; evil shouts.” Thus awareness, even without direct incentives, does have some impact. “Although PETS committees are not auditors, and although they created some tension among local government officials, and although they were not originally accepted, and although there was no open government part￾nership, now information is shared, government officials recognize that they must share public information and there is more of a partnership,” the Bahi committee secretary said. “There was no openness in those days. Government officials thought they owned the information.” Absent a public opinion survey, however, it is difficult to definitively ascertain whether public awareness about the budget, public expenditure tracking, and oversight of local government increased as a result of the STAR program. . Although Pact partners of course knew about the STAR program and the PETS committees that received assistance seemed to be acquainted with the STAR name (although not necessarily with details of the program), the communities themselves would have had no exposure to STAR itself as there was no obvious branding of STAR. However, more generically, public expenditure tracking and even community monitoring (and perhaps oversight) had much greater public currency. The Government of Tanzania itself created public awareness around both these general concepts and around PETS committees in particular. As indicated below, STAR was not the only program and USAID was not the only donor to support PETS. PMO-RALG was a constant force supporting PETS, for example. The prime minister made speeches about PETS, including a reference to one of the Pact partners in one of them Of course speeches by themselves, even those of the prime minister, have limited authority, especially in rural areas. Still, those speeches, albeit not full directives, provided a kind of informal instruction to government officials. They announce policy, presumably. In light of actual governmental directives preceding them, the speeches do provide additional indirect au- STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 16 thority for PETS activities to PMO-RALG, to the CSOs, and to the PETS committees when they encounter bureaucratic resistance. RELATIVE EFFECTIVENESS OF VARIOUS MONITORING TOOLS AND PRO￾GRAM COMPONENTS It is difficult to say with any certainty what tools or mechanisms proved most effective in help￾ing ordinary citizens monitor public decisions and service delivery. Given the absence of quanti￾tative metrics, the program did not appropriately measure the relative effectiveness of these tools. Our strong impression, however, is that most ordinary citizens would naturally want more and better services and budgetary support but would have little, if any, knowledge of PI￾MA cards, citizen report cards, or community score cards. The team did, however, observe public notice boards in some communities that were erected with Pact funding under STAR. (Photographs of some of these notice boards can be found below in Annex C.) The team only encountered these notice boards sporadically and many were in bad physical shape (e.g., lean￾ing, with broken glass, or in relatively untraveled places) with weathered, years-old budgets. As pointed out in our interview at Twaweza, it remains unclear whether ordinary citizens could make sense of these budgets or have an interest in them. Even assuming that ordinary citizens would have known what monitoring tools existed and had an opinion about their relative effec￾tiveness, a methodologically sound public opinion survey would have been required to answer with confidence the rank order of their effectiveness. For similar reasons, it was difficult to concretely determine to what extent CSO capacity build￾ing, subgrants, and Strengthening Networking Information Sharing among Tanzanian CSOs each contributed to the achievement of program objectives. Because so few PETS committees were reached, it is difficult for the team to extrapolate from these findings and determine to what extent these program elements contributed to STAR objectives. Pact partners, on the other hand, were virtually unanimous in expressing their appreciation of Pact’s training and support. Few, if any, partners mentioned network sharing or discussed its effectiveness in achieving pro￾gram objectives and no partners mentioned a PETS secretariat proposed by Pact as part of network information sharing. Subgrants augmented by training were at the heart of the pro￾gram from the perspective of Pact partners and PETS committees. However, as discussed be￾low, once these grants ended, so too did the PETS activities of most of the Pact partners, who also, in the overwhelming majority of cases, lost touch with the PETS committees they were mentoring. Consequently, while the subgrants were almost certainly the most effective of the three program element above, without sustainability their effects were diminished. TRAINING Pact provided quite a bit of training to its partners, primarily centered around grant solicita￾tions, grant management, organizational capacity, and management. Many of the Pact partners appreciated the training. “We got so many good trainings,” said the executive secretary of Women and Poverty Alleviation in Tanzania, a sentiment echoed by many other partners. In theory, but less so in practice, the Pact partners were then supposed to train the PETS commit￾tees, making Pact a trainer-of-trainers. As noted below under “Sustainability,” however, due the short timeframe of the grants (e.g., one year for Wopata, and four months for its PETS commit￾tee) and the short amount of time available for training, gains from training were often dimin￾ished or lost just one or two years later. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 17 DURATION Although the program lasted over six years, each phase was independent of the others and only lasted about two years—much too short a timeframe to yield significant, let alone sustainable, impacts. Funding for each phase came from different sources, especially the MCC-funded BONGO, and, as such, each phase had a somewhat different emphasis. At least as important for implementation purposes, USAID, Pact, Pact partners, and the PETS committees did not have any assurance that an additional phase would be funded and that their work therefore could continue. As each phase was 75 percent completed, this uncertainty naturally prompted Pact and CSO staff to search for future projects and employment opportunities. As staff attention began to drift, it can be argued that the effective period of the grant was really only 18 months, a period that was even shorter given the start-up or start-over nature of programming defined by the “lumpy” funding. The effective duration of PETS committee grants seems to have been even shorter, between three and six months. During their grant period, Pact partners were given training and other technical assistance on grants management, organizational strengthening, and other topics. While this training may have had an effect on the Pact partners, it took time and other re￾sources which might have otherwise been provided to the PETS committees, the ultimate bene￾ficiaries of the program. By the time Pact partners were finished with training, had their grants in place, and had created, identified, and nurtured the PETS committees, their two-year grants were months away from ending. Considering the months-long duration of the PETS committee grants, it is surprising and remarkable, and arguably fortuitous, that the PETS committees had any impact at all, especially given the levels of education of the villagers and the level of organi￾zation and experience of the PETS committees, many of which did not exist prior to their es￾tablishment by the Pact partners. GEOGRAPHY AND TRANSACTION COSTS The distance between Pact partners and their associated PETS committees was much too far to establish a close mentoring relationship. Pact partners, in consultation with Pact and (apparently indirectly with USAID), chose local partners in various parts of the country distant from them￾selves and from one another. For example, AFNET in Dodoma mentored a PETS committee in Hai district in Kilimanjaro, and a second in Simanjiro (Manyara), over 500 and 300 kilometers away, respectively, both of which would entail multiple bus rides on difficult roads and erratic bus schedules. Although AFNET had member affiliates in these areas, it is not clear this was the best approach. Indeed, in its proposal, each partner put forward a list of districts where they would be interested to work and many partners proposed districts where they had a communi￾ty-based civil society network presence. A total of 40 districts were finally selected for the PETS undertaking. No doubt USAID and the Millennium Challenge Corporation (which funded the BONGA phase of STAR) urged national coverage for STAR. However, Pact did not select the PETS committees on its own. The CSOs themselves proposed to Pact the PETS committees and lo￾cations for their work, in many cases because of their pre-existing relations with those groups an in those locations (for example through the CSOs’ local branches). The question then is why these relations did not endure once STAR funding ended. To the extent USAID provides this kind of funding, whether in STAR, SATC or any other program, and to the extent it retains any authority for selection of local beneficiaries, one criterion should be the likelihood of sustained STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 18 contact and mentorship once USAID funding ends. Certainly the wide geographic dispersion of the beneficiaries and their distance from the CSO’s headquarters made sustained contact more difficult and less likely unless the CSO had strong branches in close proximity to the PETS committees. Connections could be maintained by cellphone and cellphone connections are broadly but not universally available throughout the country. However, it is difficult to keep up momentum and mentorship relying solely on distant voices over cellphones phone even when the parties try earnestly to do so. The existence of truly vibrant local branches of the CSO would make that direct contact possible albeit not by the headquarters staff. Pact could have achieved national coverage by identifying geographically dispersed CSO part￾ners and asking them to work locally (for example, if it had vibrant branches), an approach which arguably might have been more sustainable after funding ended. Put differently, USAID and Pact could have achieved a national footprint by dispersing the Pact partners geographically and having them mentor local PETS committees or at least mentor several PETS committees in one area. Several Pact partners reached the same conclusion. “[The committees] would have achieved more if [they were] implemented and run by local organizations in the area,” stated a member of the South Africa Human Rights NGO Network (SAHRINGO). “Set up in the way it was meant that modalities of permanence were not there.” This alternative approach would have reduced the enormous travel and other transactions costs for Pact partners and allowed for much more continuity of relationships once the grants ended. As it was, contact between most Pact partners and their mentee PETS committees understandably atrophied dramatically once funding ended. ORGANIZATIONAL DEPTH Judging primarily by their own accounts, Pact partners created most PETS committees from scratch under STAR. Although the committee members often knew one another, particularly at the village level, they had not necessarily worked together before. In short, PETS committees were not organic CSOs with deep, historic roots in the community, and thus would not have likely existed without STAR. As such, when the funding ended, the committees did as well. This context affected their coherence, their operations under STAR, and their post-STAR sustaina￾bility. Identifying existing organizations and then supporting them to conduct public expenditure tracking might have been a more sustainable approach, rather than creating organizations anew. However, since SATC will have four (and perhaps six) uninterrupted years of funding, this will make it more feasible to create new organizations and see them through their “birthing pains.” EMPHASIS ON THE LOCAL LEVEL While the team’s interviews with Pact partners provided a good sense of what worked well and what did not, program effects were ultimately produced by PETS committees—not Pact or Pact partners—who engaged in tracing, reporting, and follow-on at the village, ward, and, less often, district levels. In fact, PETS committees seemed to have been more effective and sustainable at the village level than at the district level. It appears public funding was mostly spent at the village level. Moreover, since citizens at the village level are in much closer proximity to government officials and each other, interactions do not require significant travel and funding. As such, not only is neglect by government officials more apparent, citizens can also more easily monitor and track public expenditures and activities. Given the number of PETS committees under STAR, their dispersal, and their “half life,” a much longer, fuller effort would have been required to STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 19 trace impact at the PETS committee level, requiring a greater investment of time and resources. As such, without systematically tracing these efforts (even if a large-enough sample exists to be statistically significant), the true effects and impacts of the STAR program can only be ascer￾tained indirectly. Moreover, it was not possible to conclusively verify anecdotes of program achievements, let alone to trace causal linkages or construct a cost-benefit analysis for the pro￾gram as a whole. Still, as noted, the illustrative cases were sufficiently numerous and sufficiently dispersed among independent Pact partners and PETS activities to be convincing. The team acknowledges, however, that Pact partners may have exaggerated reports of their own impact as this would serve their own interest for further funding. The true impacts of STAR, therefore, may be more qualified in terms of impact and sustainability. As already noted, a definitive con￾clusion would require a more systematic evaluation of the work of the PETS committees them￾selves. PRIME MINISTER’S REFLECTED AUTHORITY Perhaps the most forceful, continuous, and visible form of governmental and political support for PETS and PETS committees comes from the Office of Regional Administration and Local Government. In fact, the authority PETS committees had received concrete backing from the 2007 guidance promulgated by the Prime Minister through this office. The guidance aimed spe￾cifically at LGAs, states that: “PETS are a CSO tool and are a strong, participatory monitor of the integrity, transpar￾ency, and accountability of financial management systems and decision making. They track whether the money allegedly for public services actually gets to where it has been planned and promised. The government wants PETS to grow, based on quality process￾es, reporting, and follow up. This Guidance has been prepared to clarify PETS for LGAs and what it means for the vertical system, from PMO-RALG to service delivery points. PETS can be of many depths, hence this guidance is deliberately generic, to set some pa￾rameters. Experience will tell us how it will be improved in the future.” (p.1) “The supply side is the obligation of LGAs, adhering to the principles of good govern￾ance. LGAs should ‘explain publicly, fully, and fairly how they are conducting responsibil￾ities that affect the public.’ The demand side comes from civil society (communities and organizations). It is the ability of citizens to hold the local authority to account. Elections are only every five years, therefore there has to be other ways. (sic) These are through free media, meetings, participatory processes, and now through PETS.” (pp. 3-4) “Experiences in Tanzania and elsewhere have shown that PETS have the potential to en￾hance downward accountability at all levels of government, provided that they are con￾ducted effectively, and the results are publicly discussed and actions taken to correct mistakes and to strengthen systems. ...[B]etter accountability and governance leads to better development. (p. 6, emphasis added) “... ‘LGA’ also covers both staff and elected officials of the LGA or sub-level….” (p. 13) “PETS are a tool for providing information to citizens about how government money is used. PETS compare budgetary allocations to actual spending. This involves ‘following the money’ to where it is spend, comparing budgetary allocations with records of trans￾fers and receipts at each level of government. PETS will show how much of the funds in￾tended for service providers actually reach the intended beneficiaries. It will also indi- STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 20 cate at what level and leakage of diversion take place and will provide comparable statis￾tics for administrative divisions….” (p.4) “Article 18 [of the Constitution] goes on to foster transparency and accountability through the right to information: (18(1) ‘…every person has the right to…receive and impart or disseminate information, and….the right to be informed at all times….’” (p. 7) “PETS is about making financial information publicly available in a format that is under￾standable and useable. In spite of official policies of transparency, it is exceedingly diffi￾cult for local stakeholders to get financial information from local authorities; or for LGAs to get information from central government. Local officials were (and are) often unwilling or unable to provide the financial information…. It was (and is) often difficult or not possible to tell the source of funds received….” (p. 8) “PETS depend on collaboration between CSOs and Government bodies (at whatever level). It also depends on a clear understanding of the role of [civil society]; they are not auditors.” (p. 16) Although CSOs and even the government-appointed Commission on Local Government Re￾form Programme have made several recommendations for improvements, including augmented authority for civil society, the December 2007 guidance stands as is. (Pact’s claim that the PETS authorities were the result of its work on BONGA appears, therefore, to be somewhat exag￾gerated, since BONGA did not begin until 2006.) Nevertheless, PMO-RALG continues to moni￾tor budgets and back PETS support in tracking expenditures. Since so much of the local budget comes from the central government, this backing is crucial for future PETS impact. In fact, only minor revenues, such as stall rentals and parking violations, are collected at the local level, with virtually none from direct taxation, with consequently weak local ownership of revenues that come from the center. Active support of PETS among government officials is a necessary complement to the PMO￾RALG guidance. “It wasn’t an easy relation between us and the government officials,” said the Bahi PETS committee secretary. “They thought PETS was trying to uncover their mistakes and we got conflicting budget numbers from the district treasurer, district planning officer, the DED, and the DALDO [district agriculture and livestock development officer].” A Bahi counci￾lor reaffirmed this response, albeit understandable, among government officials: “There were some good aspects and some negative aspects to PETS. They came to investigate me, so gov￾ernment officials like me were reluctant to give information when they expected to be investi￾gated, but things are changing.” Similarly, according to a PETS program officer at the Christian Council of Tanzania, “Government officials have never been friendly to PETS, because it touch￾es on their position and we had resistance in Babati (Manyara) and in Liwale (in Lindi), but we had a letter of introduction from the Prime Minister’s office and we had the Prime Minister’s guidance on PETS, so they let us in.” Or, perhaps a slightly different complaint: “Initially PETS committees were perceived as interfering with LGAs,” said a journalist at UNGO. “Others used more indirect routes. AFNET got resistance from the DED in one district so it worked through the health officers to track those expenditures.” Government officials were, of course, right to be concerned if there were any discrepancy be￾tween funds appropriated for a particular purpose and the funds actually received. Indeed, even from the government’s perspective, uncovering those discrepancies and tracing them was one STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 21 of the main purposes of the PETS committees. The PMO-RALG guidance anticipated but did not resolve the fine line between monitoring and holding officials accountable on the one hand, and auditing on the other. In fact, the guidance specifically grounds the PETS committees not only in the government’s commitment to transparency through the right of citizens to know and understand the budget, but also in the government’s concern about corruption and diver￾sion of funds, which the PETS committees would help uncover and, therefore, help prosecute. This ambiguity, even ambivalence, was evident among the Pact partners and the PETS commit￾tees. “It was not the job of the PETs committees to prosecute but only to track,” said a repre￾sentative from TACOSODE, echoing part of the guidance that states they are monitors but not auditors. But, as noted above, many of the others pointed proudly—and justifiably—to the di￾verted funds restored and the officials dismissed or transferred as a result of their work. Once again, if there was support from senior officials, like the DED, the PETS committees could have sustained greater impact and diminished ambivalence surrounding its role. As stated above, a UNGO official told the team, “When PETS began, LGA officials viewed PETS committees as inspectors, but at the end they understood that the PETS committees were there to help. As a result, there were a lot of changes as a result of PETS, for example, government budgets were posted on notice boards.” However, given STAR’s lack of quantitative indicators to measure whether integrity of public spending increased in Tanzania as a result of PET committees’ activi￾ties, no conclusive results can be drawn aside from the anecdotal evidence described above. POLITICAL AND GOVERNMENTAL SUPPORT Albeit anecdotal, the support, resistance, or opposition to the PETS committees of officials with clout was instrumental to program results. Two sets of officials stand out in particular: (1) the respective centrally-appointed village or ward executive officers, the district executive director, and the respective sectoral officials for education, health, water and agriculture; and (2) the vil￾lage chairpersons and ward councilors. The former are employees appointed and assigned on a rotating basis by the central government. The latter are elected or selected by their local com￾munities. The findings of the PETS committees needed a champion with enough power and authority to achieve significant impact. For example, the Zanzibar Current Generation Forum (ZCGF) pro￾duced a report detailing construction defects in the walls, ceilings, and toilets of a local school, including photographs of the defects. The report was handed over to local officials and the Min￾istry of Education. Although ZCGF pursued it to some extent, there was no specific response, let alone remediation, to the damage or disciplinary action for the contractor or local officials who approved the construction. Without a political champion, the report seems to have lan￾guished. In contrast, the DED in Morogoro publicized and pursued the diversion of funds UN￾GO reported to him. As a result, the Office of the Auditor General investigated the issue and 11 officials, including the district treasurer and several engineers, were removed or transferred. Absent that political or governmental interest, PETS committees were without sufficient lever￾age to create consequences for their monitoring and tracking. Several respondents complained that the appointed officials were dismissive or (perhaps with good reason) defensive about the findings of the PETS committees. In those cases, the PETS committees had a difficult time gaining traction for their findings and recommendations. Not￾withstanding the constitution, the law, and the PMO-RALG guidance, all of which empowered the PETS committees to gain access to information and demand accountability, the committees’ STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 22 demands were too easily ignored with impunity. Put differently, there were many obstacles to changing the inertia of budget execution by the local government authorities, including the per￾sonal and entrenched interests by those officials to benefit by directly or indirectly misappro￾priating funds. Without sufficient support, how were villagers supposed to bring relatively pow￾erful LGAs to account? As the Bahi PETS committee chairman said, “What recourse did we have to change things—not just inform the villagers or the village councilor or executive of￾ficer? Nothing happened. We communicated with SAHRINGO but SAHRINGO never re￾sponded to us.”3 If, however, the DED had followed through (and was willing to follow the law and the established guidance instead of, for example, overlooking or even benefitting from what PMO-RALG calls “diversions” or “leakage”), she or he could have used PETS reports to hold his subordinates accountable. To be sure, villagers were not without their own resources. They could—and we were told sometimes did—refuse to deal with an errant village or ward official. The DED could not dismiss this protest and was forced to take measures against these officials. However, these were extraordinary instances and the political balance of power favors the offi￾cials at the expense of the villagers who are disproportionately weak and suffer from collective action problems and high transactional costs. Partners understood that this imbalance quite well: “We have the capacity,” one Pact partner official told the team, “but not the clout.” SYSTEMIC IMPACT ON PUBLIC EXPENDITURE TRACKING OR ACCOUNTA￾BILITY STAR did not seem to have a plan for broader systemic impact , whether on its own or as an integral part of a larger government or donor effort to create or nurture something larger among Pact’s partners and their PETS committees. The program, for instance, was not part of a larger plan forged with the Government of Tanzania or among donors. Other bilateral donors, including the Dutch Ministry of Development Cooperation, DFID, and GIZ, had similar pro￾grams, but there was no comprehensive, systematic plan to coordinate assistance. STAR pro￾gram impacts were also not designed to “roll up” into something larger than the particular PETS committees. There may have been an intended pilot program, but this was not evident in the documentation and did not happen. Through PMO-RALG guidance, the Government of Tanzania encouraged the establishment of PETS committees across the country and provided authority for their operations. In this sense, however, donors might be seen as participating in a “national program,” there were no plans by the government or the donors to propagate these committees across the country but were instead established wherever they could find support. As such, STAR was never part of a design for systemic impact. Moreover, given the “lumpy,” two-years-at-a-time consideration of the three seemingly inde￾pendent phases of the program, there does not seem to have been a deliberate, coherent six￾year program that would “roll up” structurally or “roll out” geographically. As such, program impacts were independent, relatively self-contained, and limited both in time and geography. Moreover, with just a scattering of villages affected, program successes can also not be seen as systemic in nature. A random sample of these successes may, at best, prove that systemic ef- 3 It may never have been the responsibility of the Pact partners to take up the complaints of their PETS mentees but, if so, this was a weakness in the STAR program or an intended limitation on the program. From the perspective of the small and often isolated PETS committees, their Pact partner was much bet￾ter positioned to follow through and seek accountability it was, especially if there was indifference or resistance from the local authorities. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 23 fects would have possible with enough funding and mentoring, but they cannot really roll up without it, unless the “pilots” were to nurture a full movement. ORGANIZATIONAL IMPACT Apart from impacts on public expenditure tracking and accountability, there were impacts on the Pact partners and perhaps even on the PETS committees themselves. As stated in its pro￾gram description, Pact trained its partners to strengthen “skills and experience in democracy and good governance, organizational capacity building, grants management…and financial man￾agement.”4 Training and mentoring by Pact lasted several months and ultimately delayed the sub-grants to the PETS committees, thereby diminishing the impact of the program on the PETS committees and their tracking, monitoring, and accountability efforts. Nevertheless, almost all Pact partners appreciated the training and maintained that the assistance strengthened their or￾ganizations.5 No quantitative metrics exists to measure these efforts. CHECKS AND BALANCES Although more effective and direct, political interest does need not necessarily come from the executive branch to produce accountability. Political interest can come from elected officials like local councilors or (presumably, though no such example was elicited) members of parlia￾ment. By chance, the team witnessed a paradigmatic example of accountability in the bimonthly market in a deserted field outside Bahi village. While interviewing a ward councilor, the ward executive officer joined the discussion. The executive officer and councilor got into a heated discussion over the authority of the Bahi PETS committee in tracking budgets and identifying misappropriations. The ward executive officer insisted that those were his responsibilities and those of the ward engineers. The councilor, who had taken up a PETS committee complaint, angrily insisted that he and the committee had the right and the responsibility to act as a check on the executive branch. Were there more political competition in Tanzania and were the local officials in general as sensitive to their constituents and to their own authorities, horizontal ac￾countability would be more developed and PETS committees would have greater leverage re￾garding the misappropriations, defects, malfeasance, and corruption they uncover. As it stands, the PETS committees often felt powerless in the face of indifference or resistance by officials to whom they report misappropriations. CAUSAL RELATIONSHIPS Reports of PETS committees’ impacts suffer at times from the fallacy of temporal succession, post hoc ergo propter hoc: because it came later, it was caused by what came earlier. Even in cases where officials were transferred, suspended, or terminated after a PETS report, the causal relationship between the reports and the discipline is uncertain. Were those transfers, suspen￾sions, or terminations actually due to the reports or were there other causes? Were the PETS reports part of an accumulating set of complaints or did they stand alone? Were the transfers made in the ordinary course of hundreds of normal governmental transfers? Did the officers themselves request the transfers? Were the transfers actually promotions to better positions or more desirable postings? To ascertain the causal relationship between the actions of the 4 Pact, “Strengthening Transparency, Accountability, and Responsibility (STAR) Program Description, Resubmitted to USAID Tanzania, 22 July 2009”; p. 16. 5 Unfortunately, the team was not able to independently substantiate these statements. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 24 PETS committees and their perceived effects, a more systematic investigation of the relation is needed. INDICATIVE BUT NOT CONCLUSIVE IMPACT The reports of PETS committee impact are illustrative and suggestive but not dispositive in the absence of tighter impact reporting and, within that reporting, of demonstrated causal relations. Absolute causal certainty will almost certainly remain an ideal but will be illusive. Unable to con￾trol all other variables but PETS committee activity, uncertainty will remain about the causal relations between PETS monitoring and remedial or disciplinary action: the extent to which the latter is a response to the former. Still, greater certainty would be available in SATC with bet￾ter tracking of the PETS activities themselves and closer connection with the local officials who supervised the “offending” officials. That connection will be even less possible under SATC than under STAR however, because the Pact intermediary will be missing, however self-serving its reports might otherwise be. The Mission will almost certainly be unable to track the 25-30 PETS committees mentored or even established by the CSOs under SATC especially since the CSOs not the PETS committees will collect and report on the impact indicators and data. RELATIVE IMPACT INDICATORS Pact reported exactly what it proposed to report under each program phase. Pact proposed indicators which USAID accepted, but these impacts were relative in nature. For instance, Pact sought to “strengthen the technical capacity of CSOs to undertake advocacy activities by providing resources and support to advocacy campaigns that grow out of the PETs process and Capacity building initiatives”6 and “support government anti-corruption and good governance initiatives through awareness raising and training to government officials.”7 But, to what degree? According what measures? What was the baseline? What were the expected improvements? How much stronger would the capacity of CSOs be as a result of STAR assistance? Absent more concrete definitions and clear, let alone quantitative, targets, it is hard to know exactly what the expected impacts were under STAR. No doubt some of the vagueness derived from the uncertainty of funding and “lumpy” project structure. The weakness was in the primary de￾sign, not in deficient or improper execution of a well-considered blueprint. In that respect as well, actual PETS impacts were more fortuitous and therefore surprising than planned. LIMITED NATIONAL IMPACT If STAR had a more deliberate plan with robust indicators and a longer duration, what could STAR and similar programs funded by other donors achieved? STAR worked with 21 CSOs, each with two to six PETS committees, affecting between 60 and100 villages or wards around the country. Tanzania, however, has 30 regions, 169 districts, over 600 wards, and thousands of villages. Could STAR have made much of a dent in accountability unless these villages and wards were pilots or examples of what might be “rolled out” on a more national scale? Absent a na￾tional PETS plan with a large number of villages similarly supported by other donors or by the government itself, such a small proportion of scattered villages could not have had systemic na- 6 It would do so by “(i) CSO Capacity Building”; (ii) Sub-Grants; (iii) Strengthening Information Sharing among the Tanzania CSOs [including] a. Information, Transparency and Information, b Establish and Support a PETS Secretariat and c. Budget Advocacy.” Ibid. pp. 11-12. 7 Ibid. p. 12. It would do so by “(1) assessing and strengthening local governance and (2) local govern￾ment reform program” Ibid; p. 12. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 25 tional impact. A random sample of such successes might prove that systemic effects would be possible with enough funding and mentoring, but they could not really roll up without it. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 26 SUSTAINABILITY NO FUNDING, NO PROGRAM The single most important factor affecting the sustainability of PETS committees was funding. Pact’s own work under STAR concluded with the end of USAID assistance, as did most, but not all, the PETS committee activities and the connections between the Pact partners and the PETS committees. Once the funding ended, for instance, so too did the relationship between LEAT and its partner, WamiMbiki, and between LEAT and its PETS committees. Many respondents the team interviewed referred to the program’s perceived lack of sustainability. As a repre￾sentative from Chama chaWalemavu Tanzania (CHEWTA) put it, “We had no funds to keep going. … The program had serious problems of continuity. Once the project funding ended, it became impossible to get people together.” An official from UNGO also stated that “there was no institutional memory by Pact and no idea about the Local Governance Barometer work by UNGO.” Some relationships forged under STAR, however, did outlive USG assistance. TACASODE, for example, still maintains its contacts and representatives from AFNET told the evaluation team that the organization “stuck with [its] PETS committee” after its grant ended. In fact, the Tunajali committee turned its training into a consulting business it relies on for continued exist￾ence and work. Even if PETS committees did not survive as organizations, some members con￾tinued the PETS work. As one respondent told the team, “Individuals members of the PETS committees still monitor funds. It takes volunteerism, initiative and commitment. Nevertheless, even when contact between a Pact partner and a PETS committee continued af￾ter the end of the program, mentorship atrophied. Pact partners noted that they could not af￾ford to travel to their mentees without USAID funding. Some PETS committee members re￾ported they could not travel to local farmers, schools, or clinics beyond walking distance. Even cell phone contact, with an attendant negligible cost, withered. The program’s limited sustainability also weakened PETS committee activities. Some funding was necessary, according to several PETS committees, in order to support materials, travel, and lunches. “No funding, no program,” said one PETS committee member in a pithy synopsis. The training Pact provided its partners appeared equally tenuous. “Two or four years is a long time, and we have forgotten some of the training. And how can we do training ourselves without funding to get a room, pay bus fare or lunches?” asked a member of the Zanzibar Non￾Government Cluster for HIV/AIDS Prevention and Control (ZANGO). In part, the grants’ short duration and interruptions between program phases meant that track￾ing activities and organizational lessons learned were not reinforced long enough to endure. As noted, the Pact partners were almost entirely dependent on USAID funding since they had no organic ties to any sustaining community, going from project to project, funder to funder. In fact, GIZ was so discouraged upon finding that only five percent of the PETS committees it funded were sustainable that it completely abandoned the PETS committee approach. SNV had a similar experience but with a less far-reaching response and adjustments. As such, shortcom￾ings under STAR might be attributable to the structure of the approach. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 27 WHOSE BENEFIT? PETS committees and Pact partners in particular seem to view their funded activities more like a job they were hired by Pact and, by extension, USAID, to do, rather than as a good in and of itself. For these partners, when the funding stopped, so did the work although, within their funding and staffing constraints, they did attempt to create a coalition for some kind of sustain￾ability. The central underlying hypothesis of the entire PETS program was based on the pre￾sumption that the misallocation or misappropriation and corruption of funding meant that benefits intended for the community—such as medicines, doctors, teachers, books, fertilizers, and irrigation—were being misdirected and corrupted and that it is in the primary interest of the beneficiaries, not of USAID or Pact, to see that these benefits are delivered. Naturally, par￾ents want to see their children educated and want to see teachers show up, want to receive medical attention for their families, and want water and fertilizers for their crops. As such, it would be in their interest to track malfeasance and ensure its correction and prevention. In￾deed, AFNET reported that (presumably for just this reason) some groups, like a farmers group tracking agricultural appropriations, used their own money to continue tracking expenditures after the grants ended. The Bahi committee reported an almost identical result: when public subsidies did not arrive, an agriculture tracking committee reported the discrepancy and pri￾vately raised funds to ensure that they had crops. Why was budget tracking of a community’s public goods seen as a job provided by a donor ra￾ther than as an incentive to track its own benefits? Why wouldn’t self interest incentivize dili￾gent monitoring and corrective action? One might argue that collective-action problems led in￾dividuals to not feel responsible for expending time, effort, and funds to ensure public goods and benefits for the community at large. This is different, however, from viewing the entire ex￾ercise as a mere response to funds provided by a foreign donor. More than one respondent told the team, “It wasn’t our responsibility to continue the tracking.” But then, whose responsi￾bility is it to ensure that public benefits are delivered as promised to local beneficiaries? Whose responsibility is it to keep these activities going or will indefinite donor funding be needed to “sustain” the effort of accountability? LINKAGES TO OTHER MORE SUSTAINABLE INSTITUTIONS One possible approach to fostering more sustainable PETS committees would be to link the PETS function to other existing, sustainable organizations. Two Pact partners suggested com￾munity savings banks as organizational anchors, as villagers have a personal financial stake in these institutions. Other relatively permanent organizations, preferably where there are per￾sonal and enduring investments, that are formed for broader purposes around particular is￾sues—such as village committees or farmers’ groups similar to the one in Bahi—could be repli￾cated for other sectors, like PTA committees for schools. A former Pact staff member opined that the problem with PETS was that it was “just tracking, but had very little personal relevance to most members of the community.” Issues that surfaced in PETS committees were not always followed through on or resolved, as there was a limited ability to raise the issues sufficiently high in the political sphere. From the following discussion, it is possible to draw the following conclusions: (1) the key one is that the PETS committees need to have a political “champion” or at least some kind of politi￾cal leverage in order for any of their budget findings to have some kind of meaningful impact; (2) in some cases the centrally appointed District Executive Director (DED) took up the cause STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 28 and that could be quite effective; (3) in some cases the district or regional elected officials took up the case and found themselves at odds with the centrally-appointed officials but at least had some leverage in getting a response; (4) sometimes the village councils took a position and, for example, would refuse to work with a particular problematic DED who might be moved, just as other appointed officials might but that was an extreme reaction. For the normal cases, the PETS committees have no independent leverage other than to monitor and report. Were the political system more competitive, local opinion would have greater leverage and the PETS committee reports would matter more. Of course the lack of leverage also affects the extent to which the PETS committees remain viable and active: to the extent that they see themselves as relatively impotent to affect budget decisions, they have less incentive to track them. TAXATION AND OWNERSHIP One reason for the disassociation between individuals and the public expenditures is the ab￾sence of personal taxation, whether indirect (e.g., through sales taxes) or direct (e.g., based on income). As is the case in many developing countries, an overwhelming number of individual citizens pay no personal taxes on income. Government revenues come indirectly, such as through customs or investment taxes or corporate taxes on large, multilateral companies. Con￾sequently, citizens see government benefits more as gifts rather than as entitlements earned through personal taxation. Citizens, for the most part, appear to be happy with what they re￾ceive and less vigilant about holding officials accountable for undelivered services. As member of the former Commission on Local Government Reform Programme mused, “[There are] lots of resource leakages, but are the local people really interested?” If government budgets were de￾rived at least in part from personal taxation, many argue that community ownership and en￾gagement would increase, as citizens would be more likely track their tax dollars and social ser￾vices derived from their own pockets. As a representative from Women and Poverty Allevia￾tion in Tanzania (Wopata) told the team, “Since the irrigation project just came to the villages and [citizens] didn’t pay for it, they thought they had no right to inspect it. There was no own￾ership. When [citizens] contribute funds, they are more likely to feel ownership. No matter where the money comes from, they need to feel ownership. If they don’t feel ownership, they won’t pursue implementation as vigorously.” The problems of collective action, however, can affect the calculus of individual incentives. Ac￾cording to an official from LEAT, “If the villagers understand that the forest belongs to them, they will take action. But will villagers avert misdeeds by their fellow villagers? Will they be per￾sonal custodians of public goods? We need to help the rights holders hold the duty bearers to account.” Thus, even if there is some ownership, if it is collective, the social cost of monitoring and reporting malfeasance among community members may be too high. While personal taxation is beyond the manageable interest and scope of USAID and other in￾ternational donors, a personal or familial monetary investment might increase vigilance and par￾ticipation. As one AFNET representative noted, “There needs to be a feeling of ownership by citizens over funds. The missing link is the importance of having their priories reflected up the line to the districts and above.” If citizens’ tracking efforts at the village level do not seem to connect to the benefits they are entitled to, tracking will lose its purpose and incentive. STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 29 WHAT IS THE MEANING OF SUSTAINABILITY? While sustainability seems to be an ambiguous concept among donors, some view an organiza￾tion, project, or activity as sustainable if it continues with funding from other donors, and not from the local population. Several Pact partners—such as Civic Education Teachers Association (CETA), SAHRINGO, CELG, and Christian Council of Tanzania (CCT)—indicated that because of the experience they gained under STAR, they were able to continue PETS activities financed by other donors. Sooner or later, however, donor funding will end. A more robust definition of sustainability, however, entails the future persistence of the organization, the activity, or both after the project has ended. Sustainability should be considered in SATC: what, if any, interest does USAID have in long-term sustainability and ownership? How can the program design en￾hance both? STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 30 CONCLUSIONS AND RECOMMEN￾DATIONS As the STAR program ended two years ago, the real value of this evaluation lies in what lessons it holds for SATC and other public expenditure tracking system programs. In this section, we outline general conclusions and lessons learned and propose recommendations for future pro￾gramming. It must be noted, however, that STAR’s shortcomings were often the result of fac￾tors beyond the control of USAID. OVERALL IMPACT  Demonstrated impact. As outlined above, STAR generated several positive benefits. While there were no concrete measures of or precise reporting on program impact, the team found extensive qualitative evidence of positive program effects. While this ev￾idence may have been partially anecdotal, these anecdotes were mutually independent and too broadly dispersed to be dismissed as accidental.  Limited influence. Unless their findings garnered political or governmental interest from powerful supporters, PETS committees did not have sufficient leverage to create consequences for their monitoring and tracking.  Create national impact. If possible, greater attention should be placed at the outset of SATC to create nationwide, systematic, sustainable impact. Coordination with other donors would broaden this impact and program activities must not be restricted to the wards and districts in which “their” PETS committees operate. PETS activities need to find a way of spreading if they are to have national impact.  Increased focus on media coverage of program activities. BONGA placed sub￾stantial emphasis on media coverage and journalist training. Some interviewees spoke of successes that were augmented by media coverage of their activities and raising the is￾sues of public spending and accountability in the public consciousness. The coverage and the awareness would bolster the somewhat relative powerlessness of the PETS commit￾tees where they face political and bureaucratic resistance and should perhaps be incor￾porated into SATC. PETS COMMITTEES  Limited local ownership. Under STAR, there was not enough community ownership over the PETS committees or understanding of their benefits and beneficiaries. Conse￾quently, there was not enough personal commitment among individuals and communi￾ties to sustain the PETS committees after USAID funding ended. While poor communi￾ties would have been hard-pressed to raise even minimal amounts, PETS committees and their tracking activities did continue in some areas.  Lower expected transactional costs. The long distance between some Pact part￾ners and their mentee PETS committees made post-funding support and personal con- STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 31 tact, let alone mentoring, difficult. SATC, on the other hand, will be more geographically focused. The program will be concentrated in five regions: Dodoma, Iringa, Morogoro, Mtwara, and Zanzibar. Several of the potential SATC CSOs are based in Dar es Salaam and each organization will focus on one or two functional themes, such as health, agri￾culture, education, or natural resource management. As such, transactional costs are expected to be lower under SATC than under STAR. Still, care should be taken to min￾imize these costs.  Increase integration with local institutions. To increase the likelihood of sustaina￾bility, SATC should integrate the PETS program with existing community organizations and activities in which local people have a vested interest. Program activities stand a bet￾ter chance of having impact if they are anchored locally.  Structural power imbalances limit impact. PETS committees have little practical leverage to voice their demands or assert their rights in the face of tenured government officials with funds to distribute or withhold. As one interviewee said, legal action, even if available and accessible in theory, is drawn-out and expensive in practice. The relative imbalance of power and funding is substantial, unless the government makes clear that it will sanction, if necessary with legal prosecution, the unwillingness of its own officers to share information, respond to legitimate public inquiries, formally pursue credible allega￾tions of misappropriations and corruption, and take serious action (even against politi￾cally or economically protected officials) when those inquiries prove grounded.  Adopt a long-term view. A PETS program will probably need three to five years of funding to sustain PETS committees institutionally and ground their lessons. It will take time for relatively powerless, unorganized, and often illiterate villagers to understand principles of transparency and accountability well enough to apply them after funding ends, let alone apply them to other unfunded sectors. Moreover, unless PETS functions are grounded in existing, relatively permanent organizations or institutions, new, inde￾pendent PETS committees are unlikely to survive on two years of funding. SUSTAINABILITY AND OWNERSHIP  Place greater emphasis on sustainability. From the outset under SATC, greater concern should be placed on the long-term sustainability of the PETS committees as well as fostering an enduring relationship between the CSOs and the PETS committees they are mentoring and funding. While an eventual atrophy of the relationship between the CSOs and the PETS committees may be unavoidable, the sustainability of the PETS committees and their value to their communities should be a priority under SATC. If there is no understanding and ownership of the value of the PETS committees in their respective communities, their sustainability will be limited.  Develop and strengthen relationships with PMO-RALG. PMO-RALG continues to monitor budgets and provide PETS support in tracking expenditures. Since so much of the local budget comes from the central government, this financial support is crucial to the future impact of PETS activities. As one of the relatively powerful internal Gov￾ernment of Tanzania supporters, PETS committees should be actively seeking to culti- STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 32 vate relationships with PMO-RALG. PMO-RALG has an interest in ensuring that its own guidelines are observed and the PETS committees are one resource—a local ear to the ground, so to speak—for helping it do so. As such, a stronger connection among the CSOs, their PETS committee mentees, and PMO-RALG would be a win-win relationship in a political environment in which PETS committees and CSOs are relatively weak. Hopefully, the GoT’s IoAs will also be SATC partners and provide political support to PETS committees. A close relationship, therefore, between the SATC partners and IoAs should also strengthen impact under SATC, if those relationships can be forged and if the IoAs are willing to use their influence to support the PETS efforts.  Specify intended beneficiaries. Questions USAID and the future implementer might consider include: In effect, whose benefit is the program for? What is the fundamental hypothesis about benefits and commitment? How realistic is it? MONITORING AND EVALUATION  SATC should include a concrete monitoring and evaluation system to track impact at the local level. This system should be based on a realistic hypothesis of expected effects on transparency and accountability with a common understanding among all stakeholders of transparency and accountability and how they will be manifest. Questions USAID and the future implementer might consider include: What is the pro￾posed national impact of the program? To what extent are the CSOs and their PETS beneficiaries achieving it? What are the impediments? How can these impediments be ameliorated? Do the expected impacts need to be adjusted?  Track impact at the local level. CSOs are the vehicles for PETS impact, but the ac￾tual impact will be primarily felt at the PETS committee level. As the PETS committees are the primary beneficiaries of the program, SATC indicators should track impact for PETS committees at the district, ward, and village level.  Focus training on PETS activities. Training is important and should be focused on PETS activities, with less emphasis on grants management and organizational structure. Sustainability will depend more on the results PETS committees achieve for themselves and their communities than on their organizational structure. Local PETS committees will develop their own local organizational modalities, subject to local conditions and funding. However, no matter how well organized PETS committees are, if PETS activities are not seen to have immediate positive impact, they are not likely to be sustained ab￾sent external funding.  Monitor beneficiary impacts. Actual beneficiary impacts themselves should be moni￾tored. Reliance on illustrations and anecdotes should not substitute for (preferably quantitative) impact indicators or on reports alone, or on the temporal relation be￾tween PETS actions and accountability action as if action that succeeds a PETS report is indicative of a causal relation between the two.  Develop clear, measurable indicators. Indicators for SATC should be clear and measurable. STAR’s impacts were entirely relative, without clearly defined aspirations or STAR PROGRAM PERFORMANCE EVALUATION—REVISED DRAFT REPORT 33 specific measures. Questions USAID and the future implementer might consider include: To what degree and along what axes would committees be built, improved, or strength￾ened? According to what measures? What is the baseline? What are the expected or hoped for results and what would be considered an improvement? What are the goals, if not the achievement? To what extent can these indicators be plotted on a nominal, or￾dinal, interval or ratio scale? STAR EVALUATION—DRAFT REPORT 34 ANNEX A: STATEMENT OF WORK SECTION C–DESCRIPTION I SPECIFICATIONS/STATEMENT OF WORK EVALUATION SERVICES FOR DRG PORTFOLIO, USAID/TANZANIA C.l Purpose and Objectives: The purpose of this accountable governance program evaluation contract with USAID/Tanzania is to support the USAID/Tanzania Democracy, Rights, and Governance (DRG) team's ability to monitor and evaluate its activities. In order to adequately fulfill the monitoring and evaluation (M&E) requirements of the USAID/DRG core program, Supporting Accountability to Tanzania Cit￾izens (SATC), near-term support is needed on specific tasks that will help ensure that timely, accu￾rate and useful information is reported throughout the life of the SATC program. The outputs from this contract will feed directly into the upcoming Mission-wide M&E contract, which is to be awarded in the coming year. USAID/Tanzania is looking for an independent entity to perform specific tasks to support the im￾mediate M&E needs of the DRG portfolio over a 12 month period. These tasks should enable USAID/Tanzania to fulfill its performance monitoring, evaluation, reporting, and dissemination re￾quirements as mandated in the Automated Directives System (ADS) and other Agency guidance. The Contractor must: 1.Conduct a post-program performance evaluation of the Strengthening, Transparency, Accountability, and Responsibility (STAR) program, as a culmination of two prior programs implemented by Pact; 2.Conduct a desk review of programs in Tanzania similar to the DRG SATC program and the tools they use for measuring and assessing their programs; 3.Draft indicators for SATC in collaboration with partners already online; 4.Develop tools for measuring SATC outcomes; 5.Propose a Civil Society Organization (CSO) index that looks at outcomes from SATC inputs; 6.Conduct the first perception survey for SATC; 7.Establish baseline data for SATC program; and 8.Develop a plan for the handover of DRG M&E to the Mission-wide M&E Contractor. Guiding Principles:  Utilize an asset-based approach and lens in assessing and implementing program activi￾ties (an asset-based approach takes the assets of individuals and communities as the starting point, as opposed to their needs, and empowers individuals and communities to be partners in their own development process); STAR EVALUATION—DRAFT REPORT 35  Promote gender equity in the assessment, design, planning, and implementation of the contract activities and support women to assume leadership roles;  Employ a do-no-harm approach to reduce the exposure of targeted communities inad￾vertent risks;  Build capacity less through training than through learning-by-doing, to ensure that local partners build their ability and confidence to lead similar programs in the future; and.  Offers that creatively and appropriately adapt existing tools will be favored over pro￾posals that simply generate new tools for the sake of it. Innovation is encouraged but must be contextually sound. C.2 Program Background: USAID/Tanzania's Democracy, Rights and Governance (DRG) program, entitled "Supporting Accountability to Tanzania Citizens" (hereafter referred to as the "Program"or "SATC"), is a three-pronged, sector-based intervention to improve accountability and oversight of public re￾sources through increased citizen participation. The program will achieve greater citizen en￾gagement in service delivery and strengthened linkages between government Institutions of Accountability (loA) and civil society organizations (CSO). This approach will bridge gaps in CSOs’ and citizens' understanding of their rights and their ability to advocate for those rights. The program will catalyze change in governance as CSOs improve their connection with citi￾zens, as more citizens take advantage of the opportunity to understand their individual rights and as they better understand what role they play in the democratic process. By 2016, USAID/Tanzania will help achieve, in targeted geographic zones and in selected sectors, a more accountable, inclusive decision-making process for public resource management, where civic participation in public affairs is normalized. Tanzanians' informed and active participation in local governance will help set the stage for the 2015 presidential and parliamentary elec￾tions. More importantly, efforts to foster greater citizen engagement are expected to model and incentivize behaviors that can be replicated broadly. Such efforts are critical to creating demand for increased transparency and, ultimately, improvements in national-level government accountability. The program's Prongs One (a grant-making mechanism to local CSOs) and Two (four direct government-to-government funding mechanisms) represent the demand and supply of account￾ability respectively, which must be targeted simultaneously to stimulate motivation for sus￾tained engagement on both sides. Those first two components will be connected by Prong Three: a capacity development contractor with varied local expertise that will provide tailored Technical Assistance (TA) and capacity development support to the Tanzanian awardees, both government and CSO. Under Prong One, USAID will directly engage CSOs involved in the sectors which USAID/Tanzania is already targeting -health, agriculture/food security, natural resource manage￾ment (NRM), and education -through direct grants that will support these CSOs in advocacy, is￾sue-based networking, and social accountability monitoring. These activities will align with the focus regions of USAID's Global Health Initiative (GHI), Feed the Future (FtF), NRM, and Education Pro￾grams, namely Dodoma, Iringa, Morogoro, Zanzibar, and Mtwara. Prong Two is a set of four agreements with GoT IoAs to strengthen their local level staff, improve their information packag￾ing, facilitate outreach to the public, and improve their linkages with civil society and the media. STAR EVALUATION—DRAFT REPORT 36 USAID/Tanzania is seeking support on specific tasks that are immediately needed to ensure effective evaluation of the program over the life of SATC. C.3 Statement of Work: In order to accurately evaluate achievements made under SATC's overarching goal, as well as the success of its three-pronged approach, tasks must be completed in the next 12 months. In estab￾lishing the means for monitoring data and eventually evaluating the three-pronged approach, the Contractor should appropriately adapt and/or use data collected by other stakeholders -GoT, CSOs, Afrobarometer, other USAID/Tanzania implementing partners, other development part￾ners' analyses. The Contractor may decide to expand existing perception surveys to be frequent enough and seeped appropriately to serve the needs under this contract. New data collection tools may still be required and the Contractor must identify them and include their design, devel￾opment, and deployment for establishing baseline data and the strategy for their continued usage once the long- term Mission-wide M&E contractor takes over. Close coordination with the CDPA, which will be monitoring partner CSOs' institutional strengths, will be necessary. The Contractor will create and use a citizen/government percep￾tion survey mechanism as well as a CSO capacity and efficacy index. Attribution of impacts to USAID/Tanzania interventions must be a central consideration when developing surveys, tools, and establishing baseline data 1) Performance Evaluation of STAR USAID/Tanzania has the need for a program performance evaluation of USAID-funded support to Tanzanian advocacy organizations. This support took the form of three separate project all imple￾mented by Pact, and each building on the achievements of the previous program. This evaluation should focus on the last project, STAR, and encompass data from its parent programs Tanzania Advocacy Partnership Program (TAPP) and Building Organizational Networks for Good Govern￾ance and Advocacy (BONGA) to the extent possible. The evaluation is required to commence im￾mediately upon award. The Contractor must provide, in the proposal, illustrative information on how such an evaluation would be approached, illustrative probable costs, and possible constraints. The purposes for evaluating these programs are to 1) provide a post-program independent perfor￾mance evaluation; and 2) use lessons learned from those programs to inform USAID/Tanzania's current and future programming in similar sectors. USAID devoted considerable time and re￾sources to the STAR program and its precursors. A proper independent evaluation which provides recommendations for future programming will increase its learning base for accountable govern￾ance programming in Tanzania. Lessons learned from the evaluation of this program will assist in establishing baseline data for the monitoring and evaluation of SATC. A key hypothesis to test is the impact of Public Expenditure Tracking Systems, under donor funded programs like STAR, in “empowering Tanzanian citizens at local levels and organizations at the na￾tional level to participate and demand transparency and accountability in the systems for allocation and use of public resources, and in planning and delivery of public services for socio-economic develop- STAR EVALUATION—DRAFT REPORT 37 ment and poverty reduction.” This is about searching for missing links to the ability and willingness of citizens to engage in government oversight over the long haul. “Strengthening Transparency, Accountability, and Responsibility” Program In January 2012, Pact closed the 2-year "Strengthening Transparency, Accountability, and Re￾sponsibility" (STAR) program, which was a successor program to two previous Pact programs focused on civil society, advocacy, and participatory decision-making for good governance. Initially, Pact Tanzania implemented, from 2002-2005, the Tanzania Advocacy Partnership Program (TAPP) under funding from the USAID/DG office. This program established a cadre of advocacy civil society organizations (CSOs) in Tanzania involved in legislation and policy reform. From 2006- 2008, Pact implemented the BONGA program (Building Organizational Networks for Good Governance and Advocacy and a Kiswahili word meaning "speak out!"). The United States Government's Millen￾nium Challenge Account (MCA), through USAID, supported the program as part of the Govern￾ment of Tanzania's MCA Threshold Program. The overall goal of the BONGA program was to strengthen the capacity of Tanzanian CSOs to implement advocacy activities in order to increase public participation in the political process and holding government accountable. Ittrained over 300 journalists in investigative journalism; led to corruption stories being reported in the media, trained over 2,000 individuals on organizational development. Public Expenditure Tracking Systems activities took place in 65 districts supported by resources and training from Pact at the community level by civil society networks. The GoT, through the Prime Minister's Office Regional and Local Government (PMO-RALG), developed guidelines for Public Expenditure Tracking Systems based on BONGA's work and have distributed them to local government authorities (LGAs). Pact Tanzania subsequently followed on BONGA with the two-year STAR. STAR’s objectives included:  Building the capacity of the citizens in selected districts to enable them to participate in the development plans, monitoring of local budgets and service delivery, and, to strengthen their voice in demanding a transparent, accountable and responsive government;  Enhance capacity of CSO and Citizen Groups to engage in participatory policy making and planning process;  Enhance the capacity of CSO/Citizen groups in monitoring and evaluation of public services and goods;  Increase the ability of citizen groups to understand and demand their rights for information by working with LGAs to ensure that lower local government representatives make infor￾mation publicly available;  Strengthen the technical capacity of CSOs to undertake advocacy activities that grow out of the PETs process and capacity building initiatives. The evaluation of the STAR program must identify, to the furthest extent possible, the degree to which Pact attained the objectives laid out in the funded program description:  To what extent was/is its capacity building sustainable?  How might current or new USAID programs use, leverage, or build upon that capacity? STAR EVALUATION—DRAFT REPORT 38  What tools or mechanisms - Public Expenditure Tracking Survey committees, citizen report cards, community score cards, or PIMA cards, notice boards -proved most effective in help￾ing ordinary citizens monitor public budgetary decisions and service delivery, under what circumstances? Was this appropriately measured by the program, and if not what are meth￾ods to measure such a result? To what extent did the 1) CSO Capacity Building, 2) Sub grants, and 3) "Strengthening Networking Information Sharing among Tanzania CSOs" each contribute to the achievement of the program ob￾jective? The evaluation must also answer a question about target communities' awareness and attitudes to￾ward the STAR program, Public Expenditure Tracking, and oversight of local government. The Contractor must make use of the prior assessments and evaluations that were conducted for these activities, including:  Final Performance Audit MCA Threshold Program Tanzania, November 2008;  The Millennium Challenge Corporation’s Tanzania Threshold Country Program: End Term  Evaluation, February 2011;  Pact's End of Program Evaluation Report for STAR, March 2012. These and other relevant documentation and program reports will be provided by USAID. The audience of the evaluation report includes the USAID/Tanzania Mission, specifically the DRG Team, the USAID Africa and Democracy, Conflict, and Humanitarian Assistance (DCHA) Bureaus, other USG stakeholders, the Tanzanian Government, including the PMO-RALG, local government, development partners, USAID/DRG implementing partners and beneficiaries. USAID will use the re￾port to share lessons learned with these stakeholders. The evaluation will emphasize the 65 districts targeted for Public Expenditure Tracking and the 20 partner CSOs. The Contractor must develop an evaluation design, data collection method, and an analysis plan using a mixed methods approach based on sound social science tools used in a manner to minimize potential biases and to capture a best possible picture of the targeted districts with the resources available. These components will be presented to USAID for review and approval before conducting the evaluation. Although the STAR program is now closed, Pact maintains a presence in Tanzania, conducting other projects, including USAID-funded ones. The Contractor must propose an approach and plan for conducting STAR program evaluation and providing USAID/Tanzania with findings for the STAR pro￾gram and recommendations based on those findings for consideration in the design of any future, similar programs. The evaluation approach and plan should be fully cognizant of and state any limita￾tions, including but not limited to the closure of the program and resource constraints. Approaches should include, but not be limited to, intensive review of USAID, Pact, and beneficiary agen￾cy/organization documents, direct site observations, and key informant interviews, including other donors and NGOs working the same field such as DFID, GIZ, HakiElimu, and Policy Forum. 2) M&E for SATC Program STAR EVALUATION—DRAFT REPORT 39 While individual USAlD/Tanzania implementing partners under SATC will develop performance monitoring plans (PMPs) for their specific programs, this Contract is meant to refine the strategy indicators and targets for monitoring higher-level USAlD/DRG portfolio progress toward the de￾velopment objective of “Improved Accountability and Oversight of Public Resources through In￾creased Citizen Participation.” If needed, the Contractor must design a plan, and deploy an ap￾propriate tool(s) for collecting baseline data to analyze and report on that data as required by USAID/Tanzania. This plan should include annual data and analysis so that comparisons can be made to the baseline. Being able to discern if changes in status-quo can be attributed to the SATC program is a priority. First, the Contractor will develop indicators, based on its initial desk review, demonstrative of the program's impact to be tracked over the life of the SATC program. These should be devel￾oped in concert with implementing partners who are expected to be developing theirs within the first quarter of their year-1 activities. Most partners will sign their awards in the fiscal year 2013. Second, the Contractor must design tools and develop baselines to support the following indica￾tors (with illustrative targets). These may be fine-tuned with the COR during the first quarter of the first quarter of the contract:  Changes in citizens' and local officials' views of governance in targeted locations;  25% positive change in citizen opinion in targeted geographical locations about the relevance of CSOs for advocacy;  25% positive change in local understanding of rights around targeted issues in USAID-  targeted geographical areas;  30% increased public understanding of partner IoAs' functions;  Progress along CSO Index (would include effectiveness of citizen-driven advocacy such as the number of actions taken by USAID-supported GoT institutions in response to issues raised by citizens,· rate of unsolicited media coverage of CSO advocacy campaigns);  Change in citizens’ voicing of informed opinion. Among the measurement tools, the Contractor must develop or modify an outcome-specific CSO in￾dex that tracks impact of USAlD support directed at CSOs on the advocacy and accountability envi￾ronment in the targeted geographical regions, with the highest possible degree of attribution to the SATC activities. Third, the Contractor will carry out the first survey of citizens and civil servant/public officials for the SATC program in the field, working with project stakeholders and intended beneficiaries to capture perceptions of government engagement, responsiveness, and accountability, and of CSO usefulness, reputation, and ability to represent citizen issues related to service delivery to their government. The survey would have to be replicable on a regular basis-say every other year unless the Contractor has a strong argument for higher frequency-and will be more precise and systematic than a convenience or availability sampling. The findings of the initial perception survey and initial information for the CSO index will be analyzed and used for adjusting targets for the indicators previously created and for establishing the baseline for the SATC program. STAR EVALUATION—DRAFT REPORT 40 Monitoring data collected under this program must be compatible with and support USAID/Tanzania's Mission-wide monitoring and evaluation system, currently managed by The Mitchell Group as T-MEMS (Tanzania Monitoring and Evaluation Management System, www.tanzaniamems.com/index.cfrn), or its successor. Data collected must always be disaggregated by sex and, where possible, by age. The Contractor must address gender considerations in the collection of research data and the implications these considera￾tions have for the appropriate gender breakdown of enumerators and other researchers trained. Critical to the task of evaluating USAID/Tanzania investments will be the dissemination of information and lessons learned from evaluations. The Contractor will be responsible for analyzing data collected for writing a summary report that is clear and accessible to a variety of audiences. Comparative infor￾mation from districts and CSOs outside the program should also be shared by the contractor for evaluative and learning purposes. Measurement Tools: Appropriate tools and surveys will be developed for use by USAID/Tanzania's DRG office. These tools and surveys will be deployed for use in evaluating the SATC program by this Contractor as well as USAID and its future M&E contractor. The Contactor must acknowledge and draw upon the state-of-the-art outcome measurement and monitoring tools in the democracy and governance sector. For example, myriad civil society organiza￾tion capacity and effectiveness assessment tools exist, including but not limited to the T- MEMs advo￾cacy measurement tool (www.tanzaniamems.com/tools.cfm). Where appropriate and permissible per copyrights, general or sector-level measurement tools may be adaptable for the purposes outlined in this SOW. For example, the Afrobarometer public perception survey (www.afrobarometer.org), as conducted in Tanzania, asks questions about government respon￾siveness and civil society action for nation-level information. Also, USAID's Civil Society Organization Sustainability Index (www.usaid.gov/africacivilsociety) scores the Tanzanian civil society sector on, among other things, advocacy, organization capacity, and public image dimensions; sector-level indica￾tors may be adaptable for program-level application. The Contractor must coordinate and consolidate data collection or research actions under this con￾tract, or coordinate them with pre-existing research programs on-going in each location, and, where possible, economize expense and effort and leverage existing surveys and other research work already being undertaken. This may include conducting a program performance evaluation work simultaneous￾ly with portfolio-level data collection, funding a local institution for additional survey work to be con￾ducted in areas of interest where an existing survey instrument is already being used by another de￾velopment partner, funding a local institution for additional questions to be asked where quality assur￾ance can be assured, and collaborating with existing research projects to extend their scope into new areas of interest. Activities under this contract are clearly distinguishable from on-going M&E activities to be carried out by USAID/Tanzania's implementing partners as part of their on-going programs. Each implementing partner will be responsible for having a robust M&E system that serves as a resource in managing their programs over time and recalibrating activities to support program objectives during implementation. STAR EVALUATION—DRAFT REPORT 41 Evaluation activities conducted under this contract will be complementary to these implementing part￾ner M&E processes, albeit with a stronger focus on measuring progress on the attainment of higher order program objectives. Nonetheless, this M&E contractor must coordinate with USAID/Tanzania DRG implementing part￾ners to ensure that M&E tools and systems are compatible, mutually reinforcing, well leveraged, and properly coordinated to avoid waste of U.S. taxpayer resources. For example, to the extent the "Ca￾pacity Development Program for Partners of Accountability" implementing partner selects a particu￾lar CSO assessment tool, the Contractor should coordinate with that partner and leverage the tool as well as the partner's M&E system to measure DRG program-level indicators. 3) Collaborative Partnership Responsibilities The success of the DRG development objective requires collaboration with and among multiple partners, within the USAID Mission, the GoT, and civil society. The Contractor will therefore be responsible for working closely with a variety of partners, possibly including but not neces￾sarily limited to the following:  USAID/Tanzania's DRG technical team;  At least one Tanzanian monitoring and evaluation firm or organization--this will be particularly important in conducting the first perception survey since the Contractor's core staff will need, at a minimum, the assistance of a group of local data collectors;  Other USAID technical teams or key staff implicated by the program's cross-cutting approach and working in such areas as agriculture/food security, natural resource management, health, education, gender, and monitoring and evaluation;  USAID implementing partners who are directly involved in the SATC program activities: cur￾rently expected to be four (4) GoT partners, four (4) or five (5) grantees, and one (1) capaci￾ty building contractor–the GoT partners may be eager to measure some form of impact of their work and close collaboration with these partners may be instrumental in informing them on most effective methods of culling and analyzing relevant data for practical uses, such as commissioning local research/analytical firms;  Other donors contributing to similar DRG activities or programs;  Producers of other measurement tools and/or indices, such as the Afrobarometer, the Civil So￾ciety Index, or the Civil Society Organizations Sustainability Index). The Contractor is required to define each link and working relationship with these partners as well as any others the Contractor deems necessary to effectively monitor and evaluate attainment of USAID/Tanzania's DRG development objective: Improved Accountability and Oversight of Pub￾lic Resources through Increased Citizen Participation. C.4 Staff: USAID seeks the services of a Contractor who will provide a multi-disciplinary evaluation team who have the relevant qualifications below. The contractor-furnished team members shall meet the minimum level of academic and work experience qualifications outlined in Section B.5 (a) (2) of the IQC. STAR EVALUATION—DRAFT REPORT 42 Labor Categories–Level of Qualification Evaluation Team Leader (senior level): This individual will be responsible for completing all delivera￾bles in the time allotted. He/she will coordinate the overall evaluation effort and be responsible for editing and compilation of all deliverables. This individual will have substantial experience and knowledge of accountable governance, civil society, and advocacy issues as well as with the moni￾toring and evaluation of development interventions. East African and Tanzania regional experience is also preferred. Evaluation Specialist (senior level): This individual will have significant background in the design and undertaking of performance evaluations and long term measurement tools. He/she will en￾sure that the design of the team's evaluation plan is adequate in answering the relevant evalua￾tions questions. He/she will be instrumental in designing the data gathering tools, developing the program indicators, developing relevant indices, and managing surveys. He/she will ensure that all findings are communicated fully and accurately in the deliverables. He/she will have experience and expertise in data analysis, and in discussion facilitation. Experience in Tanzania is preferred. This person will travel with the team throughout the data collection and stakeholder meeting pe￾riod. Technical Advisor (mid-level): This individual will be responsible for data collection in Tanzania, while working closely with Evaluation Team Leader and Evaluation Specialist. This individual will be responsible for coordinating local enumerators for data collection. He/she should be familiar with national data collection and analyzing institutions and with the country and its logistics, to assist in developing travel itineraries and make necessary arrangements in country. C.5 General Considerations: The Contractor must account for the following general considerations. Gender Gender issues, in particular cultural, economic and legal gender inequalities, are a critical underlying fac￾tor in making progress in all development sectors in Tanzania. Addressing gender in all USAID solicita￾tions is a mandatory requirement, and USAID/Tanzania has identified gender as a cross-cutting theme that is built into all USAID/Tanzania programs through the strategy development process. In addition to addressing gender considerations identified elsewhere in this program- gender represen￾tation in local partner(s), enumerators, data disaggregation, etc., the Contractor must identify and address any other gender issues anticipated in the conduct of this contract. The Contractor must describe how the project will adequately address gender issues and demonstrate an integrated understanding of gender concerns in the activities and/or ap￾proaches being proposed. The program must fully subscribe to the USAID Gender Policy and ADS 201.3.11.6., which requires a description of how gender relations will affect the achievement of sustainable results and how the proposed results will affect the relative status of men and women. The Contractor must address relevant and important gender issues and disparities, and the necessary human skills, material and financial resources, and time needed to address identified STAR EVALUATION—DRAFT REPORT 43 gender issues. The Contractor's PMP must also include indicators and benchmarks to monitor progress. C.6 Environmental Considerations: This project consists of interventions entirely within the categories listed in paragraph (c)(l) and (c)(2) “Categorical Exclusions”of Title 22 CFR Part 216. The Initial Environmental Evalua￾tion has determined that these activities are categorically excluded from any further environ￾mental review requirements. Any activities found to be outside the scope of the approved Regulation 216 environmental documentation shall be halted until an amendment to the documentation is submitted and writ￾ten approval is received from USAID. C.7 Avoidance of Conflicts of Interest: The implementer of this USAID/Tanzania evaluation contract will be excluded from participation in other USAID contracts or agreements that are part of the SATC program to be monitored and evaluated based on this Scope of Work. END OF SETION C STAR EVALUATION—DRAFT REPORT 44 ANNEX B: LIST OF INTERVIEWS NAME AFFILIATION Daniel Dedeyan USAID/Tanzania Magdalena Hiza USAID/Tanzania Salim R Olotu Executive Director, Civic Education Teachers Association (CE￾TA) Safari Fidelis Executive Secretary, Civic Education Teachers Association (CE￾TA) Goodluck Justine Project Coordinator, Civic Education Teachers Association (CE￾TA) Julieth Mushi Secretary, Civic Education Teachers Association (CETA) Martin Mwinyi Bakari Member, Chama cha Walemavu Tanzania (CHAWATA) Kinon￾doni Bakari Athumani Member, Chama cha Walemavu Tanzania (CHAWATA) Kinon￾doni Sikujua Mbwembwe Member, Chama cha Walemavu Tanzania (CHAWATA) Kinon￾doni Said J. Machaku Secretary, Chama cha Walemavu Tanzania (CHAWATA) Kinon￾doni Elias Masamaki Secretary General, Chama cha Walemavu Tanzania (CHA￾WATA) Kinondoni Subira Semsimbazi Chairperson, Chama cha Walemavu Tanzania (CHAWATA) Ki￾nondoni Martina M. Kabisama National Coordinator, Southern Africa Human Rights NGO Network (SAHRINGON) Tanzania Chapter Adam Lingson Program Officer, Oxfam & Former Pact (STAR) Staff Todd Malone Country Director, Pact Peter K. Mwakabwale Chief of Party, Capacity Development for Partners of Accounta￾bility(CDPA)/Chemonics - Pamoja Twajenga Camilius Kapela M&E Officer, Capacity Development for Partners of Accountabil￾ity(CDPA)/Chemonics - Pamoja Twajenga Julie Adkins Program Leader, SNV Richard Komba Planning, Monitoring and Evaluation Advisor, SNV Philipp Schattenmann Governance & Accountability Advisor, Deutsche Gesellschaft fur Internationale Zusammenarbeit (GIZ) Mjengi Lissu Advocate/Legal Adviser, Centre for Environmental Law and Gov￾ernance (CELG) Tanzania STAR EVALUATION—DRAFT REPORT 45 Sylvia Madata M&E Officer, Centre for Environmental Law and Governance (CELG) Tanzania Alex Donati Senzia Country Coordinator, PELUM Theo Macha CEO, Devimpacttz Ms. Sara Mwaga Executive Director, Anti Female Genital Mutilation Network (AFNET) Dodoma Ms. Mbiki Mumba Executive Director, Anti Female Genital Mutilation Network (AFNET) Dodoma Joshua Daudi Mbijili Member, Bahi Development Group (BADEGRO) Siwazuri Dismas Member, Bahi Development Group (BADEGRO) Mwanahamisi Ally Member, Bahi Development Group (BADEGRO) Miraji Bakari Member, Bahi Development Group (BADEGRO) John W. Makasi Village Executive Officer (VEO), Bahi Juma R. Idd Ward Executive Officer (WEO), Bahi Ramadhan J. Mtawa Councilor, Bahi Mwancity Lyeme Secretary, CCM Women Association - Bahi District Luciana Sangati Treasurer, CCM - Bahi District Mwasiti Liemey Citizen, Bahi Nicholaus Damiani Kosey Citizen, Bahi Aaron Mazubesi Citizen, Bahi Eng. Victor Noah General Secretary, Christian Council of Tanzania (CCT) Rev. Dr. Leonard Mtaita Former Program Director Governance, Christian Council of Tanzania (CCT) Angelus Mapunda Financial Management Officer, Prime Minister's Office Regional Administration Local Government (PMORALG) Mariam Rajab Program Coordinator, Prime Minister's Office Regional Admin￾istration Local Government LGRP (PMORALG) Johnson Nyingi Assistant Program Coordinator LGRP/LGDG, Prime Minister's Office Regional Administration Local Government (PMORALG) STAR EVALUATION—DRAFT REPORT 46 Angelista Kihaga Permanent Secretary, Prime Minister's Office Regional Admin￾istration Local Government (PMORALG) Jumanne A. Sagini Assistant Director ICT, Prime Minister's Office Regional Admin￾istration Local Government (PMORALG) Erick J. Kitali Executive Secretary, Union of Non-Governmental Organization (UNGO) Morogoro Iddi Mdanku Manager, Union of Non-Governmental Organization (UNGO) David Mbezi Teacher, Union of Non-Governmental Organization (UNGO) Mussa H. Matola Secretary, Union of Non-Governmental Organization (UNGO) Ernest Shija Education-Coordinator Anna Joseph Journalist Fitina Haule Secretary, GSTF John L. Mengele Member, Union of Non-Governmental Organization (UNGO) Ahmed M. Mdangu Member, Union of Non-Governmental Organization (UNGO) Shebe Mzapakolo Ex-Treasurer, Union of Non-Governmental Organization (UN￾GO) Masudi Makullah Women and Poverty Alleviation in Tanzania (WOPATA) Joseph Puppa Executive Secretary, Women and Poverty Alleviation in Tanzania (WOPATA) Angelina Mziray Head, Twaweza East Africa Rakesh Rajani Vice-Chair, Zanzibar NGOs Cluster (ZANGOC) Hassan M. Mcha Member, Zanzibar NGOs Cluster (ZANGOC) Makane M. Silerice Member, Zanzibar NGOs Cluster (ZANGOC) Hanis Suuleiman Member, Zanzibar NGOs Cluster (ZANGOC) Malok Salayey Member, Zanzibar NGOs Cluster (ZANGOC) Malish Sugai UMAWA Khamis S. Kasnga Member, Zanzibar NGOs Cluster (ZANGOC) Mbarouic Said Ali YOSEFO STAR EVALUATION—DRAFT REPORT 47 Mwanaisha Haji CHAKUWAZA Slimu I. Miraji Ex-Commission Member, Zanzibar NGOs Cluster (ZANGOC) Jumps Ghum Jump Member, Zanzibar NGOs Cluster (ZANGOC) Haji Khatib Haji Acting Commissioner, Department of Human Resources, Plan￾ning and Development, President’s Office of Finance, Economy & Planning Development, Revolutionary Government of Zanzibar; Department of National Planning, Poverty Reduction Section Seif Shaaban Mwinyi President, Zanzibar Current Generation Forum (ZCGF) Yahya Omari Ex-Commission Member, Zanzibar Current Generation Forum (ZCGF) Anas Malim Treasurer, Zanzibar Current Generation Forum (ZCGF) Nassar Said Mohamed Program Associate, Search for Common Ground Eli Groener General Secretary, Zanzibar Current Generation Forum (ZCGF) Abushir Khatib Member, Zanzibar Current Generation Forum (ZCGF) Rajab Omar Jumal Board Member, Zanzibar Current Generation Forum (ZCGF) Fatma Saleh Member, Zanzibar Current Generation Forum (ZCGF) - Execu￾tive Committee Anas Malim Member, Zanzibar Current Generation Forum (ZCGF) Rashid Khalfan Said Director Development Advisory Service, KPMG Michael Ward Director, DfID Programmes in Tanzania (KPMG) Kate Dyer Program Director, Accountability in Tanzania Programme (AcT) KPMG (DfID) Tixon T. Nzunda Assistant Secretary Political Leader Research Unit, Ethics Secre￾tariat (ES) Evodia Pangani Statistician, Ethics Secretariat (ES) Cosmas Kaitani Head-Information and Communication Technology (ICT), Ethics Secretariat (ES) Masegese Kamulika Chief Accountant, Ethics Secretariat (ES) Suleimani Dbalula Assistant Secretary Public Leader, Ethics Secretariat (ES) Zahara S Guga Director Administration and Human Right Management, Ethics Secretariat (ES) STAR EVALUATION—DRAFT REPORT 48 Emma P. Lymo Chief Internal Auditor, Ethics Secretariat (ES) Margareth Chiwenda Head of Government Communication Unit, Ethics Secretariat (ES) Sheiba Bulu Economist, Ethics Secretariat (ES) Heri marco Director of Planning Monitoring and Evaluation, Ethics Secretariat (ES) Felista Meena Acting Secretary Political Leader, Ethics Secretariat (ES) Bazilio Mwanakatwe Manager Administration and Human Resource, Public Procure￾ment Regulatory Authority (PPRA) Dorothy Mmbena Senior Procurement and Supplies Officer (SPSO) Hirtrude Jisenge Internal Auditor (IA), Public Procurement Regulatory Authority (PPRA) Colnely Mwamukonda Procurement and Supplies Officer (PSO), Public Procurement Regulatory Authority (PPRA) Fenias Manasseh Public Relation Officer (PRO), Public Procurement Regulatory Authority (PPRA) Coleta Mmjamani Director Corporate Service (DCS)/Project Coordinator, Public Procurement Regulatory Authority (PPRA) Hannah Mwakalinga Head Planning Unit, National Audit Office Tanzania (NAOT) Natima M.C. Nkiliji￾wando Principle Economist, National Audit Office Tanzania (NAOT) Pole John Magesa Economist, National Audit Office Tanzania (NAOT) Josephina Melkiol Economist, National Audit Office Tanzania (NAOT) Ali Kasingo STAR Committee Member, Southern Africa Human Rights NGO Network (SAHRINGON) Tanzania Chapter Seif Mahumbi Member, Community Partnership Development Association (COPADEA) Emanueli Maanda￾mano Frank Kaduma Project Coordinator, T-MEMS (Mitchell Group) Dr. Rugemeleza A.K. Nsala Executive Director, Lawyers Environmental Action Team Runyaro Adolf Lawyers Environmental Action Team Richard Mushe Member, Former Commission on Local Government Reform Programme STAR EVALUATION—DRAFT REPORT 49 Kapinga Executive Director, Tanzania Council for Social Development (TACOSDE) Mathew Kihulya Executive Director, Governance Programme, Irish Aid STAR EVALUATION—DRAFT REPORT 50