DISCLAIMER This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. DRG LEARNING, EVALUATION, AND RESEARCH (DRG-LER) II ACTIVITY USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE EVALUATION Prepared under Contract No.: GS-10F-0033M / Order No. 7200AA18M00016 Tasking No. 027 DRG LEARNING, EVALUATION, AND RESEARCH ACTIVITY II USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE EVALUATION (FEBRUARY 2023) Prepared under Contract No.: GS-10F-0033M / Order No. 7200AA18M00016 Tasking No. 027 Submitted to: Matthew Baker Submitted by: Betty Alvarado de Lozan Ingrid Rojas Arellano Janet Porras Mendoza Omar Narrea Contractor: NORC at the University of Chicago 4350 East West Highway, 8th Floor Bethesda, MD 20814 Attention: Matthew Parry Tel: 301-634-5444; E-mail: parry-matthew@norc.org DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | I TABLE OF CONTENTS ACRONYMS.......................................................................................................................................II ABSTRACT.........................................................................................................................................III EXECUTIVE SUMMARY................................................................................................................. IV I. BACKGROUND.......................................................................................................................1 II. EVALUATION PURPOSE AND EVALUATION QUESTIONS .....................................3 III. EVALUATION METHODOLOGY AND LIMITATIONS ...............................................6 IV. FINDINGS..................................................................................................................................9 V. CONCLUSIONS.................................................................................................................... 38 VI. RECOMMENDATIONS....................................................................................................... 42 ANNEX I. INTEGRITY STANDARDS....................................................................................... 44 ANNEX II. STATUS OF SELECTED PROJECTS (APRIL 2022)............................................ 47 ANNEX III. LOGIC MODEL OF THE INTEGRITY NETWORKS’ STRATEGY .............. 49 ANNEX IV. FACTORS RELATED TO THE INTEGRITY NETWORKS’ FUNCTIONING ............................................................................................................................. 50 ANNEX V. EVALUATION METHODOLOGY ....................................................................... 51 ANNEX VI. REGRESSION RESULTS ......................................................................................... 54 ANNEX VII. DOCUMENTS REVIEWED .................................................................................. 55 ANNEX VIII. LIST OF KEY INFORMANT INTERVIEWS AND FOCUS GROUP DISCUSSIONS ................................................................................................................ 58 ANNEX IX. DATA COLLECTION INSTRUMENTS ............................................................. 62 ANNEX X. TEAM MEMBERS....................................................................................................... 89 LIST OF TABLES Table 1. Number of KIIs and online survey responses by region ..........................................................................V Table 2. Number of KIIs and online survey responses by region ..........................................................................6 Table 3. Number of companies in the construction sector by region (2021)..................................................14 Table 4. Members of the Integrity Networks (September 2021 and June 2022).............................................16 Table 5. MCLCP’s annual approved budget (2019–2022)......................................................................................32 Table 6. Number of interviews per type of organization.......................................................................................52 CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | II ACRONYMS CAPECO Cámara Peruana de la Construcción (Peruvian Chamber of Construction) CGR Contraloría General de la República (Office of the Comptroller General of the Republic) CONFIEP Confederación Nacional de Instituciones Empresariales Privadas (National Confederation of Private Business Institutions) COVID-19 Coronavirus Disease 2019 CUI Código Único de Inversión (Unique Investment Code) CRA Comisión Regional Anticorrupción (Regional Anticorruption Commission) CSO Civil Society Organization DiD Differences-in-Differences DRG Democracy, Human Rights, and Governance DRG-LER Democracy, Human Rights, and Governance Learning, Evaluation, and Research Activity FGD Focus Group Discussion IN Integrity Network IP Implementing Partner IS Integrity Standard KII Key Informant Interview MCLCP Mesa de Concertación para la Lucha contra la Pobreza (Roundtable for the Fight against Poverty) MEF Government of Peru’s Ministry of Economy and Finance MEL Monitoring, evaluation, and learning OI Oficina de Integridad (Integrity Office) OSCE Organismo Supervisor de las Contrataciones del Estado (State Procurement Supervisory Body) PIP Proyecto de Inversión Pública (Public Investment Project) SDID Synthetic Differences-in-Differences SIP Secretaría de Integridad Pública (Secretariat of Public Integrity) TPI Transparent Public Investment Activity USAID United States Agency for International Development USAID/Peru USAID Mission in Peru CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | III ABSTRACT The purpose of this midline evaluation is to determine the efficacy of the Transparent Public Investment (TPI) Activity’s Integrity Networks (INs) in Peru. TPI created and trained INs to apply and monitor a set of Integrity Standards (ISs) to measure compliance with good practices during the procurement and construction of public investment infrastructure projects. The Evaluation Team (ET) used a mixed-methods design, combining qualitative and quantitative techniques. The ET administered an online survey, collected administrative data, and conducted key informant interviews, focus group discussions, and nonparticipant observations. The main findings by evaluation question (EQ) are: EQ1: Did TPI achieve its goals related to INs? To what extent is TPI’s support of INs contributing to the activity’s broader goals? • TPI created five INs with members from the public sector and civil society organizations, with equal participation of women and men. • The INs monitored compliance with 14 of 15 ISs for 9 of 10 selected projects in 2021. EQ2: What factors enabled or inhibited TPI from achieving its goals related to INs? • Enabling factors: IN members’ commitment, the use of open data, the digitalization of project information, and visits to the construction sites. • Constraints: members’ high turnover and lack of time and monetary resources. Technical issues during the pre-construction period also hindered the application of ISs and project delivery. EQ3: To what degree did TPI’s INs’ activities affect downstream outcomes? • The ET did not find significant effects of the INs on the projects’ budget execution, physical execution, or changes in investment costs. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | IV EXECUTIVE SUMMARY BACKGROUND The Transparent Public Investment (TPI) is a five-year activity (2020–2025) implemented by Chemonics International that seeks to strengthen the integrity of key government processes to reduce corruption in public investment and procurement in Peru. TPI is expected to: 1) strengthen appropriate controls and supervision mechanisms to reduce corruption; 2) enhance civic oversight of public investment processes; and 3) build effective alliances with private sector firms to promote public integrity and combat corruption. The TPI timespan included initial “co-creation” and “proof-of-concept” phases, before a “scale-up” phase. During the proof-of-concept phase, TPI supported the establishment of Integrity Networks (INs) in five regions (Cusco, Lambayeque, Lima, Loreto, and Piura) and promoted the collaboration of diverse stakeholders (public sector institutions, business organizations, civil society organizations [CSOs], and individual citizens) in each IN. TPI trained INs to apply and monitor a set of Integrity Standards (ISs) during the procurement and construction of two infrastructure projects in each region. The ISs measure compliance with good practices during those processes (see Annex I). The INs aim to increase accountability through action plans focused on the use of INs to monitor selected public investment projects. EVALUATION PURPOSE AND QUESTIONS The purpose of this evaluation is to determine the efficacy of TPI’s INs. This midline evaluation provides findings from TPI’s proof-of-concept phase to inform the development of the scale-up phase. The proposed evaluation included this midline evaluation and an endline evaluation in 2024. The evaluation questions are: (1) Did TPI achieve its goals related to INs? To what extent is TPI’s support of INs contributing to the activity’s broader goals? • Immediate goals are: i) to create INs comprising public and private sector institutions, CSOs, and citizens, ensuring that these networks include women who promote anticorruption actions; and ii) for INs to improve the compliance of selected public investment projects (Proyectos de Inversión Pública, PIPs) with ISs. • The broader goals look for: the adoption of tools, processes, and codes of ethics. Likewise, they also include the adoption of integrity standards and an anti-bribery model by the private sector and the inclusion of research journalists monitoring public investment projects. (2) What factors enabled or inhibited TPI from achieving its goals related to INs? This question explores: • the conditions or factors under which INs can successfully monitor the ISs CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | V • the causal pathways through which the implementation of INs leads to increased citizen oversight, improved government procurement processes, and greater integrity • if the current IN members are likely to maintain coordination in the future • if there are resources available for the INs • if there are synergies between the INs and other CSOs overseeing public investments (3) To what degree did TPI’s INs’ activities affect downstream outcomes? For the midline evaluation, the Evaluation Team (ET) measured the following outcomes from the execution phase: • Completion of projects: Budget and physical execution • Total cost of projects: Changes in investment cost as a percentage of budget approved in feasibility studies METHODS The evaluation employs a mixed-methods design, combining qualitative and quantitative methods to collect and analyze information. For the qualitative analysis, the ET administered 55 key informant interviews (KIIs), 5 focus group discussions (FGDs) with IN members (1 per region), and an online survey (39 responses from 70 IN members) and conducted nonparticipant observations during 10 network meetings (two in each region). In addition, the ET collected procurement, budgetary, and other project data from treatment and comparison projects for the quantitative analysis. The ET also reviewed available project documentation and performance monitoring data. Table 1 shows the number of KIIs and online survey responses by region. Annex V presents details of the evaluation methodology. Table 1. Number of KIIs and online survey responses by region Piura Lambaye que Lima Province Loreto Cusco Lima city Total Key informant interviews 9 11 6 9 8 12 55 Online survey responses 6 10 6 9 8 -- (*) 39 Note: The ET conducted KIIs with specialists from the TPI project (including regional coordinators) and the national government and IN members. The online survey was administered to former and current IN members from CSOs and the private and public sectors in the five regions with INs. (*)Lima city hosts TPI’s headquarters but does not have an IN. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | VI FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1. ACHIEVEMENT OF GOALS INTEGRITY NETWORKS’ PURPOSE AND MEMBERSHIP • TPI has successfully supported the creation of five INs as planned. • The common good (fighting corruption) and commitment prevails over particular interests, especially for civil society representatives. • The number of member organizations has increased from September 2021 to June 2022. The goal to have a diverse membership with CSO and public members has progressed while the private sector membership still needs to grow. Organizations like the professional associations of engineering and journalism and public sector practitioners are seen by IN members as key members or partners to foster the networks’ development. • TPI promotes women’s participation and active voice in anticorruption activities. The number of women involved in the INs increased from 21 in September 2021 to 40 in June 2022. Female members hold key posts to promote citizens’ oversight. The Piura, Lambayeque, Cusco, and Loreto regions had or have women from the Roundtable for the Fight against Poverty (Mesa de Concertación para la Lucha contra la Pobreza, MCLCP) as Technical Secretariats. • The MCLCP, a network that gathers public and CSO organizations, has had an essential role in articulating the work of the INs due to its legal status and positive public perception. SELECTED PROJECTS • CSO members would like to participate in the selection of projects to be monitored in the future. • There was a consensus among respondents that the Technical Construction Studies (expedientes técnicos) are poorly designed or outdated. This affects the projects’ execution and limits the application of the ISs. • The selected projects are similar to each other in terms of cost increase and delays. However, those executed via direct public procurement, in comparison to those with private contracting, require more effort to monitor the procurement of a larger number of services and goods. MONITORING TOOLS • INs adopted ISs to oversee infrastructure projects. IN members were able to apply or monitor ISs that required information from governmental open data or practical activities like the civil works site visits. In contrast, a set of ISs is more difficult to apply or monitor because it requires specialized analysis from procurement or investment practitioners. • The ISs are used to press for transparency from the regional government, but members also expect to influence project delivery and avoid construction halts. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | VII • Having infrastructure experts in the INs also helps the non-expert members to embrace the ISs. • IN members welcomed the application and monitoring of the Preventive Capacity Index (PCI), backed by the Public Integrity Secretariat. The PCI provides a comprehensive view of the regional government status and plans to improve their integrity. Members showed interest in learning more about the State Procurement Supervisory Body’s (Organismo Supervisor de las Contrataciones del Estado, OSCE) risk management approaches. • Members recognized the usefulness of digital tools, like the regional government of Piura’s IS platform, to monitor compliance with information provided by the same regional government, the quick response (QR) code to visualize updated and comprehensive project information, and the Ojo en la Obra platform that presents a simplified version of the Contraloría General de la República’s (CGR’s) control reports. TRAINING AND COACHING • Even though the members considered the procurement process complex, they valued the training they received and looked forward to more case studies. Complementary practical training activities like site visits also helped the IN members understand the ISs. • IN members have a favorable opinion about the training on open information sources used to monitor the regional project’s compliance of ISs. These online sources allowed members to collect and analyze more information than what was provided by regional governments to monitor the selected projects. EVALUATION QUESTION 2. ENABLING AND LIMITING FACTORS ENABLING FACTORS • National preventive and anticorruption policies convey a positive environment for the discussion of integrity in the public sector by different stakeholders, including civil society. • The negative national and regional political environment has triggered civil society’s commitment to oversee PIPs. • Modernization of the control system, with the application of CGR’s preventive actions like the concurrent control since 20171 and the receptiveness of the CGR’s participation and feedback strengthened the INs’ activities and oversight tools. • The INs have successfully gathered members with diverse backgrounds, complementary skills, and a shared purpose. • The commitment and dedication of CSO members is one the main assets of the INs. • IN members value the experience and background of the MCLCP and the technical support of TPI. 1 CGR. (2021). La reforma del control gubernamental en el Perú. Balance al trienio de su implementación. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | VIII • IN members’ previous oversight experience is an asset. • INs successfully brought youth organizations into their networks, and those youth organizations brought technical capacities to the INs. • The efficacy of the Integrity Office (Oficina de Integridad, OI) has been positive in regions with a specialized office in the regional government structure. HINDERING FACTORS • The lack of a tradition of accountability in regional public agencies increased the risk of ineffective use of public resources and a lack of transparency, leading to CSOs’ frustration. • High turnover of regional government staffers and CSO members limited networking and capacity￾building. • Lack of incentives and the limited dissemination of the IN’s achievements could limit the participation of CSO representatives in the future. • Members’ limited availability and lack of monetary resources for overseeing activities have a negative effect on CSOs’ participation. PREVIOUS OVERSIGHT EXPERIENCE • There is a potential synergy between CGR’s Monitores program and the INs’ work, as both encourage trained CSOs to oversee public works. The main difference is time spent in each infrastructure project: Monitores specialize in observational visits to several construction sites, while the IN members cover all the procurement process, including site visits of two selected projects. INSTITUTIONALIZATION AND FUTURE COLLABORATION • In one regional government, the OI, as a member of the network, has institutionalized the ISs tool. • TPI has designed and implemented digital tools that could be used in other oversight programs. • Members have expressed interest in continuing to use the ISs. While dissemination and collaboration among networks and regional governments might happen in the future, there is no clear path for more institutionalization at the time of this midline evaluation. EVALUATION QUESTION 3. EFFECTS ON OUTCOMES • The ET evaluated budget execution, physical execution, and changes in investment costs and did not find significant effects of the INs on these three outcome variables that proxy for corruption. We did not expect to find statistically significant differences at midline due to the small number of treatment projects and the limited amount of time INs have been working. We plan to repeat this analysis at endline (with more data) as the INs might need more time to affect these outcome variables. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | IX • The budget and physical execution of the treatment projects (i.e., those monitored by the INs) are not significantly different from that of comparison projects selected in the same regions and in other regions without INs. However, the increases in the total investment costs of the projects monitored by the INs are higher than of comparison projects in regions without INs. RECOMMENDATIONS OVERALL • Promote the establishment of specialized OIs outside the regional governments’ human resources office. This model has benefited the work of regional governments and the INs. Members doubt that a general manager or human resources office would have enough time to deal with all the integrity responsibilities of a regional government. • Working along with national agency stakeholders, explore the implementation of an incentive program that rewards more transparency and accountability in regional and municipal governments. MEMBERSHIP • Include the Regional Council (Consejo Regional) as a potential member of the network as it has regulatory and control functions over the regional government’s executive branch. • Invite private sector actors like the Regional Chapter of the Chamber of Commerce and Production to the networks. As users of the infrastructure projects, their interest is related to their needs to reduce costs and increase productivity. • Promote a strong involvement of representatives of the Professional Association of Engineers and other members with sound technical expertise in the infrastructure field, so they can share their practical knowledge with non-expert members and identify the projects’ main problems. SELECTED PROJECTS • Involve the IN members in the selection of projects to promote their participation in the networks. • Examine how the projects foster regional economic growth to select the ones that the IN will monitor. Explore if the ISs can capture the need to have an updated Technical Construction Study before the construction phase. MONITORING TOOLS • The Technical Secretariat could carefully explain to the members that the monitoring tools seek to elevate transparency and good practices in procurement but do not guarantee faster and more efficient project execution. This will prevent members’ frustration with delays during the project execution. • Support the exchange of initiatives and information among networks, including the design and creation of the QR that summarizes information on the projects’ execution stage and the Ojo en la CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | X Obra platform with user-friendly information of the CGR’s concurrent control reports. Support the dissemination of the ISs platform developed by the Piura OI to other regions • Continue supporting the PCI. The PCI gives a comprehensive view of the preventive processes and tools of the public entities and is supported by the Prime Minister Office’s bylaws. Considering that some ISs are harder to monitor than others, the PCI is an adequate tool that can be monitored by all members. • Look for synergy with the CGR’s Monitores program to gain expertise and share costs during project site visits. TRAINING AND COACHING • Develop guidelines on how to apply and monitor each standard. Include who should apply and who should monitor the ISs and how to access information sources. Guidelines should also define goals, indicators, and metrics. • Personalize training sessions based on each member’s diverse academic background and work experience. • Prepare case studies on cancelled or delayed projects to educate members on how to deal with or avoid those issues in future projects. • Share lessons learned with new regional governments (elected in October 2022). RESOURCES • An institution like the MCLCP could host the INs if they decide to keep working after TPI ends its operations, and the MCLCP is able to leverage funds. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 1 I. BACKGROUND Under the Democracy, Human Rights, and Governance – Learning, Evaluation, and Research (DRG-LER) II Activity, the United States Agency for International Development (USAID) has requested NORC at the University of Chicago to design and implement an evaluation of the Transparent Public Investment (TPI) Activity in Peru. TPI is a five-year activity (2020–2025) implemented by Chemonics International that seeks to strengthen the integrity of key government processes to reduce corruption in public investment and procurement in Peru.2 Specifically, TPI is expected to: 1. Strengthen appropriate controls and supervision mechanisms to reduce corruption, 2. enhance civic oversight of public investment processes; and 3. build effective alliances with private sector firms to promote public integrity and combat corruption.3 TPI’s approach was to support the establishment of Integrity Network (INs), which aimed to find novel ways for actors to collaborate to improve procurement and public investment processes. The INs would involve a diversity of stakeholders in each region, including public sector institutions, business organizations, civil society organizations (CSOs), and individual citizens. Its goals are to formalize actors’ commitments, incentivize collaboration, and increase accountability through action plans focused on prioritized public investment projects (Proyectos de Inversión Pública, PIPs). The TPI timespan included an initial “co-creation phase” and a “proof-of-concept” phase. After these two initial phases ended in August 2022, TPI will scale up to new regions and expand the core of its work. During the co-creation phase (January–September 2020), TPI engaged in a dynamic interaction with stakeholders to ensure a mutual understanding of the problem and to validate a group of prioritized regions and PIPs. During the proof-of-concept phase (September 2020–August 2022), the project supported INs in five regions (Cusco, Lambayeque, Lima, Loreto, and Piura), and each IN monitored two public investments within each region. Subsequently, during the scale-up phase (September 2022–August 2024), the project will expand to support additional INs in other regions and monitor more public investments in each region. TPI developed a set of 15 Integrity Standards (ISs) or milestones during the procurement and execution processes of the PIPs. The standards cover three phases: 1) the preparation for public bidding: i.e., market studies and technical specifications; 2) the selection process: i.e., announcements, queries, review of queries and answers, final announcement, evaluation of proposals, and contractor selection; and 3) the contractual execution phase: i.e., contract signing, budgeting, construction, and final acceptance of the infrastructure work. 2 TPI design was guided by the following theory of change: “If stronger government control and supervision are combined with public integrity, and information/feedback channels are open, and companies comply with anti-corruption standards in procurement, then there will be a greater likelihood that corruption will be prevented, detected and sanctioned, leading to a reduction in the overall occurrence of corruption in public investment, and therefore an improvement in public service management and quality in Peru.” See: USAID. (2020, August 3). Transparent Public Investment Project. Monitoring, Evaluation, and Learning Plan. V.3. p. 3. 3 Contract 72052720C00002 SECTION C – PWS. Section C - Description / Specification /Performance Work Statement. C.1. Title of Activity Transparent Accountable Public Investment (TAPI) Activity (n.d.), 23 pages. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 2 ISs are applied to the PIPs prioritized by the regional governments. TPI and the regions selected projects based on the following criteria: 1. Projects with an investment cost higher than 10 million soles and with budget available in 2021 2. Projects with a Unique Investment Code (Código Unico de Inversión, CUI) that close gaps related to health, education, agriculture and infrastructure 3. Projects that benefit vulnerable populations 4. Projects that can be part of the General Comptroller’s Office (Contraloría General de la República, CGR)’s Control Simultáneo 5. Projects validated by the regional governments In practice, only projects in the health, education, and transportation sectors were selected. Also, all projects benefit vulnerable populations, typically identified by feasibility studies or documentation declaring a state of emergency that requires these projects to be implemented. Additionally, the projects’ construction sites are close to urban areas (to facilitate the fieldwork of IN members). Before the INs started functioning, one project was in the preparatory activities phase, and nine were in the execution phase. Moreover, four (two from Piura and two from Lambayeque) projects were part of the Reconstruction Agency (Autoridad de Reconstrucción con Cambio, ARC4) portfolio. As of January 2022, 8 of 10 original projects were audited by the CGR’s control simultáneo/concurrente models.5 As of June 2022, 9 of the 10 original projects remained under the INs’ monitoring, and 2 additional projects were added in Piura. Annex II includes more information about the projects prioritized in the five regions. PROJECT LOGIC Annex III presents a logic model that depicts how the INs’ implementation and function support the reduction of corruption in the procurement processes of PIPs. According to the project’s logic, organized multi-stakeholders networks received training and coaching in integrity, procurement processes and tools like the ISs. That training and coaching strengthened their capacities to apply and monitor public procurement and investments projects. Public stakeholders, represented by the regional government, would apply the standards, while an informed civil society would monitor compliance with the standards. The network would create an environment of dialogue, and the regional government would be accountable to the network’s members and society. One specialist in Lima city commented: “The model looks for a new citizen’s oversight practice…. Networks open the door to increased civil society knowledge.” 4 Central government agency created in 2017 to coordinate the reconstruction process after El Niño in the Northern Coast and Lima. 5 This audit model has been implemented in Peru since September 2017 and has three modalities: concurrent audit, control visits, and guided mandate (control concurrente, visitas de control, y orientación de oficio). The auditing visits, with a preventive approach, look for gaps that, if not solved, could turn into risks of non-completion or lead to substandard infrastructure quality. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 3 II. EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of this evaluation is to determine the efficacy of TPI’s INs in Peru. The evaluation of the INs considers their distinctive characteristics and particular strategies to promote the application of the ISs by the regional governments. Additionally, the Evaluation Team (ET) will consider previous initiatives that fostered the oversight of public monies and investments in the regions. The proposed evaluation includes this midline evaluation and an endline evaluation in 2024. EVALUATION QUESTIONS This evaluation addresses three main questions: 1. Did TPI achieve its goals related to the INs? To what extent is TPI’s support of INs contributing to the activity’s broader goals? 2. What factors enabled or inhibited TPI from achieving its goals related to INs? 3. To what degree did TPI’s INs’ activities affect downstream outcomes? EVALUATION QUESTION 1. DID TPI ACHIEVE ITS GOALS RELATED TO THE INTEGRITY NETWORKS? TO WHAT EXTENT IS TPI’S SUPPORT OF INTEGRITY NETWORKS CONTRIBUTING TO THE ACTIVITY’S BROADER GOALS? The first question relates to whether INs performed as intended and whether they contributed to broader activity goals. TPI’s immediate goals related to the INs are: 1. To create INs comprised of public sector institutions, business organizations, CSOs, and citizens, ensuring that these networks are inclusive of women. 2. For INs to improve selected PIPs’ compliance with ISs. The evaluation focuses on the analysis of the following performance indicators: • Number of selected PIPs that exceed the compliance practices of the public procurement ISs • Number of joint public-private action plans implemented among IN actors to oversee prioritized investment projects. • Number of public ISs applied to PIPs and promoted by INs. • Number of women participating in INs and promoting anti-corruption actions. In turn, according to TPI’s results framework, INs are expected to contribute to broader goals, monitored by the following eight performance measures: • Public investment processes, tools, and/or protocols that reduce corrupt practices become government of Peru rules that are widely disseminated and adopted by Peru’s national and local public entities. • Emblematic public investments follow the procurement standards and identify lessons learned and best practices to be replicated in other similar procurement processes. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 4 • An IN composed of key government of Peru actors, CSOs, and business associations is operational with clear plans to oversee and improve public investment integrity at all stages. • National research journalist network (infomediaries) is trained to investigate and monitor emblematic PIPs, report risks of corruption, and disseminate best practices. • INs are aware of and use access to information mechanisms and best practices through an open government lens. • At least two major business associations—Confederación Nacional de Instituciones Empresariales Privadas (CONFIEP), Cámara Peruana de la Construcción (CAPECO), and targeted regional chambers of commerce—adopt and implement integrity compliance standards and publicize periodic reports, including on performance measurement and success stories. • Voluntary codes of ethics and other integrity initiatives are designed and adopted in prioritized sectors and regions. • The anti-bribery model developed by Peruvian business associations for big companies is adapted to medium and small businesses, particularly in prioritized sectors and regions. EVALUATION QUESTION 2. WHAT FACTORS ENABLED OR INHIBITED TPI FROM ACHIEVING ITS GOALS RELATED TO THE INTEGRITY NETWORKS? The second question asks about the conditions under which TPI’s INs can be successful and the causal pathway(s) through which INs achieve their goals. The specific sub-questions to be addressed are: • What conditions/factors help INs be successful? • What conditions/factors hinder INs’ success? • What are the causal pathways through which IN implementation leads to increased citizen oversight, improved government procurement processes, and greater integrity? • Are the current actors of the INs likely to maintain coordination and collaboration to oversee public investments in the future? Have they institutionalized and/or shared their practices, norms, and/or procedures to promote integrity after the TPI ends? What happens to the lessons built by the INs when the selected projects’ execution is finalized and delivered to the government? • What are the monetary and non-monetary resources the networks use to perform their activities and what share of these resources is the TPI project responsible for? Are these resources going to be available after the monitored projects or the TPI project end? • How do the INs compare to Programa Monitores Ciudadanos de Control organized by CGR and to the Veedurias Ciudadanas of the Citizen Engagement for a Transparent Reconstruction Program? Are there synergies or overlaps? Are the members of these later initiatives also participating in TPI’s INs? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 5 EVALUATION QUESTION 3. TO WHAT DEGREE DID TPI’S INTEGRITY NETWORKS’ ACTIVITIES AFFECT DOWNSTREAM OUTCOMES? For the midline evaluation, the ET measured the following outcome variables that proxy for corruption from the execution phase:6 1. Completion of projects − Budget execution, measured as the total budget spent by the regional government as a percentage of the total investment cost. We expect treatment projects to have higher budget execution rates than comparison projects, which would indicate that the regional government is spending available funds to implement the selected projects. (In Peru, most regional and local governments lack the capacity to spend all available funds that are transferred to them in a given fiscal year.) − Physical execution, as reported by the regional governments’ investment offices. We expect the treatment projects to have higher physical execution rates than comparison projects, showing that funds are being used to increase available infrastructure (and possibly equipment), as all analyzed projects are infrastructure projects. We measure this indicator because budget execution alone does not provide enough information about project implementation. 2. Total cost of projects − Changes in investment cost as a percentage of the budget approved in the feasibility studies. We expect the treatment projects to have increases in the total investment cost that are lower, on average, than the cost increases of the comparison projects. While increases in the investment costs happen due to increases in the costs of materials and labor, we expect the regional governments to avoid cost overages that the IN members could detect when they monitor the selected projects. 6 The ET had originally proposed to measure front-end outcomes from the procurement process. However, only 1 of the 10 originally prioritized projects was in the procurement phase when the intervention started. Therefore, the midline evaluation focuses on back-end outcomes from the execution phase. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 6 III. EVALUATION METHODOLOGY AND LIMITATIONS The evaluation used a mixed-methods design, combining qualitative and quantitative methods to collect and analyze information. For the qualitative analysis, the ET administered 55 key informant interviews (KIIs) with USAID/Peru and TPI staff, TPI consultants and partners, government counterparts, IN members, and other stakeholders; 5 focus group discussions (FGDs) with CSO network members, and an online survey (with 39 responses from 70 target respondents). The ET also conducted 2 non￾participant observations during the IN meetings in each region. Table 2 shows the number of KIIs and online survey responses by region. Table 2. Number of KIIs and online survey responses by region Piura Lambayeque Lima Province Loreto Cusco Lima city Total Key informant interviews 9 11 6 9 8 12 55 Online survey responses 6 10 6 9 8 -- (*) 39 Note: The ET conducted KIIs with specialists from the TPI project (including regional coordinators) and the national government and IN members. The online survey was administered to former and current IN members from CSOs and the private and public sectors in the five regions with INs. (*) Lima city hosts TPI’s headquarters but does not have an IN. In addition, the ET collected procurement, budgetary, and other project data from treatment and comparison projects for the quantitative analysis. The ET also reviewed available project documentation and performance monitoring data. By drawing on these multiple sources of information, the ET was well equipped to report on the findings and develop conclusions and recommendations. The ET designed a conceptual framework to organize and analyze the qualitative information. Annex V presents more details of the evaluation methodology. CONCEPTUAL FRAMEWORK The proposed framework7 has two dimensions, as represented in Annex IV. The framework fits different institutions with voluntary participation and horizontal decision-making processes. The first dimension considers the evolution the networks experience to accomplish their goals. The networks evolve and pass from a phase of contact-building and information-sharing to a phase where the members coordinate and increase involvement. The members provide feedback on the services or tasks in which they are involved and expect to participate in the planning and decision￾7 The conceptual framework is an adaptation of the following documents: 1. Collective Minds. (2021). What capacities do networks need to succeed?; 2. Leading Inside Out & Collective Impact Forum. (2017). Community engagement toolkit; and 3. ORSIMPACT & Spark Policy Institute. (2018). When collective impact has an impact. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 7 making processes. As these two phases progress, members execute the action plan together, and they collaborate to enhance the regional governments’ openness to and compliance with the ISs. The second dimension considers the internal elements and resources that a network is expected to have to function successfully, as well as the external factors that affect their work. INTERNAL FACTORS MEMBERSHIP. The IN has a preferred membership structure regarding the representation of public entities, private companies, and CSOs. The IN might express the need to incorporate other stakeholders to meet a goal. PURPOSE. The IN members should support a common goal. Each group may have its own interests, which is not problematic as long as they overlap on relevant shared goals, such as fighting corruption. Each group may join the network for different reasons, but the participation of all members should lead to fulfilling the IN’s purpose. LEADERSHIP AND GOVERNANCE. Networks are self-governed, but they could have basic rules. For example, the IN members may need to reach a consensus to produce an announcement regarding public policies, or they may need to define how to approve an action plan. OPERATIONS AND RESOURCES. Besides the leadership, the networks need a management structure, financial resources, and members’ time dedicated to the network, among others. EXTERNAL FACTORS Bureaucratic rules may make it more difficult for INs to fulfill their mission. The network must work with regional organizations, which are accustomed to the status quo of little or no citizen oversight. Nevertheless, the INs also have direct or indirect support from other public agencies, their policies, and institutional bylaws. QUANTITATIVE ANALYSIS The ET measured the effect of the INs by comparing the outcomes of the prioritized projects in the five proof-of-concept regions with those of similar projects in other regions that did not establish INs. In addition, we examined whether the INs have spillover effects on non-selected investment projects by comparing selected and non-selected projects in the proof-of-concept regions. Annex V describes the methodology followed to identify the comparison projects. The ET used monthly administrative data to measure the effects of the INs on the outcome variables. In the case of the physical execution and project cost variables, the ET used data that we collected monthly from December 2021 to May 2022. The INs were already monitoring the selected projects by the time we started gathering this information. We calculated the INs’ effects using random effects models. For the budget execution outcome, we used monthly data available since January 2020. We decided to exploit the availability of budgetary data before and after the creation of the INs employing a Differences-in-Differences (DiD) methodology. The INs were created between December 2020 (in Cusco and Lambayeque) and June 2021 (Piura); the networks in Lima and Loreto were created in CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 8 February 2021. Given that the treatment did not start at the same time in all regions, the ET decided to use a staggered treatment design.8 As proposed in the Evaluation Design Report, the ET implemented the Callaway and Sant’Anna’s Difference-in-Differences with Multiple Time Periods (CSDID) methodology using Stata’s csdid command. The ET decided to also use the Synthetic Difference-in￾Differences (SDID)9 estimator because our comparison groups violated the parallel trends assumption necessary to use the CSDID estimator.10 SDID statistically constructs comparison groups that satisfy the parallel trends assumption, so it is a good estimator when the parallel trends assumption is initially violated. The ET implemented the SDID methodology using Stata’s sdid command.11 We report the results of both methods in Annex VI. LIMITATIONS AND MITIGATION STRATEGIES PUBLIC SERVANTS’ TURNOVER AND ROTATION AFFECTED THE INTERVIEWS The ET could not administer in-person interviews with two CGR regional managers (Cusco and Lambayeque) but was able to reschedule one remotely. Also, the team was able to interview a new regional CGR manager, who had enough knowledge and interest in the network model. At the national level, two high-level civil servants connected to preventive and regulation agencies stepped down from their positions in their respective public agencies, and it was not possible to connect with them. The team replaced those two interviews with two other senior managers who provided substantial information for the research The collected information was sufficient and useful; there was no impact on results. SECURITY MEASURES LIMITED THE ADMINISTRATION OF THE ONLINE SURVEY Some target respondents did not receive the links to the web-based survey due to security measures against unwanted emails implemented by their organizations. The ET collected the IN members’ secondary email addresses and re-sent the survey. The ET also followed up via text messages to increase the number of responses. Out of 70 IN members, 39 answered the survey. Considering the small universe of 70 members from five networks, a 56 percent could be considered a good level of response. 8 This methodology extends the canonical 2x2 DiD strategy with only two groups (control and treatment) and only two periods (pre- and post-intervention) to an application with several time periods and when the intervention may start at different moments for the treated units. 9 Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W., & Wager, S. (2021). Synthetic difference-in-differences. American Economic Review, 111(12), 4088-4118. 10 Callaway and Sant’Anna’s estimator assumes the treatment and control projects would have parallel trends in the absence of the treatment. If treatment and control projects have parallel trends, then control projects are a reasonable counterfactual for what would happen in the treatment projects if they had not been treated; the difference between treatment projects and control projects is thus the effect of treatment. However, if treatment and control projects do not have parallel trends, then control projects are not a reasonable counterfactual for what would happen in the treatment projects; consequently, the difference between treatment and control projects is not the effect of treatment. The ET tested the parallel trends assumption using the pretrend postestimation option of Stata’s csdid command. The test rejected the parallel trends assumption for the in￾region comparison and the out-region comparison, indicating that the control projects are not a good estimate of the treatment projects if they had not been monitored by the INs. 11 More information available at https://github.com/Daniel-Pailanir/sdid CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 9 IV. FINDINGS EVALUATION QUESTION 1. DID TPI ACHIEVE ITS GOALS RELATED TO THE INTEGRITY NETWORKS? TO WHAT EXTENT IS TPI’S SUPPORT OF INTEGRITY NETWORKS CONTRIBUTING TO THE ACTIVITY’S BROADER GOALS? The research explored whether INs performed as intended and whether they contributed to broader activity goals. We present our findings under each of TPI’s immediate goals. GOAL A. TO CREATE INTEGRITY NETWORKS COMPRISED OF PUBLIC SECTOR INSTITUTIONS, BUSINESS ORGANIZATIONS, CSOS, AND CITIZENS, ENSURING THAT THESE NETWORKS ARE INCLUSIVE OF WOMEN For this goal, the report reviewed two results: • Number of joint public–private action plans implemented among IN actors to oversee prioritized investment projects. • Number of women participating in INs and promoting anti-corruption actions. The analysis focuses on the areas covered by the following performance measures (PMs): P.M. 2.1. An integrity network composed of key government of Peru actors, CSOs, and business associations operational with clear plans to oversee and improve public investment integrity at all stages. P.M. 2.2. National research journalist network (infomediaries) trained to investigate and monitor emblematic PIPs, report risks of corruption, and disseminate best practices. P.M. 2.3. INs that are aware of and use access to information mechanisms and best practices through an open government lens. P.M. 3.1. At least two major business associations (CONFIEP, CAPECO, and targeted regional chamber of commerce) adopt and implement integrity compliance standards and publicize periodic reports, including on performance measurement and success stories. P.M. 3.2. Voluntary codes of ethics and other integrity initiatives are designed and adopted in prioritized sectors and regions. P.M. 3.3. The anti-bribery model developed by Peruvian business associations for big companies is adapted to medium and small businesses, particularly in prioritized sectors and regions. TPI has successfully supported the creation of five INs, one in each selected region, and promoted national policies and tools benefitting INs’ mission. While all networks have been functioning, only the one in Lima Province region has designed a joint public–private plan because the CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 10 chamber of commerce is a member of the IN.12 Ahead, the ET describes the findings related to membership, the IN’s plans, and activities conducted to oversee the projects. The networks have implemented different strategies using new tools besides the ISs, have made efforts to communicate with the government, and have incorporated new members, like youth organizations and direct infrastructure users.13 Additionally, the network’s purpose and a review of the membership structure underlined the interest of the past and current committed individuals. The division of tasks among members also revealed how members undertook the plans and post-actions and how diverse members interacted in each organization (P.M. 2.1, 2.2). Additionally, TPI has worked to improve the context that has affected the operation of the networks, like working with the Secretariat of Public Integrity (Secretaría de Integridad Pública, SIP), CGR, and the State Procurement Supervisory Body (Organismo Supervisor de las Contrataciones del Estado, OSCE). TPI embarked on a constant training program that allowed members to understand the ISs, the tools behind the application or monitoring of the standards (P.M. 2.3). The common good prevails over particular interests, especially for civil society representatives. Most respondents recognized the network as a forum or alliance where civil society can hold the regional government accountable for reducing corruption. Others saw the central objective as the opportunity to join forces with different CSO and the regional government, to fight corruption, or at least, they said, increase transparency. A public sector member in Piura explained: The purpose of the IN is clear: to make all levels of the public function transparent. At the regional level, now we are focused in public investment projects … but actually the goal’s to make everything that the regional governments are working on transparent for the citizens. The statements revealed variations that go from a general perspective of the principles to a more instrumental goal, but most members care for the excellent use of public resources. Some examples of the networks’ purpose are: • It pursues truthfulness, transparency, and efficiency from civil servants. • Having participant actors from three different spheres (public, private, and CSO) eliminates the belief that the INs are conducting a witch hunt14 against the regional government, enhances a better budget execution, and identifies potential risks of resource misappropriation. • CSO leads the oversight processes and promotes integrity practices to reduce corruption in public works. • The network is a regional space where members can influence the improvement of the procurement system, focusing on the areas of contractor selection, budgeting, project implementation, and post￾investment. 12 USAID (2021). Proyecto Inversión Pública Transparente. Informe Anual de Desempeño 2021. Pages 27-28. 13 USAID op. cit. Pages 6-7. 14 Negative political campaign CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 11 • The network monitors those projects, so they are completed without interference or difficulties. • The networks raise awareness and support for the strict compliance with ISs. • Networks provide evidence of adverse situations that could arise in the project and oversee if the standards are applied correctly during the contracting process. CSO members commented that the purpose of the IN was shared by most of the individuals. However, they recognized that some individuals “might have a particular interest,” like gaining watchdog experience, learning new tools, or using the future infrastructure.15 The recent inclusion of direct beneficiaries of the selected projects increased the interest in monitoring the construction of that investment. Likewise, a respondent commented that most members had shown commitment to the network, but a few might not have understood the value. Stakeholders’ opinions about the primary factors causing corruption and the approaches to prevent it influences how stakeholders view the network’s strategies and tools. Out from the interviews with different stakeholders, the ET identified three complementary views: one emphasizes the public agencies’ processes improvement and institutional development to enhance their transparency and culture of accountability; the second focuses on the efficacy of citizen oversight; and the third one emphasizes the training of civil servants, including on ethical behavior. TPI works currently around these three topics; together with an informed citizen oversight, members apply and monitor the ISs and support the Preventive Capacity Index (PCI), which in turn looks for the improvement of organizational transparency and accountability. TPI also expands its training programs to regional civil servants involved in the procurement areas, but specialists in Lima and Cusco and a CSO member in Lima Province commented that the regional government should also work on a comprehensive program of ethics and other capacity-building activities for civil servants. As a senior practitioner in Cusco said: “Corruption is a voluntary decision, independent of the salary level, so it was not an economic issue. Instead, it deals with their values.… I would prioritize improving the person’s ethics over the improvement of processes.” Servir, the public agency regulating the civil service, has approved the national policy of civil service training that includes integrity as a transversal topic. 16 Respondents also expressed their view on the use of the 15 ISs. One group of standards is based on OSCE’s rules, another group relates to the analysis of the project feasibility and technical construction studies, and others relate to the compliance of rules during the construction phase, supported by the CGR control reports. Specialists consider that the ISs approach acts as a preventive measure against corruption. A senior specialist in Lima explained that the ISs connected to the preparatory phase look for competitive and efficient acts during the contracting process. One senior practitioner in Cusco added that based on the importance that the networks give the CGR’s reports, the INs are basically an auditing initiative. 15 According to the conceptual framework, each group may have its own interests, which is not problematic as long as they overlap on relevant shared goals, such as fighting corruption. 16 Identifying the phases of planning, diagnosis, execution, and impact evaluation. One of the prioritized transversal topics is “ethics”: https://www.gob.pe/institucion/servir/informes-publicaciones/2920820-materias-de-capacitacion-transversales-gdc CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 12 The number of public and CSO member organizations has grown, but the goal of a diverse membership with private members is still in progress. The number of members and organizations they represent vary among regions. Overall, the number of organizations increased from 38 to 45 between September 2021 and June 2022. The number of public organizations also increased from 16 to 18, and the number of CSOs increased from 21 to 26 organizations in the same period. In the same line, the number of members has increased from 48 to 66. The composition of each IN has also changed. Some members have stopped attending the network activities due to the rotation of personnel in public offices and NGOs, taking time off to run for public office, or waiting to sign agreements with TPI. In addition, some initial members dropped out of the IN once they realized the activities were unpaid. The time that members have been involved in the INs varies. According to the online survey, 11 out of 39 respondents have been members for the last 18 months, 6 for the last year, and 6 for the last 6 months. The member turnover explains the need for constant training. The members from the public sector are representatives of the regional governments and other governmental organizations, such as the regional comptroller office, the public prosecutor's office, and the Ombudsman’s Office. The involvement of the regional governments in the IN has varied between regions. In one region, the integrity office (Oficina de Integridad, OI) recognized their participation as an IN member. Moreover, one-third of the online survey’s respondents believed that the regional government has received the recommendations and has intended to improve the integrity of its procurement process. However, another third believed the regional government has not been interested in making any improvements. The interviews also revealed drastic opinions regarding the low participation of the regional governments. Members wondered if the regional government is an IN member, and in one region, one CSO member asked “why the regional government is considered a member if the staff do not want to provide information.” CSOs such as national networks focused on youth empowerment, anticorruption advocacy, community organizations, and neighborhood associations are involved in the INs. The professional association chapters of Journalism, Engineering and Law are also represented in the IN. The Roundtable for the Fight against Poverty (Mesa de Concertación para la Lucha contra la Pobreza, MCLCP), a mixed public sector and civil society organization, participates in the IN, too. The MCLCP has had an essential role in articulating the work of the IN. It is seen as a neutral space that offers an environment of transparency in which participants from public and CSO organizations can communicate and work together. The MCLCP’s perceived image strengthens its ability to identify and convene critical actors in different areas. Another characteristic of the MCLCP that supports the operation of the INs is its legal status. An interviewee commented that due to the networks' lack of legal status, any request for information or questioning about the PIPs' operations must be processed through an established organization like the MCLCP.17 I believe the MCLCP has support from the State, and it has a prestige earned over the years.… It does not have a political or government bias, but it is all about the well-being. CSO member, Loreto. 17 Or the CRA in the case of Cusco. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 13 In the case of the public organizations, the CGR has been involved in teaching the skills and supporting the analysis performed by the IN. One of the policies of the CGR is to interact and join ties with CSOs. The CGR representatives provide training, collect complaints from citizens, schedule random site visits, and conduct other preventive activities. The IN members consider that the involvement of the CGR is relevant because of its function, its credibility, and its contribution to the analysis they performed while they applied the ISs. Some IN members consider that the CGR could use the reports prepared by the IN to put more “pressure” on the regional governments. Cusco is a particular case where the IN was hosted by the Regional Anticorruption Commission (Comisión Regional Anticorrupción, CRA)—with public and CSO members—and was organized as a Technical Task Group. In this region, the respondents considered that only the representatives from the public sector could make decisions while the civic society representatives could only share their opinions.18 In comparison, other INs have an environment where all actors have the same rights, and decision are made democratically. Since public organizations closely follow their mandate, some could be reluctant to reach an agreement with other actors. TPI trained journalists who have produced research articles promoting transparency and denouncing faults during the procurement process after the training. However, one respondent pointed out that they run into limitations to continue working on the topic due to lack of resources to pay for the working hours, equipment, and services (P.M. 2.2). The ET encountered ambivalent opinions regarding the participation of journalists. Some believe that journalists’ reports with incomplete information might damage the relationship with governmental agencies. There is also a separate angle to this participation. In Piura and Lambayeque, the regional chapters of the Professional Association of Journalists are members of the IN. One member pointed out that these professionals could support the development of a comprehensive communication of the network accomplishments. Little participation of private construction companies as INs members. The private sector's participation is one of the IN’s goals (P.M. 3.1, 3.2, 3.3), but its involvement has been minimal. The ET found that two reasons explain this outcome in all the TPI regions. First, the number of formal companies in the construction sector outside the capital city (Lima) is very small. This is especially true in the case of medium and large companies, so few local construction companies have the capacity to win public bids. Hence, firms from bigger cities, even from faraway regions, usually win the contracts. 18 Defensoría del Pueblo. (2021). Análisis y perspectivas del nuevo marco regulatorio de las comisiones regionales anticorrupción. Pages 19-22. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 14 Table 3. Number of companies in the construction sector by region (2021) Region Total number of companies Number of medium and large companies (Annual sales equal to $2 million or more) Lima 31,104 522 Huaura Lima Province 446 1 Cuzco 2,083 8 Lambayeque 1,461 5 Loreto 1,780 5 Piura 3,741 31 Source: Ministry of Production, Economic Studies Office. Tejido Empresarial en el Perú.19 Second, interviewees also pointed out that contracting companies cannot be IN members due to lack of interest or conflicts of interest. The lack of interest of these construction companies to join the INs could be explained by their focus on maximizing profit through short-term contracts with the public sector instead of adopting good practices of due diligence to strengthen business relationships, as reported by respondents. Technical secretariats in Lima and Lambayeque also agree that construction companies privileged a relationship with local and regional governments where they can influence them to not stop their projects even if they recognize technical flaws. However, the respondents also recognize that the participation of business leader associations could add value to the INs if they don’t have a commercial interest involving the local governments. The only association of this kind that has joined an IN is the Commerce and Production Chamber of Huaura (Lima province) that gathers 161 associates from the construction, tourism (13), services (74), trade (36), and other sectors. There is a clear distinction between the role played by the technical secretariat and the rest of members. For many members, the technical secretariat represents the IN and is in charge of scheduling meetings, articulating the interaction between the IN members and public organizations, and identifying organizations to include in the IN, among other responsibilities. While the regional coordinator helped prepare the documents produced by the IN, the technical secretariat revised, signed, and sent them to the regional government. The technical secretariat has been described mostly as a person with fair knowledge and experience in public management and with social skills like empathy and commitment. Some members have argued that it would be better to have a civil servant as the technical secretariat because it could increase the attention and reaction on the problems identified in the prioritized 19 https://app.powerbi.com/view?r=eyJrIjoiOTg1MDMxOTctZDYzMC00MDVlLWI3YjQtZGY5OGZlMTFhOGYxIiwidCI6ImMzN GNjOGM2LTJiMjctNDA0Mi1hMGE2LWI3OWZiMmM0NDE4NyIsImMiOjR9&pageName=ReportSection CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 15 projects. However, other members considered that, due to the rotation of personnel, it could risk the continuity of their activities or their autonomy. About the dynamic within the IN, the respondents described having periodic meetings in which the proposals are presented and discussed in a consensual manner between all members. Proposals are made available to the members and the different opinions are heard and voted on. All members have participated; we have had horizontal communication. CSO member, Lambayeque. Members valued training; even though they considered the topic complex, they looked forward to more case studies practice and planning improvements. Even though the members valued the training, they believe that the implementation, schedule, and content could improve. Due to the pandemic, the training took place online, but some members were not very familiar with using some web-based platforms; in addition, they preferred the chance to interact with each other. Also, most of the members considered they should receive more training and that it should include more empirical examples (P.M. 2.3). The online survey shows that 39 percent of respondents (15 out of 39) believed that their knowledge about public procurement in public investments had improved, and 61 percent (24 out of 39) believed that their competencies to analyze the procurement information improved.20 Additionally, 82 percent of respondents (32 out of 39) expected to use the ISs in their work to promote public sector accountability.21 On the other hand, 5 out of 10 respondents from the public sector believed that not all the regional staff have been able to or know how to apply the ISs. Complementary, members selected three subjects they would like to receive more training: monitoring the improvements of the PCI22 of the regional government, monitoring the recommendations of the concurrent audit reports of the Comptroller’s Office, and monitoring the reduction of the risks related to the regional government procurement processes. TPI organized the networks with the valuable participation of women promoting anti￾corruption actions. This is one result regarded as positive by TPI.23 Overall, the number of women involved in the IN increased from 21 to 40 between September 2021 and June 2022. Women represented 60 percent of members (taking into account representatives from public and private sectors and CSOs) in June 2022. The result is close to the percentage reported in other citizen’s oversight experiences in Peru (only CSO members).24 The two regions that stand out are Cusco and Lima, where the number of women involved in the IN went up from 4 to 11 (increase of 150%) and from 2 to 9 (increase of 350%), respectively. TPI’s strategies to promote women’s participation achieved an increase of their membership but also created the opportunity for CSO women to hold important roles in decision-making promoting anticorruption actions. The Piura, Lambayeque, Cusco, and Loreto regions had or have women from the 20 For both cases, the respondents were: Piura 6, Lambayeque 10, Lima 6, Loreto 9, Cusco 8. 21 Piura 4 respondents, Lambayeque 9, Lima 6, Loreto 7, Cusco 6. 22 The Preventive Capacity Index (PCI) evaluates the implementation of the Integrity Model in public entities on a scale from 0 to 100 where 0 means the absence of standardized corruption prevention mechanisms and 100 is an optimal level prevention. See Integrity reports – December 2021. https://reporteintegridad.servicios.gob.pe/ 23 USAID. (2021). Proyecto Inversión Pública Transparente. Informe Anual de Desempeño 2021. Pages 61-62 24 USAID. (2021, August). Civil Society Engagement Assessment. Page 32. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 16 MCLCP as Technical Secretariats. Their experience maintained a cohesive and active network. According to the latest annual report, their role in the IN has allowed them to enhance their capacities as leaders in their institutions to promote the use of ISs to monitor PIPs and to mobilize other women to join their efforts.25 In the regional government of Piura, where the OI is a member of the IN, two women, the head and a senior staff, have been champions in promoting an agreement with TPI and creating a platform that applies the ISs to any regional project. Likewise, the female member that leads the OI in Lambayeque worked together with the former Technical Secretariat, also a woman, to consolidate the beginning of the IN in the region. Also, two female members lead the monitoring of integrity in the Lima and Loreto regional governments, where the integrity responsibilities are under the human resources offices. Another interesting case is the head of the Commerce and Production Chamber in Lima, a woman leading 190 associated businesses. For the first time, as she said, her organization is taking steps to be part of the citizens’ oversight of public investments. In Cusco region, where the IN is hosted by the CRA, besides the leadership of the MCLCP, there are four female members representing public agencies involved in the anticorruption system, including the CGR, the Board of Superior Prosecutors, the Superior Court of Justice, and the Provincial Prefecture. Any decision-making is impacted by their participation. Other noteworthy cases are the participation of young women leading the Interquorum CSO in Cusco and Piura. They have shown commitment and are bringing new ideas and knowledge to their respective INs. Finally, the participation of women as direct users of the infrastructure projects has fostered the dialogue between the regional government and the networks in Piura (Los Algarrobos health facility and currently the Miraflores urban development projects) and in Lima region (Corazon de Jesus school project). Table 4 depicts the membership structure of each IN. Table 4. Members of the Integrity Networks (September 2021 and June 2022) REGION IN’s name Date of creation Date of data Members Organizations Individuals Public sector Private sector CSOs* Total Women Total PIURA Red Integridad Piura 06/2021 09/2021 2 - 6 8 8 13 06/2022 1 6 7 8 11 LAMBAYEQUE 12/2020 09/2021 3 - 5 8 3 10 25 USAID. (2021). Proyecto Inversión Pública Transparente. Informe Anual de Desempeño 2021. Pages 62. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 17 REGION IN’s name Date of creation Date of data Members Organizations Individuals Public sector Private sector CSOs* Total Women Total Red Integridad Lambayeque 06/2022 2 7 9 6 10 LIMA PROVINCIAS Mesa de Seguimiento de Proyectos de Inversión de la MCLCP 02/2021 09/2021 2 1 6 9 2 11 06/2022 4 1 5 10 9 18 LORETO Grupo de Trabajo Integridad dentro de la Dimensión Institucional de la MCLCP 02/2021 09/2021 4 - 3 7 4 7 06/2022 3 4 7 6 8 CUSCO Comité Técnico en CRA 12/2020 09/2021 5 - 1 6 4 7 06/2022 8 - 4 12 11 19 Total 09/2021 15 1 22 38 21 48 06/2022 17 1 27 40 40 66 Source: Transparent Public Investment Project. Summarized by NORC. (*) CSOs include NGOs, citizen’s organizations, and professional association chapters, including journalists and the MCLCP. GOAL B. FOR INTEGRITY NETWORKS TO IMPROVE SELECTED PUBLIC INVESTMENT PROJECTS’ COMPLIANCE WITH INTEGRITY STANDARDS. In the last annual report,26 TPI underlined the use of 14 out of 15 standards27 for nine projects. Other noteworthy progress was creating and using a QR providing project status information, a novelty tool to monitor the projects in the Loreto region, and a platform called Ojo en la Obra (Eye on the construction site) to report a simplified version of the control concurrente report and the further use of the ISs in 10 more projects in the Piura region.28 This section analyzes two results: • Number of selected PIPs that exceed the compliance practices of the public procurement ISs. • Number of public ISs applied to PIPs and promoted by INs. NUMBER OF SELECTED PUBLIC INVESTMENT PROJECTS THAT EXCEED THE COMPLIANCE PRACTICES OF THE PUBLIC PROCUREMENT INTEGRITY STANDARDS.INs fostered the 26 USAID. (2021). Proyecto Inversión Pública Transparente. Informe Anual de Desempeño 2021. Pages 15-16. 27 The report mentions “29 ISs” because it considers the times the ISs were used. 28 USAID. op.cit. Pages 21-23. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 18 adoption of more information than normally provided by regional governments to monitor the selected projects, thanks to equipping the members with more expertise and organized evidence. However, the networks face two limitations: relying on an instrument that cannot sort out technical problems generated in the Technical Construction Study and the regional public officers’ lack of will and/or technical capacity to address deficiencies in the project design. Interviews and the online survey revealed that training in how to use systematic information sources (open data in Government of Peru’s Ministry of Finance [MEF], OSCE, and CGR) and involving infrastructure practitioners, like engineers, in the network improved CSO members’ capacity and credibility to monitor various projects. The evaluation found that some selected projects reported null execution progress. Interviewees expressed that monitoring projects since the preparatory acts help to reduce risks in the procurement phase that could hinder financial or physical progress in the next phase. Therefore, choosing only projects under execution does not necessarily narrow the incentives for corruption. In fact, a member from Loreto claimed that it could be good to monitor a project from the beginning so the network could contribute to guaranteeing that the project’s goals are aligned with the population’s needs (P.M. 1.6). A senior specialist in Lima and a senior civil servant member in Piura pointed out that one of the major merits of the networks is gathering members able to verify the projects’ results and efficiency. In fact, IN members can share previous experiences to test the projects’ relevance and help identify those with no social impact.29 The representative of Piura stated that the problems are present with the procurement of infrastructure and equipment as well as other goods and services: The town’s mayor ordered to build statues that resembled his family.... Let’s see the medicines.... They need a thousand units, and they buy six thousand. Then, the medication expires.… The selected projects showed that the weaknesses in the Feasibility Studies and the Technical Construction Studies affected the fulfillment of the standards, even if the monitoring was made in any of the three stages.30 In all the regions, the interviews showed that issues during the pre-construction period limit the execution of projects. One of the many causes mentioned was the project designers’ insufficient capacity or dishonesty. After the feasibility studies are approved, the next step is a more elaborated document, the Technical Construction Study (expediente tecnico). There was also a consensus that Technical Construction Studies were a source of wrong scope and design (P.M. 1.6).31 Thus, when network members tried to monitor projects in the construction stage, 29 Out of 81 risks identified by OSCE, the pre-purchase phase (close to the definition of preparatory phase) carried 23 risks, the selection phase had 21 risks, contracting had 17 risks, and transversal to all the process, 20 risks. In OSCE. (2021). Informe de resultados de línea base de gestión de riesgos. Pages 7-9. See also, OSCE. (2020). Diagnóstico y Estrategia para la Gestión de Riesgos en Contratación Pública. Pages 33-40. 30 Preparatory acts, procurement, and execution. 31 The CGR’s reports have also indicated that this topic is a common problem. See, for example, Shack, N., Pérez, J., & Portugal, L. (2020). Cálculo del tamaño de la corrupción y la inconducta funcional en el Perú: una aproximación exploratoria. Pages 19, 26. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 19 they often found that the delay or change in the technical specifications responded to apparent failures of these studies.32 For example, a network member in Piura claimed that: We haven’t discovered the wheel … that is something that has been going on for years. The deficiency comes from the preparation of the projects’ Technical Construction Studies. The interviewees believe that well-trained members of the networks could review the Technical Construction Studies to test the scope of the study or even to assess some technical specifications that could jeopardize the construction quality or the implementation calendar. A direct beneficiary in Piura also commented that her housing association had registered some neighborhood engineers to review 12 volumes of the project; the regional government promised the volumes. The governor himself delivered the volumes of the projects in his office.33 As a result of the failures in the Technical Construction Studies, contractors typically request budget increases from the administration to deal with negative circumstances that were not under their control or were not identified in the pre-investment studies. This originates delays in different phases or stops the execution phase, making it difficult to monitor the ISs. Public officers reduce their roles in specific stages instead of taking a holistic role in the project procurement and construction cycle. Public officers’ lack of will or capacity was a common limitation to progress in delivering projects on time and with technical assurance.34 At least in two regions (Piura and Lima), interviewees mentioned that when private supervisors are in charge of monitoring the construction, public officers did little or nothing to follow up on the projects. Instead of double-checking the supervisor’s information, they accept these results. In these regions, this lack of responsibility potentially opened the door to corrupt behaviors. For example, a public sector member in Piura stated: Regional governments think that they don’t have to do anything after they hired the person who will execute the construction work and the person who will supervise the execution of the construction work. And then, suddenly, this executor and this supervisor get into an agreement.… Support from infrastructure practitioners makes the monitoring process easier to implement. The online survey showed that CSO members’ selected the different levels of experience with public procurement and investment as one of the three main limiting factors (48 percent of respondents, or 16 out 33).35 It is therefore no surprise the special credit given to the guidance provided by the members with most experience in the infrastructure sector, like the representatives of 32 According to OSCE’s findings, the inadequate formulation of requirements (Terms of Reference, Technical Specifications, and Technical Construction Studies) limits competition or directs the purchase to a specific contractor. The latter is the second risk with highest probability of occurrence and impact in Peru’s public procurement in regional governments. In OSCE. (2021). Informe de resultados de línea base de gestión de riesgos. Pages 7-9. 33 It was submitted in a CD at the beginning of September and in a printed version at the Governor’s office two weeks after. 34 According to OSCE’s findings, the limited training of officials and servants of the unit responsible of contracting and/or of the different organic units of the entity is the first risk, with highest probability of occurrence and impact, in Peru’s public procurement in regional governments. In OSCE, op. cit. Pages 7-9. 35 Piura 1 respondent, Lambayeque 3, Lima 1, Loreto 5, Cusco 5. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 20 the Engineering Professional Association. It was suggested that the institutional involvement of these actors improved the networks’ capacity and credibility to deal with various projects. Lambayeque showcased how network members widely hail the contribution of the Engineering Professional Association representative. A Lambayeque’s municipality member commented: If they are planning to go to, for example, an irrigation canal, [the Engineers Association] should bring an agricultural engineer to provide support and guidance ... because we are not engineers, and it will help the team to reach better conclusions. However, members also showed concern about continuity. Since the membership is based on the institutions they represent (and not as an individual citizen), some members might have to step down because their role within their institutions could change. For example, the Presidency of the Professional Association of Engineers only lasts one year. IN members are concerned about the contributions of the individuals representing the organizations because the organizations could remain as members with the same or new agreement with the network. 36 Open information has been critical to monitor the projects, but the networks’ interest in other tools is growing. Interviewees valued the training and support from all Regional Coordinators to understand the websites and open information on the public procurement and investment project (P.M. 1.3). Considering the variety of backgrounds in the networks, running workshops on how to use the data from these platforms was an intelligent strategy to close the expertise gap between the members. New tools developed by the integrity project implementers are also considered vital to enhancing information intake (P.M. 2.3). For example, a CSO member of the Loreto network showcased: There is another app that we call Ojo en la Obra. It is quite interesting and very simple to use. It’s a graphical representation of each project’s status. The online survey showed that 57 percent of respondents (19 out of 33)37 pointed out that using open data platforms from public agencies to find information from regional government investments and procurement helped them monitor the ISs' application. Also, 51 percent (17 respondents out of 33)38 consider that concurrent control provides valuable information to monitor the projects. Members also expressed in the online survey that they would like to receive more training on the Preventive Capacity Index and the identification and monitoring of OSCE’s procurement risks. Around half of respondents considered these topics a priority for the next training sessions (16 out of 33 respondents).39 36 According to specialists, individuals and institutional members have unique potential contributions. Having individuals as network members could have benefits of flexibility to commit during agreements, while institutional members could represent organizations with missions that relate to the network’s shared purpose. In Collective Minds. (n.d.). Module 2 - Developing your network’s membership model. Page 6. 37 Open data: Piura 3 respondents, Lambayeque 4. Lima 2, Loreto 7, Cusco 3. 38 Concurrent control: Piura 2 respondents, Lambayeque 0, Lima 1, Loreto 8, Cusco 6. 39 PCI: Piura 1 respóndenos, Lambayeque 2, Lima 2, Loreto 5, Cusco 6. Concurrent control: Piura 2 respondents, Lambayeque 3, Lima 2, Loreto 3, Cusco 4. OSCE risks: Piura 3 respondents, Lambayeque 4, Lima 1, Loreto 5, Cusco 3. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 21 Monitoring the evolution of the projects in one year revealed unforeseen changes with little or no influence from the network; however, the standards are still applied and pressing for transparency. Whereas the conformation of networks in the five regions brought more organized information and skills to oversee the transparent management of the public investments to members, this type of monitoring has shown little effect on accelerating the efficiency of the infrastructure in regions, with serious issues related to works remaining at a standstill.40 Compared with their initial appraisal, at least 5 of the 12 selected projects had overcosts and 7 are delayed. Also, one of the projects initially selected in Piura (Algarrobos Health Facility) was canceled during the monitoring period. On average, the performance of the selected projects is not worse than the implementation of the comparison projects but still is far from showing efficiency. This context has raised the aspirations of the networks as CSO members showed they want to influence results beyond only reporting the status of standards. Various interviewees think that the Technical Secretariat should follow up with the response of the regional government after receiving the IN’s reports with recommendations. Both procurement modalities (contrata y administración directa)41 showed similar results in terms of project delivery. Still, direct public procurement projects are more difficult to oversee, and the risks are concentrated in the government. Overall, the monitored projects showed budget expansion (over the initial approved budget) and delays, whether the procurement modality is executed by private contracting or the regional government. In addition, the interviews in the regions that prioritized projects implemented by direct public procurements provided insights about a potential lack of due diligence from regional governments. The CGR expressed concern about the projects implemented via direct administration due to problems with the technical specifications, which are not set according to the real operations capacity (including the equipment) of local and regional governments. The CGR has presented a legislative initiative to improve the control of these type of projects.42 Funding investments with direct administration (Contratación Directa) modality creates the opportunity to misuse the appropriated budget in items outside the project. The interviews with two practitioners of public procurement revealed that implementing PIPs through direct contracting could allow the regional authorities discretion to hire workers with the project’s funds. Thus, project funds could be used to finance office staff instead of project workers or site supervisors. Similarly, in the project in Lima executed by the same modality, the funds allocated for personnel exceeded their initial budget due to personnel contract extensions beyond their initial plan with no justification. Not surprisingly, both projects are delayed. However, monitoring these types of projects by the networks faces more difficulties since supervising workforce requires regular and continuous site visits to oversee if the number of hired workers are deployed according to the construction plan. 40 A GCR report on the number of works that had been temporarily stopped indicate the projects halted: in Cusco 412, Lima 157, Loreto 65, Piura 138, and Lambayeque 54. See GCR. (2022). Reporte de obras paralizadas en el territorio nacional al II trimestre 2022. Page 11. 41 Direct Administration: Projects constructed directly by any level of government. The regional government, in this case, procures services, goods and hired the construction crews directly. 42 In CGR press released (2022, January 17) Contraloría advierte que más de 2300 obras permanecen paralizadas a nivel nacional. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 22 NUMBER OF PUBLIC INTEGRITY STANDARDS APPLIED TO PUBLIC INVESTMENT PROJECTS AND PROMOTED BY INTEGRITY NETWORKS. The members applied the 15 milestones set by the ISs tool, but they expressed ownership on those that involved practical activities. CSO members, especially those without experience with public procurement, had limited capacity to interpret the ISs. The online survey revealed that 63 percent of respondents (21 out of 33),43 other than the regional staff, thought that not all the members have been able or knew how to monitor the application of the ISs. Those 21 respondents stated that the ISs were difficult to understand or apply, and the individuals monitoring the ISs needed working experience in the public sector to understand complexities of the public procurement process. The interviews in all the visited regions revealed that the most-used ISs are those that involved practical activities or the review of verifiable documents. However, there are other ISs that require specialized analyses. For example, fieldwork activities involve verifying the update of the site’s Supervisor Logbook (standard 11), or they involve high-profile documents such as the Comptroller’s recommendations (standard 14, also supported by Ojo en la Obra platform) or the sworn statement of the procurement officers (standard 3). All those activities allowed the IN’s members to make their efforts visible. These standards permitted easy access to information and more participation and connections among the members. Standards 5, 6, 7, and 8 require information from OSCE’s platform; members with adequate experience navigating on the internet can monitor these standards and disseminate the information among the rest of members. The application or monitoring of standard 9 looks for the review of documentation submitted by the winning contractor.44 Regional governments, according to an interview, could make a random selection of documents without a specific deadline. The bylaws do not set a deadline for completing these tasks, which runs against the CSO members’ actions. The compliance of other ISs is more difficult to establish. In the cases of standards 1 and 4, CSO members, not the regional staff, would need a technical opinion about the Feasibility Studies and the Technical Construction Study. The objective is to see if the Technical Construction Study has been updated and includes all changes in the construction area, like new utility cables, water and sewerage pipes, etc. IS 2 requires internal information of the regional government administration, and it specifically requires IN members to follow a well-established decision-making process during procurement and infrastructure execution in the regional government. One specialist commented that monitoring that process in Cusco would require examining the information of six different offices and evaluating the level of coordination among them. A specialist in Lima Province suggested that the activity would be simplified if the all the “functional activities” (actos funcionales) are openly published in the regional government’s portal. Another senior adviser in Lima mentioned that he felt that some units of the regional government tried to delay publication of information on their organization’s web platform. 43Piura 5 respondents, Lambayeque 2, Lima 4, Loreto 6, Cusco 4. 44 There is, however, a case where the financial warranty letters might be no longer valid, not because the contractor failed to provide adequate information but because the bank or credit union issuing the document no longer has a good rating in the banking system. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 23 Complementary practical training activities helped the IN’s members own the standards. Interviewees preferred practical training because fieldwork activities provide unique firsthand experience about the actual progress of infrastructure projects. Through such training, the members felt more confident dealing with procurement officers or with the infrastructure supervisor. Moreover, visiting the site let them identify new practices like the “cuaderno digital de obra” to fulfill IS 11 as part of the works supervision (PM 1.3). Planning ahead of the site visit is also crucial since the private contractors need to deploy their personnel and their documentation to receive the network members. There was a case in Loreto when the visit to the hospital construction site was canceled due to lack of coordination. Practical experiences support deep and accelerated learning among members and was selected as the main reason for the massive support for using the standards beyond the network. The online survey showed that 82 percent of respondents (32 out of 39)45 indicated they would use the standards in their work or advocacy activities promoting public sector accountability. EVALUATION QUESTION 2. WHAT FACTORS ENABLED OR INHIBITED TPI FROM ACHIEVING ITS GOALS RELATED TO INTEGRITY NETWORKS? The ET identified factors that affected the networks differently, supporting or limiting their governance, operations, and technical support from TPI. The following analysis lists all the identified factors and includes examples and excerpts from stakeholders when possible. WHAT CONDITIONS/FACTORS HELP INTEGRITY NETWORKS BE SUCCESSFUL? The national preventive and anticorruption policy created the environment to promote a multi-actor dialogue, including CSO participation.46 The IN strategy capitalized on the objectives and intent from the policy and the experience of national and regional specialists who are part of the anticorruption entities supported by this policy. The government created the National High-Level Commission (Comision de Alto Nivel) in 2010 (and amended in 2013) and the CRAs in 2017. The CRAs comprise the justice administration agencies (courts, district attorneys and prosecutors), the Ombudsman’s Office, the CGR control agency, and private and civil society sectors, all related to the fight against corruption. The governance of the CRAs considers a group of entities with a voice and vote and another group only with voice.47 Respondents in Lambayeque indicated that their experience in the CRAL was essential in their work in the IN. In the case of Cusco, the IN is a CRAC Technical Commission, and even though they had time availability constraints, their members individually showed interest in the IN’s work. In Lambayeque, a CSO member commented: Here in Lambayeque, the anticorruption commission worked like no other with verified results.… The work was consolidated with monthly meetings. 45 Piura 4 respondents, Lambayeque 9, Lima 6, Loreto 7, Cusco 6. 46 Comisión de Alto Nivel Anticorrupción (CAN). (2017, September 14). Política Nacional de Integridad y Lucha Contra la Corrupción. Decreto Supremo 092-2017-PCM. 47 Defensoría del Pueblo. (2021). Análisis y perspectivas del nuevo marco regulatorio de las comisiones regionales anticorrupción. Pages 19-22. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 24 Additionally, the Prime Minister’s Office created the OIs in all public sector agencies in 2019. As of June 2022, the OIs are the Technical Secretariat of the CRAs. They are part of the INs and could facilitate the coordination between the network and the regional governments, independently of their degree of openness. For example, the Piura OIs promoted an agreement letter with TPI, enabling interactions with the IN members. The OIs are autonomous units from the regional administration in Piura, Lambayeque, and Cusco, while they are under the regional governments’ human resources offices in Lima Province and Loreto. The experience of the Piura Integrity Office facilitated the adoption and digitalization of the ISs methodology by the regional government administration. After 11 years of practice as a regional anticorruption office, this unit adopted the functions of the regional OI. As member of the IN, the OI promoted the application of the ISs to 10 other regional projects (besides the two prioritized projects). Moreover, the office has developed a platform to monitor the compliance of the 15 ISs plus one more identified by the same OI. The platform is administered by credited staff from different departments, mainly logistics and investment offices, who collect information from their administrative data or central government agencies’ sources and upload the information to the platform. The availability of the information collected by the regional government’s platform would simplify the network’s work tremendously and be a step toward more transparency. The regional government has formally installed the ISs’ methodology with a regional bylaw (directiva regional).48 IN members value the experience and background of the MCLCP and its ability to convene different stakeholders. As of June 2022, MCLCP holds the Technical Secretariat of four out of five networks. Created in 2001 under the support of national legislation,49,50 with an annual budget financed with public resources, it already congregates public sector agencies and CSOs. Specialists and network members pointed out that one problem faced by the INs is that they do not have legal status and their recommendations are not binding. The participation of the MCLCP partially covers this gap, signing the communications with its letterhead together with the regional implementing partner (IP) and the networks’ logo. In addition, in 2017, the MCLCP signed an agreement with the CGR to oversee the reconstruction investments, taking advantage of its capacity to assemble the regional and local governments and civil society as well as their institutional experience monitoring public investments. The MCLCP has been promoting governability agreements (acuerdos de gobernabilidad) among political parties in all regions during election periods since 2002. Members saw this event as an opportunity to insert the topic of integrity and improve regional and local governments’ transparency after the elections in October 2022. All five regional governments with INs have signed an agreement for the 2023–2026 period. The receptiveness of the regional CGR offices strengthens the INs’ preventive control activities. The regional CGR managers have expressed interest in the INs’ work, which complements their preventive work. In one case, a respondent manifested the close collaboration between the regional CGR and the IN, especially the MCLCP, exchanging information, improving reports, and 48 Directiva Regional N°06-2021/Grp-100030. (2021, October 12). Lineamientos para Implementar los Estándares de Integridad en Contrataciones e Inversión Pública en el Gobierno Regional Piura, Aprobada con Resolución Ejecutiva Regional N°650- 2021/Grp. 49 See I) Decreto Supremo N° 001-2001-PROMUDEH. (2001, January) and II) Decreto Supremo N° 014-2001-PROMUDEH. 50 Ley N° 27867 de 2002, Ley Orgánica de Gobiernos Regionales. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 25 planning future joint project visits. A CSO member in Lambayeque provided an example of this collaboration: The Comptroller always provided suggestions on issues that we also identified. They are more specialized than us and that has been very valuable. Regarding the mutual collaboration, there has been a case where the members have let the regional CGR know that a project was on hold after visiting the construction site. As a result, the CGR took action to look for the causes of the problem. The CGR, as member of the network, has provided feedback to the IN’s plans and reports; this was well received and led to improvements in these documents. However, in regions where the CGR regional manager has been recently appointed, this cooperation is developing. The members see the membership of organizations led by young members as a strength due to their digital knowledge, abilities, and leadership. For example, members pointed out that young members could support information search in some platforms and some ISs’ analyses. Other members added that it’s vital to encourage more young individuals to participate and were willing to transfer the knowledge they have accumulated as community leaders. Likewise, IN members from youth organizations also commented that young volunteers are innate leaders and can contribute to society. For example, a youth organization in Lambayeque that was overseeing unfinished water projects requested to join the IN. A CSO member in Piura expressed her interest on having young members: We want to empower young prospects to work in a specific IS, in the concurrent control reports, mostly in the technological part. Regional coordinators and young professionals have been instrumental to follow-up training sessions and practice, support the use of the novel digital tools prepared by TPI, and maintain an informed network and close coordination with the Technical Secretariat. During the focus groups discussions, representatives from youth organizations suggested to establish agreements with a university to provide training, which could eventually be translated into college credits. CSO members’ watchdog experience brought assets and opportunities for the development of the INs. Some INs’ members had previous experience as citizen observers (monitores) or as community representatives that oversee public infrastructure projects in their own localities. These members strengthened the work of the INs in different ways, including: (i) using their knowledge and skills acquired in the CRA (Lambayeque) to propose indicators and guides for the application and monitoring of ISs. (ii) bringing a critical view on how the pace of the construction work should proceed after their experience participating in the Reconstruction with Changes agency in 2017.51 (iii) showing leadership congregating local engineers from the beneficiary area of an infrastructure project selected by TPI. 51 In 2017, after El Niño devastated the Northern Coast of Peru, the national government created a centralized agency to finance and managed the development of the reconstruction efforts. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 26 A CSO member in Lambayeque gave an example of their work: In the first visit experience [to the civil works], there was not an idea on how to prepare the report. Since we had experience in monitoring, we prepared a path on how to present that report. These and other examples showcased what the members brought to the IN, especially when their institutions are networks in themselves and have been working in monitoring social and economic indicators or overseeing the participatory budget, including infrastructure works. INs’ members recognize regional coordinators and implementing partners (IPs)52 as an important source of technical assistance and coordination support. Members valued support of the IP during and after training. Members also appreciate the full-time support of the regional coordinators to set meetings, conduct visits, process information, and follow up with the requests to the regional governments. A CSO member from Lambayeque commented: We have full-time jobs, so we cannot completely engage even though lots of topics require dedication. Thus, the participation of the regional coordinators is key to follow up the requests of information, conduct visits, or set up the meetings. Their presence was instrumental in engaging with organizations, like the bar association in one region, and direct users and youth organizations to join the network. In the site visits, the coordinators helped participants identify the topics they should oversee and designed a spreadsheet to collect the information in one case. After data gathering, coordinators input and process the data for reporting. IPs have been instrumental in developing the Ojo en la Obra platform that reports the progress of the control concurrente recommendations in a graphic way (resembling a traffic light). It also includes documentation to verify compliance, like the regional government’s letters and reports addressed to the CGR providing information on the state of the CGR’s observations to the projects. CSO members’ commitment and dedication to the INs supported the network’s objectives. The network has committed to reducing corruption, and the members value its inclusiveness. Also, they think that trained and committed civil society members can inspire other citizens. A regional coordinator mentioned that they had not observed any procrastination, internal conflict, or disinformation. On the contrary, they maintained themselves connected and committed. In Lambayeque, a CSO member added: It’s not like you get paid, but the network gives you this opportunity to participate as a citizen, and we can learn how the public civil works are implemented. The country’s negative political environment has triggered the commitment of civil society to oversee governmental activities and investments. A senior specialist in Lima pointed out that one element needed for the networks to function was trust, and the TPI team worked toward that objective in an environment where even the national government leader showed a lack of transparency and accountability. While the latter could have discouraged the consolidation of the INs, other opinions supported the idea that it promoted CSO participation. For example, in Lambayeque a CSO member 52 Central administration of TPI Lima, Proética, and Asociación Transparencia. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 27 mentioned that young individuals of the Leonardo Ortiz city (with numerous problems of corruption and unfinished sanitary projects) were diligently overseeing the municipality projects and requested to join the network. Also, in Piura, a specialist indicated that the failure of authorities to close the basic needs gap had triggered local organizations to join the network, which is perceived as a channel to exert pressure on their leaders, he commented: The [integrity] standards arrived in times when there were lots of corruption complaints in the region, and false guarantee letters were discovered. When the proposal of the IN arrived, it was a method to begin organizing efforts to have an adequate verification of all civil works documentation. WHAT CONDITIONS/FACTORS HINDER INTEGRITY NETWORKS’ SUCCESS? The limited technical capability of the regional public agencies has increased the risk of ineffective public resource use. This incapacity creates an ideal environment for corruption. A senior practitioner in Lima indicated that the regional staff, especially those from the logistics office, need to be trained to avoid inefficacy of the procurement process. Another member in Cusco commented that there are elected authorities and civil servants that need training in public administration to avoid making mistakes that other individuals could take advantage of. In Piura a senior practitioner added: Lately, we were witnessing that maybe a part of the problem was due to corruption only…. Now we are witnessing more incapacity, which generates an alarming combo. Other respondents commented that some public servants do not know their responsibilities, even though they are well defined in their own organization’s bylaws. For example, staff from the regional investment office could wrongly think that once the region signs a contract for public works, the contracting company and the supervisor are the sole responsible entities in charge. The pandemic has been a central factor that limited training and delayed networking among members. The initial online format of the training faced logistic challenges. For some members, this was difficult because they are/were not very familiar with the internet or video￾conferencing platforms. The situation was even more cumbersome in the Loreto region, where the internet connection was unstable. During the interviews, members expressed their satisfaction with the in-person training and coordination sessions. In addition, the return to face-to-face training has allowed them to interact with each other and identify which concepts may require reinforcement or if additional subjects need to be included in the training. Network’s and regional governments’ loss of built capacity due to high turnover of regional government staffers and CSO members. From the information gathered, the ET learned that one regional government had had several investment managers in a few months. In another region, the governor was impeached, and interviewees felt the new administration was not interested in the network’s work and changed the selection of the projects. A CSO member in Lambayeque explained: We had an elected governor who changed managers and the head of the Integrity Office also left. Then, that implies that we are beginning one more time when we meet again … always beginning again. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 28 Regional and local governments held elections on October 2, 2022, so the heads of the different offices, including the OI, could have changed. Respondents also mentioned that there are periods of representation mandated by law—for example, the rotation and appointments of regional control managers, and the election of district attorneys, and superior courts judges—affecting the CRAs, especially in Cusco.In Lambayeque, CSO members were also concerned about the changes: The Supreme Court President’s appointment ends in December.… We are going to have a new regional government; maybe we will meet the new governor, besides the new head of the regional anticorruption office. The high turnover occurred not only in the government sphere but also in the CSOs. For example, one member commented, “[This organization] joined the network in March, so there will be disparities dealing with the topics compared with the members that joined the network in 2021.” Also, representatives of the regional offices of the MCLCP changed after their election period. The time that members can devote to the INs’ activities is very limited. Members work to earn an income, so the coordination and planning of joined meetings and work plans was a significant challenge. In addition, irregular attendance to training and coordination negatively affects the members’ learning curve. Some members expressed their limitations in attending all training meetings, so they arranged for other individuals to attend some training sessions. This likely causes the loss of any comprehensive knowledge of the methodology. The lack of accountability tradition in the country’s public sector drives the regional government’s unwillingness to provide information and builds networks’ frustration. As a result, civil society members show mistrust and frustration when requesting information from the governments. One respondent believed that governments do not like to be watched and can give you a thousand excuses to avoid disclosing information. The respondents differentiate the regional government administration from the regional anticorruption offices, which now have the OI’s function in some cases, such as in Piura. While the OIs could engage with the INs, the regional government could be less open to providing information or attending a network meeting, like one organized by an IN to negotiate with the contractor to reactivate a halted project. While the online surveys showed that 36 percent of CSO respondents (12 out of 33) found that regional governments were not interested in making any improvement, another 36 percent of respondents (12 out of 33) found that the regional governments accepted the INs’ recommendations and intended to improve their procurement process. In addition, 24 percent of CSO respondents (8 out of 33) thought that the regional governments have adopted recommendations and have taken action to improve their procurement processes.53 Finally, members shared the need to go beyond signed papers and agreements. They expressed that policies and guidelines are essential but need to be implemented to have a real commitment. 53 Respondents that thought regional governments intended or took actions for improvement: Piura 2, Lambayeque 8, Lima 3, Loreto 5, Cusco 2. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 29 Members lacked resources to fully participate in the network. INs’ members cannot cover the costs of internet connection, transportation, procedural fees for government documentation requests, and protective gear. A respondent added that the significant problem is that the citizen’s watchdog responsibilities are recognized by law but not financed by the public budget. In one region, the lack of transportation made network members depend on the supervisor’s vehicle, and they felt compelled to limit their negative comments. Members also lack an appropriate internet bandwidth to launch searches and download data from public platforms, especially in Loreto. Another limitation was the lack of protective gear to visit construction sites. According to labor security bylaws, the guests need adequate protective equipment (construction boots, helmets, etc.) to enter the working areas. In addition, the pandemic made site visits even more expensive due to the requirement to show negative COVID-19 tests. Members expressed the need to have incentives to encourage the participation of the civil society. A respondent pointed out that the members’ work should be valued and acknowledged. A CSO member in Piura commented: More than anything, recognize everybody’s work. A monetary recognition is not needed. A simple recognition helps and incentivizes a little bit and motivates the people to continue working. Respondent pointed out that there was not enough media presence. The network was not noticed by the community or had no voice. The INs did not have a professional in communications or a link to the media that could help them appear in an interview to complain or demand that public entities use resources adequately. Another topic centered on the need for an accredited educational entity to provide training certificates. If the training hours had value in college credits, more young members would be interested in participating. The hierarchy inside the government administration could debilitate the effectiveness of the OI. The regional governments’ OI have been instrumental in the dynamics of the networks; moreover, they recognized themselves as part of the network. The location of the OI in the regional government’s organizational chart is important to enforce practices and accountability. Regional governments have the option to create a specialized office, like in Piura and Cusco, or assign the function to the human resources office, like in Lima and Loreto. The OECD54 recommendation of an autonomous OI is closer to the model in Piura (11 staffers) and Lambayeque, where those offices are dependent from the governor, and Cusco, where the office is under the general manager. The regional government of Lima Province has implemented the integrity function through the Office of Human Resources (by delegation) and has a team of 2 people; in the case 54 According to OECD recommendations, it is better to have a specialized and autonomous integrity office, which reports directly to the head of the organization. In OECD (2019). Las Oficinas de Integridad Institucional en el Perú. Hacia la implementación de un sistema de integridad. Page 39. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 30 of Loreto, the involvement of the regional government in the project is associated with the human resources office55 with a team of 5 people.56 The work of an autonomous integrity office versus a task force inside the human resources correlates with the PCI performance. The PCI evaluation released in July 202257 estimated the PCI of the five TPI regions: 83% in Piura, 67% in Cusco, and 55% in Lambayeque, while Lima and Loreto only had a PCI of 21% and 10%, respectively. But the OIs have encountered resistance from the infrastructure and administration personnel. In one instance, the officer requested the administration to organize two working groups: one group would apply the ISs, and a second group will work the preventive capacity index. Later, the administration told the integrity officer that they did not have the resources to provide incentives for staff to work in those groups. A connected issue is that professionals leading the OI might have had to step down in January 2023 after the October 2, 2022, elections. The rules and regulations could limit the IN hosted by a CRA. The Regional Anticorruption Commission comprises regulation, control, and justice administration agencies, the Ombudsman’s Office, the regional government represented by the OI, and civil society, However, this structure has limitations in supporting IN preventive activities, like requesting information from the regional government to monitor IS compliance. According to the legal competencies and functions of the justice administration, agencies like the district attorney cannot request administrative information from the regional government for preventive purposes; they could only intervene if there is suspicion of fault. Also, the system of voice and vote among members has recently determined that some regional members like the General Comptroller Office and civil society will have a voice but not vote, and the Ombudsman’s Office will remain neutral. In the case of Cusco, the MCLCP has joined the IN as Technical Secretariat. However, it has limited decision-making power, and their proposals have to be cleared by the head of the CRAC. Regional governments’ logistics units manage technical specifications with insufficient rigor, leading to possible weak results during construction. Furthermore, according to specialists and network members, the Technical Construction Studies are not followed by Compatibility Reports (informes de compatibilidad) where the physical and economic specifications could be updated to avoid delays or extra costs for contractors and the regions. Another raised topic that could be out of the control of TPI or the networks is that the supervision of projects focuses primarily on the execution of contracts instead of the optimal delivery of infrastructure projects, based on their technical specifications and costs. According to a specialist, the procurement guidelines have been designed to favor cost savings instead of maximizing quality. 55 Extracted from https://www.gob.pe/institucion/regionloreto/noticias/629506-gorel-realizo-taller-estandares-de-integridad￾aplicados-a-proyectos-de-inversion-publica. 56 Extracted from https://reporteintegridad.servicios.gob.pe/doc/EVALUACION_GORES_ICP_JULIO_2022.pdf. 57 PCI is categorized in five levels: remarkable (81–100% value), desirable (61–80% value), acceptable (41–60% value), regular (21–40% value), and low (1–20% value). CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 31 The lack of monitoring tools limited members to properly follow up on the ISs. One network reflected on the gap they found at the beginning, lacking indicators to monitor the standards. To improve the situation, the network innovated and identified the indicators for what each standard was expected to measure and an observation tool for the site visits. A CSO member in Lambayeque gave an example of proposed tools to facilitate the application of the ISs: None of the standards included indicators, which is a weakness of how the project was conceived. One does not know exactly what is being measured. Thus, when applying the standards, we tried to measure some indicators related to what we were interested in following up. ARE THE CURRENT ACTORS OF THE INTEGRITY NETWORKS LIKELY TO MAINTAIN COORDINATION AND COLLABORATION TO OVERSEE PUBLIC INVESTMENTS IN THE FUTURE? HAVE THEY INSTITUTIONALIZED AND/OR SHARED THEIR PRACTICES, NORMS, AND/OR PROCEDURES TO PROMOTE INTEGRITY AFTER THE TPI ENDS? The regional government of Piura, specifically the Anti-Corruption Office, is the champion example of good practice incorporating the ISs in their work. This office even incorporated a 16th standard, which looks into the financial closing of the projects (Liquidación). For the rest of CSO members, most online survey respondents indicated that they would continue using the ISs in their work or advocacy activities. During the interviews, they also referred to extending the experience to monitor the provision of public services like education. However, the ET did not find leads for future collaborative work. WHAT ARE THE MONETARY AND NON-MONETARY RESOURCES THE INTEGRITY NETWORKS USE TO PERFORM ITS ACTIVITIES, AND WHAT SHARE OF THESE RESOURCES IS THE TPI PROJECT RESPONSIBLE FOR? ARE THESE RESOURCES GOING TO BE AVAILABLE AFTER THE MONITORED PROJECTS OR THE TPI PROJECT END? The most important resource is the time CSO organizations provide voluntarily to the network; financial resources are limited for actors. According to the online survey, three Technical Secretaries worked 15 hours or more per month. Moreover, out of 36 responses from members other than the Technical Secretariat, 11 percent (4 respondents) devoted 15 or more hours per month to the network, 17 percent (6 respondents) invested from 10 to 14 hours, 33 percent (12 respondents) participated from 5 to 9 hours, and 36 percent (13 respondents) dedicated 4 or fewer hours per month.58 Civil servants who participated in the network must share their time with other governmental activities and responsibilities. TPI currently finances the salary of its regional coordinators in charge of maintaining the network functioning. The ET does not have information of future resources after the project ends. CSO members use their own resources to participate in the network’s activities, including their home internet, as the networks do not have a physical residence. Likewise, fieldwork demands equipment (helmets and safety footwear and garments) that are difficult to afford by members. 58 Respondents who spent 10 to 14 hours: 3 in Lima and 3 in Cusco. Respondents who spent 5 to 9 hours: 2 in Piura, 4 in Lambayeque, 1 in Lima, 3 in Loreto, and 2 in Cusco. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 32 Table 5. MCLCP’s annual approved budget (2019–2022) Year Current Soles Constant Soles 2019 4,112,432 4,736,840 2020 4,689,624 5,297,039 2021 5,068,335 5,379,024 2022 5,088,826 5,088,826 Source: MEF, SIAF, consulted Sept 13, 2022 https://www.sunat.gob.pe/indicestasas/index.html Interviewers explained that the activities of the IN are greatly complementing the Institutional Dimension Agenda of the MCLCP. However, when representatives of the MCLCP join the network, they are expanding their activities with the same allocated budget. Table 5 summarizes the MCLCP’s annual budget (nominal and real), showing that it did not change greatly from 2020 to 2021 and decreased in 2022.59 HOW DO THE INTEGRITY NETWORKS COMPARE TO PROGRAMA MONITORES CIUDADANOS DE CONTROL ORGANIZED BY CGR AND TO THE VEEDURIAS CIUDADANAS OF THE CITIZEN ENGAGEMENT FOR A TRANSPARENT RECONSTRUCTION PROGRAM? ARE THERE SYNERGIES OR OVERLAPS? ARE THE MEMBERS OF THESE LATER INITIATIVES ALSO PARTICIPATING IN TPI’S INTEGRITY NETWORKS? The CGR Monitores program was decentralized in August 2021, which allowed the regional CGR managers to lead the program more closely to their needs. The CGR has experience training citizens (monitores) and providing accreditation after a specific number of hours of volunteer work. The agency has a roster of trained monitores, together with information on the level of education, so they are able to contact the closest local monitores to construction sites that are far from urban communities. During the field visits, monitores could go together with an engineer and learn about the quality of the infrastructure and how to record entries in the construction logbook. Monitores can execute the successful visual inspection of infrastructure projects with few technical complexities. According to a respondent related to the government’s program, the monitores visit the sites with safety gear and prepared guidelines (visitas estructuradas).60 After receiving the field visit reports, the CGR could issue an order to send specialized staff to follow up on the observations. The INs work at the same time as the CGR program so there is a potential synergy. The difference with the INs in terms of scope is that the CGR monitores program includes several small projects and topics but with a limited horizon to oversee. Monitores focus on the construction phase, and they observe the physical progress, the presence of the supervisor engineer, and the logbook. In comparison, the IN 59 To obtain a budget increase, the organization would have to provide evidence for the need of more funds to its Budgetary Unit (Unidad Ejecutora) and Budget Head (Pliego presupuestario), which in turn submits the budget project to MEF, one year before the fiscal year. The MCLCP would have to formally include the new responsibilities in their bylaws as a support. If the request is not approved at the end by MEF, staff and advocates could work the case with MEF or later with the Budget Commission of Congress. 60 Shack, N. & Arbulú, A. (2021). Una aproximación a los mecanismos de participación ciudadana en el Perú. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 33 methodology allows members of one network to focus on a couple of projects, with a value of 10 million soles or more, and keep track of their progress in a longer period with a more complex tool. The citizen oversight experience (veedurias ciudadanas) under the Citizen Engagement for a Transparent Reconstruction project was initiated in 2018 with the goal of strengthening citizen’s oversight and monitoring the infrastructure financed by the Reconstruction with Changes Agency (implemented by subnational and national agencies).61 The geographical location of the project covered the areas struck by the El Niño flooding in 2017, mainly the North Coast. The implementation strategy entailed a central implementor coordinating with national agencies like the CGR, OSCE, MEF and SIP, while the regions had an established Promoting Committee (Comite Impulsor) in charge of the oversight. The program veedurias ciudadanas oversaw education, health, and agriculture projects. Together with a structured training program, the methodology covered the monitoring of the procurement processes in the OSCE platform and visits to the infrastructure premises under reconstruction. The process included interviews with the beneficiaries of the investment projects, such as school parents, health professionals, and neighbors, to enrich the analysis and verify any possible distortions. After the site visits, the monitors compared the information gathered with the government platforms reports.62 The veedurias ciudadanas had some similarities to the current strategy of the INs, including their coordination with the regional governments. However, the INs go beyond that and intend to include the regional governments as members. Although the veedurias ciudadanas are not active now, some former members joined the INs and brought the experience they gained when they were part of the veedurias. Some members have received training from the CGR and have been involved in its “Monitores Ciudadanos de Control” program as well. In the case of Piura, one IN member who is a direct beneficiary of one the projects selected in 2022 by the regional government and TPI also participated in the program of community representatives managed by the regional OI. The office institutionalized citizen oversight at least 11 years ago by setting an accreditation of CSOs and establishing normative guidelines for the community. The volunteers received training and monitored projects in their jurisdictions. WHAT HAPPENS TO THE LESSONS BUILT BY THE INTEGRITY NETWORKS WHEN THE SELECTED PROJECTS’ EXECUTION IS FINALIZED AND DELIVERED TO THE GOVERNMENT? One of the members’ and specialists’ concerns was that the IN recommendations were not binding. So, even when networks have provided recommendations or alerted about possible halts on the investments, they cannot influence regional government’s future behavior to improve their procurement processes. 61 USAID. (2021). Civil Society Engagement Assessment. 62 Observatorio de Integridad. (2018). Op. Cit. Veeduría Ciudadana. (2019ª). Op. Cit. Vigilancia ciudadana en instituciones educativas. Op. Cit (2019b) Vigilancia ciudadana en establecimientos de salud. Op. Cit. (2020ª) Operativo a Instituciones Educativas. Op. Cit. (2020b) II Operativo Virtual Operativo Arranca Perú. Segunda fase: Acceso a la información pública. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 34 At the time of the mid-evaluation implementation, no project had been finalized yet.63 Also, there have been changes in the selected projects because of lack of allocated budget or arbitration due to differences between the contractors and the regional governments. WHAT ARE THE CAUSAL PATHWAYS THROUGH WHICH INTEGRITY NETWORK IMPLEMENTATION LEADS TO INCREASED CITIZEN OVERSIGHT, IMPROVED GOVERNMENT PROCUREMENT PROCESSES, AND GREATER INTEGRITY? The logic model in Annex III presents a set of causal factors related to the implementation and development of the INs. The factors identified should lead to increased citizen oversight to influence improvements in the government procurement processes. Networks made progress in three phases, encountering enabling and limiting factors in each. • Phase one is defined as contact-building and information-sharing. TPI promotes the creation of the networks by inviting interested organizations, prioritizes projects, and provides training on public procurement, public investments, ISs, and integrity in general. • In phase two, members coordinate and increase their involvement in the INs. Members are expected to provide feedback on the working plans and tasks, apply and monitor the ISs, and discuss, analyze, and determine the projects’ compliance with the standards. Furthermore, as phases one and two progress, the IN members begin to make their own decisions and execute the action plan together. • Lastly, in phase three, members collaborate. The regional governments are expected to provide information, increase transparency, and be willing to use and comply with the ISs. The three phases can be considered a continuum. Once they reach phase three, the networks could still be developing activities of the first two phases, like welcoming new members or improving training materials monitoring tools. Based on the analysis of the collected information, the following process and variables were observed: PHASE ONE • The INs were created with diverse members. The high involvement of CSOs and the role of the MCLCP was fundamental to roll out the networks. Also, the membership of the regional CGR and professional associations added technical support. The participation of the OIs was instrumental to channel communication with the rest of the regional government. Lack of resources was the main limiting factor. • TPI and regional governments selected PIPs. Some of them had to be replaced due to lack of budget, contract delays, and arbitration. Procurement issues emerged in the projects, either on the execution or formulation phase. 63 Proyectos de inversión pública priorizados con contrapartes del Perú. Proyecto inversión pública transparente DE USAID/ PERÚ. Updated April 21, 2022. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 35 • TPI designed and validated the ISs. The tool considered 15 standards and their respective risks. ISs related to compliance with the CGR concurrent control and field visits to construction sites stand out. In addition, other ISs reflect some of the OSCE procurement risks. • TPI designed the training content and materials. The initial training program was run virtually because of the pandemic. However, this modality only allowed limited interaction and networking. PHASE TWO • IN trained in ISs and transparency tools. The IN members appreciated the practical examples and training provided by practitioners. Some standards (use of open data platforms and observation visits) were more appealing to CSO members. Weaknesses in the Technical Construction Studies hinder project execution, limiting the use of the standards. TPI implemented constant training to cope with members’ turnover and newcomers. • Journalists reported integrity gaps in public works after receiving training. There was a positive production of media reports. However, the main limiting factor is the lack of resources in regional media companies. • IN and private sector prepared joint action plans. The engagement of the private sector was possible in only one IN, which allowed the preparation of a joint action plan between the CSOs and the public and private sector members; in the rest of INs, the CSO and public sector members came together to prepare their plans. New young members brought technological knowledge, and the inclusion of infrastructure users supported the implementation of the plans. • TPI disseminated the integrity tools and methods, and the INs began using them. TPI’s engagement with national entities enabled the use of a complementary integrity tools (besides the IS), like the PCI of the SIP and the evaluation of procurement risks of OSCE. These tools were used for oversight and applied to local and regional governments. PHASE THREE • Application of ISs to PIPs are replicated in the regions. Innovation played an important role in consolidating the method and tools to oversight of public investments. An OI designed a monitoring platform to follow up on the standards in 10 more projects. Moreover, TPI’s partner designed a QR and a platform (Ojo en la Obra) to monitor projects’ concurrent control compliance, which led to regional government engagement with the IN. • CSO with enhanced oversight capacity applied lessons learned. While this action is still developing, there are examples that illustrate this phase: One IN decided to monitor the PCI of a municipality and apply the IS with its own indicators and metrics to an additional local sanitation project. Members recognized that they can only monitor a limited number of standards, depending on the execution phase of the selected projects (construction stage in most cases). One project under direct administration of the regional government was monitored with an adjusted IS tool. • Monitored regional governments are open to accountability and improve their internal control and compliance capacity. There are various experiences in the five networks. Regional governments with a specialized OI are more inclined to work with the IN; other networks CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 36 strategically use technological tools or improve the communication with regional and local governments through key stakeholders. Members and other interviewed stakeholders pointed out that public sector staffers’ capacities limit the effective procurement and investment supervision, which adds to the issue of lack of transparency and misconducts. There is, however, a positive effect of the national preventive and anti-corruption policies that TPI is supporting, and this benefits the work of the networks and regional government practices. INs’ main achievements The Piura Integrity Network successfully integrated the regional IO as a member, which permitted the digitalization and adoption of the ISs by the regional government. In turn, the regional government worked toward the application of the ISs to other regional projects. The Lambayeque Integrity Network took advantage of the CRA and MCLCP members’ experience with monitoring tools and proactively expanded its oversight to municipal projects. It also embraced the practice of the PCI to test the application of the tool. The Lima Province Integrity Network, despite its difficult territory, has been effectively coordinated by the MCLCP. The network successfully incorporated infrastructure users and the Chamber of Commerce and Production as IN members. The Loreto Integrity Network, with a majority of CSO organizations as members, and with support of regional coordinators, has initiated the use of a QR to show the updated information of the projects, and of the Ojo en Obra platform, which presents the CGR’s control reports and the follow￾up of the regional government in a friendly format. The regional government has positively responded to the tool and engaged with the network. The Cusco Integrity Network has the same support with the QR and Ojo en la Obra platform. The network has successfully included more CSO members to complement the work of the CRA. IN members, with support of TPI, are adjusting ISs to use them in a project with direct administration. EVALUATION QUESTION 3. TO WHAT DEGREE DID TPI’S INTEGRITY NETWORKS’ ACTIVITIES AFFECT DOWNSTREAM OUTCOMES? The ET did not find significant effects of the INs on PIPs’ budget execution, physical execution, or changes in investment costs—we are considering a 5% threshold to determine statistical significance. The ET compared projects prioritized by the regional governments and monitored by the INs with similar projects in the same regions and in other regions where TPI did not implement the INs during the proof-of-concept phase. We did not expect to find statistically significant differences at midline due to the small number of treatment projects and the limited amount of time INs have been working. We plan to repeat this analysis at endline and expect to find significant differences between prioritized projects and other projects. Annex VI includes our estimation results, using data collected until May 2022. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 37 Out of the three variables that proxy for corruption that we studied, we find that the treatment and comparison PIPs have similar levels of budget and physical execution (i.e., the mean budget/physical execution of the treatment group is not lower or higher than the mean budget/physical execution of the comparison groups). In the case of changes in investment costs, treatment projects show higher increases in the total cost as a percentage of the budget approved in the feasibility studies than comparison projects in other regions. However, we did not find a significant difference when comparing the budget increases of the treatment projects and the comparison projects in the same regions. Moreover, on average, PIPs in the treatment and both comparison groups ended up being more expensive than originally planned when they were approved by the Investment Offices. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 38 V. CONCLUSIONS EVALUATION QUESTION 1 OVERALL • Networks face two main limitations to receiving response to their monitoring’s recommendations: the regional public officers’ lack of will and/or technical capacity to address deficiencies in their project management and the monitoring instruments that cannot sort out technical problems generated in the pre-construction phase. • Public servants could conduct very limited oversight during the construction phase, so the application of the ISs could encounter more challenges when private constructors face no close supervision from the regional governments. INTEGRITY NETWORKS’ PURPOSE AND MEMBERSHIP • TPI has successfully supported the creation of five INs as planned. • The strongest element of the networks’ functioning is their members’ commitment to oversee the use of public resources. • The common good (fighting corruption) and commitment prevails over particular interests, especially for civil society representatives. • The number of member organizations has increased from September 2021 to June 2022. However, the goal to have a diverse membership with CSOs and public and private members is still in progress. Organizations like the professional associations of engineering and journalism and public sector practitioners are seen as key members or partners to foster the networks’ development. • TPI promotes women’s participation and their active voices in anticorruption activities. The number of women involved in the IN increased from 21 to 40 between September 2021 and June 2022. Female members hold key posts to promote citizen’s oversight. The Piura, Lambayeque, Cusco, and Loreto regions had or have women from the MCLCP as Technical Secretariats. • The MCLCP has had an essential role in articulating the work of the IN due to its legal status and positive public perception. SELECTED PROJECTS • CSO members expressed interest in participating in the selection of projects to be monitored in the future. • There was a consensus among respondents that the Technical Construction Studies (expedientes técnicos) are poorly designed or outdated. This affects the projects’ execution and limits the application of the ISs. • The selected projects are similar to each other in terms of cost increase and delays. However, those executed via direct public procurement, in comparison to those with private contracting, require more effort to monitor the procurement of a larger number of services and goods. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 39 MONITORING TOOLS • The ISs are used to press for transparency from the regional government, but members also expect to influence project delivery and avoid construction halts. • INs adopted ISs to oversee infrastructure projects. IN members were able to apply or monitor those ISs that required information from governmental open data or practical activities like the civil works site visits. In contrast, some ISs are more difficult to apply or monitor because they require specialized analysis from procurement or investment practitioners. • Having infrastructure experts in the INs helps non-expert members embrace the standards. • IN members welcomed the application and monitoring of the PCI backed by the Public Integrity Secretariat. The PCI provides a comprehensive view of the regional government status and plans to improve their integrity. Members also showed interest in learning more about the OSCE’s risk management approaches. • Members recognized the usefulness of digital tools like Piura’s IS platform to monitor the compliance of the standards with information provided by the regional government, the QR to visualize updated and comprehensive project information, and the Ojo en la Obra platform that presents a simplified version of CGR’s control reports. TRAINING AND COACHING • Even though the members considered the procurement process complex, they valued the training they received and looked forward to more case studies. Complementary practical training activities like site visits helped INs members understand the ISs. • IN members have a favorable opinion about the training on open information sources used to monitor the regional project’s compliance of ISs. These online sources allowed members to collect and analyze more information than what was provided by regional governments to monitor the selected projects. EVALUATION QUESTION 2 OVERALL • Factors affected IN in different degrees, supporting or limiting their networking, governance, and operations. ENABLING FACTORS • National preventive and anticorruption policies convey a positive environment for the discussion of integrity in the public sector by different stakeholders, including civil society. • The negative national and regional political environment has triggered the commitment of civil society to oversee PIPs. • The modernization of the control system with the application of CGR’s preventive actions like the concurrent control since 201764 and the receptiveness of the CGR’s participation and feedback strengthened the INs activities and oversight tools. 64 CGR. (2021). La reforma del control gubernamental en el Perú. Balance al trienio de su implementación. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 40 • The INs have successfully gathered members with diverse backgrounds, complementary skills, and a shared purpose. • The commitment and dedication of CSO members is one the main assets of the INs. • IN members value the experience and background of the MCLCP and the technical support of TPI, especially regional coordinators. • IN members’ previous oversight experience is an asset. • INs successfully brought youth organizations into their networks, and those youth organizations brought technical capacities to the INs. • The efficacy of the IO has been positive in regions with a specialized office in the regional government structure. HINDERING FACTORS • The lack of accountability tradition in regional public agencies increased the risk of lack of transparency and ineffective use of public resources, leading to CSOs’ frustration. • High turnover of regional government staffers and CSO members limited networking and capacity￾building. • The lack of incentives and the limited dissemination of the IN’s achievements could limit the participation of CSO representatives in the future. • Limited members’ availability and the lack of monetary resources for overseeing activities have a negative effect on CSOs’ participation. PREVIOUS OVERSIGHT EXPERIENCE • There is a potential synergy between CGR’s Monitores program and the INs’ work, as both encourage trained civil society to oversee public works. The main difference is the time spent in each infrastructure project. Monitores specialize in observation visits to several construction sites, while the IN members cover all the procurement process, including site visits to two selected projects. INSTITUTIONALIZATION AND FUTURE COLLABORATION • In one regional government, the OI, as a member of the network, has institutionalized the ISs tool. • TPI has designed and implemented digital tools, which could be used in other oversight experiences. • Members have expressed interest in continuing to use the ISs. While dissemination and collaboration among networks and regional governments might happen in the future, there is no clear path for more institutionalization at the time of the midline evaluation. EVALUATION QUESTION 3 • The ET evaluated budget execution, physical execution, and changes in investment costs and did not find significant effects of the INs on these three outcome variables that proxy for corruption. We did not expect to find statistically significant differences at midline due to the small number of treatment projects and the limited amount of time INs have been working. We plan to repeat this analysis at endline (with more data), as the INs might need more time to affect these outcome variables. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 41 • The budget and physical execution of the treatment projects (i.e., those monitored by the INs) is not significantly different from that of comparison projects selected in the same regions and in other regions without INs. However, the increases in the total investment costs of the projects monitored by the INs are higher than of comparison projects in regions without INs. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 42 VI. RECOMMENDATIONS OVERALL • Promote the establishment of specialized OIs outside the regional government’s human resources office. This model has benefited the work of the regional government and the INs. Members doubt that a general manager or human resources office would have enough time to deal with all the integrity responsibilities of a regional government. • Working along with national agency stakeholders, explore the implementation of an incentive program that rewards more transparency and accountability (reflected, for example, in a better score in the Preventive Capacity Index [PCI] or other milestones) in regional and municipal governments. MEF has different models under implementation that could be taken as a reference.65 MEMBERSHIP • Include the Regional Council (Consejo Regional) as a potential member of the network, as it has regulatory and control functions over the regional government’s executive branch; this would ease the requests of information to the regional administration. • Invite private sector actors like the regional chapter of the Chamber of Commerce and Production to the networks. As users of the infrastructure projects, their interest is related to their needs to reduce costs and increase productivity. • Promote a strong involvement of representatives of the Professional Association of Engineers and other members with sound technical expertise in the infrastructure field, so they can share their practical knowledge with non-expert members to identify projects’ main problems. SELECTED PROJECTS • Involve IN members in the selection of projects to promote their participation in the networks. • Examine how projects foster regional economic growth to select the ones that the IN will monitor. • Explore if the ISs can capture the need to have an updated Technical Construction Study before the construction phase. MONITORING TOOLS • The Technical Secretariat could carefully explain to the members that the monitoring tools seek to elevate transparency and good practices in procurement but do not guarantee faster and more efficient project execution. This will prevent members’ frustration with delays during project execution. • Support the exchange of initiatives and information among networks, including the design and creation of the QR that summarizes information on the projects’ execution stage and the Ojo en la Obra platform with user friendly information of the CGR’s concurrent control reports. Support the dissemination of the ISs platform developed by the Piura OI to other regions. 65 MEF. (n.d.) Incentivos para el gobierno nacional, gobiernos regionales y gobiernos locales. In www.mef.gob.pe CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 43 • Continue supporting the PCI. The PCI gives a comprehensive view of the preventive processes and tools and is supported by the Prime Minister Office’s bylaws.66 Considering that some ISs are harder to monitor than others, the PCI is an adequate tool that can be monitor by all members. • Look for synergy with the CGR’s Monitores program to gain expertise and share costs during the site project visits. TRAINING AND COACHING • Develop guidelines on how to apply and monitor each standard. Include who should apply and who should monitor the ISs and how to access information sources. Guidelines should also define goals, indicators, and metrics. Personalize training sessions based on each member’s diverse academic background and work experience. • Prepare case studies on cancelled or delayed projects to educate members on how to deal with or avoid those issues in future projects. • Share lessons learned with new regional governments (elected October 2, 2022). RESOURCES • An institution like the MCLCP could host the INs if they decide to keep working after TPI ends its operations and the MCLCP is able to leverage funds. 66 For example, the results of the first evaluation of the PCI could lead to strategies to improve the organizational culture and management of risks of the regional governments. The reports indicated that while Piura regional government obtained a high score of 70 percent, the Lima Province region obtained the low score of 8 percent. In between, Cusco obtained a score of 50 percent, Lambayeque and Loreto obtained the same score of 38 percent. In Integrity reports – December 2021. https://reporteintegridad.servicios.gob.pe/ CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 44 ANNEX I. INTEGRITY STANDARDS This annex presents the integrity standards and risks as exactly as they are described in the TPI Public Contracting Integrity Standards English Version. PUBLIC CONTRACTING PHASE N° INTEGRITY STANDARD INTEGRITY RISK PREPARATORY PHASE 1 The investment is linked to closing gaps in basic services in accordance with the prioritization criteria Warn and avoid against the selection of an alternative investment that does not respond to the interests of its user population and that does not represent a comprehensive solution to the problem which has been identified. 2 The areas involved in the investment manage the project in a comprehensive manner, have a protocol and a manual or automated mechanism for alerts and communication strategies to prevent and resolve incidents and controversies The mechanisms for transparency and access to information can be affected if key actors do not coordinate in a clear and timely manner to warn against or jointly respond to situations (allegations, questions, unforeseen changes, among others) which can put in risk the continuity of the investment. 3 The obliged subjects that participate in the management of the contracts submit their sworn declarations of interests within the time period defined in the applicable regulations. Avoid that staff or public servants engage in conflicts of interests which can affect the transparency and integrity of contracting. 4 The user area submits the complete, updated and organized technical file of the work to the Body in Charge of Contracting Avoid that the requisition lacks the minimal conditions for its formation. On occasions continuity is given to subsequent stages of contracting without having verified that the technical file of the work and the requisition of goods and services are consistent and complete, putting at risk transparency, integrity and competition. 5 The market study gathers a plurality of suppliers interested in participating in the selection process. Avoid cases of bid rigging, price fixing or collusive market sharing among suppliers to reduce competition, due to the fact that greater dissemination and the demonstration of transparency to the market could generate confidence to participate in the selection process. SELECTION PROCESS 6 The announcement of the terms of reference and selection procedure is carried out in a complete and timely manner in the SEACE and in accordance with what was approved in the preparatory acts. Avoid illegal modifications to the information contained in the approved documents in the preparatory phase or delays to the announcement of terms of reference for reasons which are not justified or which lack transparency. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 45 PUBLIC CONTRACTING PHASE N° INTEGRITY STANDARD INTEGRITY RISK 7 The acquittal of queries and observations are resolved responsibly, promote competition and plurality and within the established time frames. Situations can arise in which the special committee discourages the submission of queries or observations from a participant by indicating a requirement that violates regulations and principles of competition and equal treatment, and consequently, disincentivizes participation in the selection process. As such, the possibility that established time frames are not satisfied should be reduced, as this can result in the invalidation of a procedure. 8 A plurality of proposals exists, and they are reviewed in accordance with the terms of reference and the regulatory framework for contracting in order to select the winning bidder. Cases exist in which the best proposal does not win, due to influence or steer having been applied during the evaluation, or changes not motivated by the regulatory framework, or as result of political pressure, which can put at risk the quality and opportunity for delivery 9 The entity conducts an audit after the documents have been submitted by the winning bidder to verify the accuracy of the information provided. Most sanctions applied to suppliers are the result of infractions linked to the submission of false or inaccurate information during the selection process. Consequently, it is necessary to verify the accuracy of the documents presented in a timely manner, to avoid putting the contract at risk due to dishonest behavior. CONTRACTUAL EXECUTION 10 The documents for the signing of the contract are evaluated in accordance with the requirements of the terms of reference and the regulatory framework for contracting This seeks to avoid any arbitrary decision to request additional documents beyond that which are formally established, and which can affect the principle of integrity and transparency during the process of signing the contract 11 The supervision of the delivery of the works is permanent and designed to prevent and resolve aspects which could affect their continuity. During the delivery of the contract the supplier can engage in all types of conduct to co-opt the supervisor and the staff from the entity, so that modifications can be made to the workplan in order to reduce the level of investment in the works and to secure deadline extensions without the existence of adequate justification. 12 Additions or amplifications are evaluated within the established time frames and justifications are provided in accordance with the applicable conditions One of the most questioned aspects is that works begin with an initial budget but then conclude with a much larger one. With this in mind, it is important to evaluate the compliance of the conditions for approving increases to budgets for works, in such a way that principles of integrity, transparency, efficacy and efficiency and not affected. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 46 PUBLIC CONTRACTING PHASE N° INTEGRITY STANDARD INTEGRITY RISK 13 The valuation and payment are made within the established time frames and in accordance with the requirements of the regulatory and administrative frameworks. During the delivery of a contract significant pressure exists for the manifestation of conformity for the good or service, or the approval of valuations and payments, in order to ensure greater liquidity for the suppliers, which can give rise to the use of unethical or illegal mechanisms to achieve this, which can affect principles of transparency and integrity. 14 The recommendations made by the internal control body of the entity during the contracting stages were implemented. Situations exist in which the internal control body warns of risks or compliance failures due to omission or for decisions made which violate the regulatory framework, which in many cases result in recommendations which are not implemented in a timely manner. 15 The final reception of the works is done in accordance with the applicable time frames and procedures. From the moment in which the best alternative investment is selected in response to a public need, many problems exist in achieving the reception of the completed works. In addition to assessing the physical and financial progress of the works, the risks which may affect its conclusion and transfer for timely use should also be identified in a context involving changes to management or the absence of budget in the final stage of the process. Source: Integrity Standards for Public Contracting. USAID/PERU Transparent Public Investment Project. In TPI Public Contracting Integrity Standards English Version 20.05.2021. Manuscript. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 47 ANNEX II. STATUS OF SELECTED PROJECTS (APRIL 2022) N PROJECT TYPE OF INVESTMENTADMINI STRATION / PROCESS PHASE ANTICIPATE D DATE OF END AS OF APRIL 21 2022 INVESTMENT COST AS OF APRIL 2022 MM SOLS 1 2402677 PIURA TRANSPORTATION MEJORAMIENTO DE LA CARRETERA DEPARTAMENTAL RUTA PI-101: EMP. PI-100 - NEGRITOS-VICHAYAL-PUENTE SIMON RODRIGUEZ-PUEBLO NEVO DE COLAN - COLAN-EMP. PI-102 Y RUTA PI-116, EMP. PE 1N TAMARINDO-AMOTAPE-EMP. PI-101, DISTRITO DE PARIÑAS - DISTRITO DE LA BREA - PROVINCIA DE TALARA, DISTRITO DE VICHAYAL - DISTRITO DE AMOTAPE - PROVINCIA DE PAITA, DISTRITO DE ARENA INDIRECT ADMINISTRATION / WITH CONTRACT CONTRACTUAL EXECUTION October 2023 (Date extended from March 2022) According to TPI source, the project is in its final execution phase. 207 Adjusted from 201 MM 2 2437053 PIURA HEALTH MEJORAMIENTO DE LOS SERVICIOS DE SALUD DEL ESTABLECIMIENTO DE SALUD LOS ALGARROBOS DISTRITO Y PROVINCIA DE PIURA-DEPARTAMENTO PIURA. RCC INDIRECT ADMINISTRATION / WITH CONTRACT CONTRACTUAL EXECUTION / CONTRACT TERMINATED. October 2021 (Adjusted from January 2022) Extension requested by contractor was not approved 64 3 2397009 PIURA EDUCATION MEJORAMIENTO Y AMPLIACIÓN DEL SERVICIO EDUCATIVO DEL INSTITUTO NACIONAL INDUSTRIAL FEMENINO N°48 DE LA CIUDAD DE SULLANA - DISTRITO Y PROVINCIA DE SULLANA - PIURA CENTRO POBLADO DE SULLANA - DISTRITO DE SULLANA - PROVINCIA DE SULLANA INDIRECT ADMINISTRATION / WITH CONTRACT SELECTION PROCESS October 2022 12 4 2463218 PIURA TRANSPORTATION MEJORAMIENTO SERVICIO DE MOVILIDAD URBANA EN LAS CALLES DE LA URBANIZACIÓN MIRAFLORES I, II Y III ETAPA DEL DISTRITO DE CASTILLA - PROVINCIA DE PIURA INDIRECT ADMINISTRATION / WITH CONTRACT SELECTION PROCESS April 2023 35 5 2378916 LAMBAYEQUE EDUCATION MEJORAMIENTO Y RECUPERACIÓN DEL SERVICIO EDUCATIVO EN LA I.E. PEDRO ABEL LABARTHE DURAND, DISTRITO CHICLAYO Y PIMENTEL INDIRECT ADMINISTRATION / WITH CONTRACT CONTRACTUAL EXECUTION December 2022 Adjusted from 2024 25 6 2413934 LAMBAYEQUE EDUCATION MEJORAMIENTO DEL SERVICIO EDUCATIVO EN LA I.E N° 10836 APLICACIÓN, DISTRITO DE JOSE LEONARDO ORTIZ - PROVINCIA DE CHICLAYO - DEPARTAMENTO DE LAMBAYEQUE INDIRECT ADMINISTRATION / WITH CONTRACT PREPARATORY PHASE October 2022 Date updated from April 2023 38 CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 48 N PROJECT TYPE OF INVESTMENTADMINI STRATION / PROCESS PHASE ANTICIPATE D DATE OF END AS OF APRIL 21 2022 INVESTMENT COST AS OF APRIL 2022 MM SOLS 7 2378107 LIMA PROVINCIA EDUCATION MEJORAMIENTO, AMPLIACIÓN DE LOS SERVICIOS EDUCATIVOS DEL NIVEL PRIMARIO Y SECUNDARIA DE LA I.E. N 20523 CORAZON DE JESUS DEL DISTRITO DE SUPE, PROVINCIA DE BARRANCA, REGION LIMA INDIRECT AND DIRECT ADMINISTRATION / CONTRACT AND DIRECT SERVICES PROCUREMENT PART CONTRACTUAL EXECUTION AND PART EXECUTION DIRECTLY BY GOVERNMENT June 2022 Date extended from May 2021 20 8 2223388 LIMA PROVINCIAS HEALTH MEJORAMIENTO DE LOS SERVICIOS DEL CENTRO DE SALUD DE QUILMANA DEL, DISTRITO DE QUILMANA - CANETE - LIMA INDIRECT ADMINISTRATION / WITH CONTRACT CONTRACTUAL EXECUTION May 2023 Extended from October 2021 21 Adjusted from 19 MM 9 2255793 LORETO HEALTH CONSTRUCCIÓN Y EQUIPAMIENTO DEL NUEVO HOSPITAL DE IQUITOS CESAR GARAYAR GARCIA / PROVINCIA DE MAYNAS INDIRECT ADMINISTRATION / WITH CONTRACT CONTRACTUAL EXECUTION February 2023 Extended from December 2021 306 Adjusted from 279 MM April 2021 10 2194704 LORETO TRANSPORTATION MEJORAMIENTO DE LA VIA VECINAL EMPALME LO - 103, HASTA EL CENTRO POBLADO SANTO TOMAS Y ACCESO A LA COMUNIDAD SANTA CLARA, DISTRITO DE SAN JUAN BAUTISTA - MAYNAS - LORETO INDIRECT ADMINISTRATION / WITH CONTRACT CONTRACTUAL EXECUTION February 2023 Extended from October 2022 60 11 2194625 CUSCO TRANSPORTATION MEJORAMIENTO CARRETERA RIO BLANCO - MOLLEPATA - CU 109, DISTRITO DE MOLLEPATA, PROVINCIA DE ANTA, REGION CUSCO DIRECT ADMINISTRATION EXECUTION DIRECTLY BY GOVERNMENT January 2023 Considered delayed, extended from September 2021 48 (41 MM as of April 2021) 12 2192979 CUSCO TRANSPORTATION MEJORAMIENTO INTEGRAL DE LA VIA EXPRESA DE LA CIUDAD DEL CUSCO: OVALO LOS LIBERTADORES - PUENTE COSTANERA - NODO DE VERSALLES BEGINING CONTRACTUAL EXECUTION / WORLD BANK RULES June 2023 Adjusted only one month 326 (324 MM as of April 2021) Prepared based on the following sources: Proyectos de inversión Pública priorizados con contrapartes del Perú. PROYECTO DE INVERSIÓN PÚBLICA TRANSPARENTE DE USAID-PERU. Doc. actualizado a septiembre 2021. Proyectos de inversión pública priorizados con contrapartes del Perú. Proyecto inversión pública transparente DE USAID/ PERÚ. Documento actualizado al 21 de Abril de 2022. Note: The execution modality of all the project is through contracts with third parties, except for one project in Cusco (that is directly administered by the regional government) and one in Lima (using both modalities: contract, and direct administration). CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 49 ANNEX III. LOGIC MODEL OF THE INTEGRITY NETWORKS’ STRATEGY REGIONAL LEVEL NATIONAL LEVEL Simultaneous control applied to prioritized projects by the CGR. Tools and regulations of open government data, citizen oversight and others made available by Integrity Secretariat and OSCE Integrity Standards designed and validated INs (IN) created with diverse members Public Investment Projects (PIP) prioritized Training material, guides, protocols available IN trained in integrity standards and transparency tools Journalists trained and reporting integrity gaps IN and private sector prepare joint action plans Integrity tools and methods disseminated/used RRegional government opens to accountability, and improves its internal control and compliance capacity Application of integrity standards to PIPs are replicated CSO with enhanced oversight capacity apply lessons Private sector adopts integrity standards Strengthened integrity in investment and procurement processes with collaboration among parties Enhanced culture of public integrity present among IN members CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 50 ANNEX IV. FACTORS RELATED TO THE INTEGRITY NETWORKS’ FUNCTIONING CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 51 ANNEX V. EVALUATION METHODOLOGY DATA SOURCES DOCUMENT REVIEW The purpose of the systematic document review was to develop a strong understanding of the INs’ strategy (including the implementation timeline and activities carried out to strengthen their members’ capacity) and analyze the ISs’ application in each phase of the investment cycle. The ET also reviewed institutional documentation of the CGR, the Ombudsman’s Office, OSCE and PCM. The documents are listed in Annex VII. NON-PARTICIPANT OBSERVATIONS Two embedded researchers attended two meetings of each of the five INs. The work entailed four short fieldwork trips and/or remote coordination between February and March 2022. The researchers gathered valuable information and administered a few interviews, testing some questions included in the instruments for the KIIs. The non-participant observations allowed the ET to get initial information on the INs and improve the interview questionnaires. KEY INFORMANT INTERVIEWS The ET conducted 55 KIIs and 5 FGDs (1 per region); Table 6 presents the breakdown of the interviews by organization type while Annex VIII lists all the key informants interviewed. In Lima city, we interviewed representatives from USAID/Peru and IPs, as well as CGR and OSCE managers, and the Public Integrity Secretariat (Secretaria de Integridad Pública, SIP) sub secretary. The ET also interviewed at least one member of each organization that participates in the five INs, including CSOs, the regional governments’ staff, and CRA members. From a total of 66 IN members as of June 2022, we interviewed 39. In addition, we conducted KIIs with other regional stakeholders, such as CGR regional managers, journalists, and regional coordinators in Piura, Lambayeque, Lima, Loreto, and Cusco. Likewise, the ET conducted FGDs with CSO members to explore the INs’ performance and validate the information obtained from the interviews. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 52 Table 6. Number of interviews per type of organization Organizations Number of Interviews Civil Society Organizations 20 Mixed Public Sector-Civil Society Organization: Mesa de Concertación de Lucha Contra la Pobreza (MCLCP) 6 National and Regional Public Sector Agencies 16 IPs and USAID Peru 13 TOTAL 55 Respondents’ Sex: Female 30 Male 25 WEB-BASED SURVEY The ET administered a web-based survey after conducting all the KIIs. The objective was to assemble a general panorama of available information, triangulating the survey responses with the qualitative data from the KIIs. At the end of June 2022, the ET sent almost 70 invitations to respond the survey and got 39 responses. ADMINISTRATIVE DATABASES The ET collected the data needed for the quantitative analysis from public sources, either from the Ministry of Economy and Finance (MEF) website (https://www.datosabiertos.gob.pe/) or submitting formal requests of information to the CGR and the OSCE. We used MEF’s data to analyze the budget execution as a percentage of the total investment cost starting in January 2020. Additionally, starting in December 2021, we downloaded information regarding the progress of the sampled projects from MEF’s Investments Monitoring System (Sistema de Seguimiento de Inversiones, SII) monthly. We gathered data for investment projects in regions where INs are being implemented and in other regions; data collection in regions with INs will include prioritized and non-prioritized investment projects. SELECTION OF COMPARISON PROJECTS The treatment group is comprised of 9 of the 10 original prioritized projects; the regional government of Piura stopped the execution of the other project. For each prioritized project, The ET selected three comparison projects in the same region (Piura, Lambayeque, Lima, Cusco, and Loreto) and another three comparison projects in similar regions. While the in-region comparison projects may be comparable in many characteristics to the treated projects (e.g., work culture, and political prioritization of projects), they may end up being a contaminated control since they could benefit from spillovers from the networks. To account for that, we also used out-of-region comparison projects, which are much CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 53 less susceptible to benefit from the networks given that they pertain to other geographical jurisdictions and are being executed by non-treated public servants. The ET selected the comparison group using data from 2020 (before the INs started functioning). We considered the sector (health, education, and transportation) and contract type (direct or indirect) of each treatment project to restrict their respective pool of potential comparison projects. We only considered projects with a CUI and removed those with an investment cost lower than 10 million soles from this pool. We used the project phase (approval, formulation, or implementation) as of December 2020 as an additional filter—when the sample size was too small, we selected the comparison projects from those projects in the closest phase to the treatment project. Finally, from the pool of eligible comparison projects, we selected those that are most similar to each treatment project. The similarity between projects was measured in terms of investment cost and budgetary execution capacity (calculated as the percentage of the available budget that was executed in 2020). The former is one of the INs’ selection criteria for their projects, while the latter is used as a proxy for ease of implementation. We have a total of 60 projects in our sample: 9 treatment projects, 24 comparison projects in the treatment regions, and 27 comparison projects in other regions. The pool of possible comparison projects for both education projects prioritized in Lambayeque includes the same three projects. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 54 ANNEX VI. REGRESSION RESULTS Outcome variable In-region comparison Out-region comparison Effect Std. Error P￾value N Effect Std. Error P￾value N Budget execution (1) (2) 9.491 5.270 0.072 928 9.508 5.270 0.071 986 Budget execution (1) (3) 6.169 5.436 0.256 928 7.485 6.556 0.254 986 Physical execution (4) 3.509 15.600 0.822 191 -3.635 14.568 0.803 204 Increase in investment cost (4) (5) 28.251 23.177 0.223 192 39.281 20.745 0.058 204 Notes: (1) Measured as a percentage of the updated investment cost. (2) We report ATT using CSDID. (3) We report ATT using SDID. (4) We report random effects models' regression coefficients. (5) Measured as a percentage of the budget approved in feasibility studies. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 55 ANNEX VII. DOCUMENTS REVIEWED Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W., & Wager, S. (2021). Synthetic difference￾in-differences. American Economic Review, 111(12), 4088-4118. Bonifaz, J. L., Urrunaga, R., Aguirre, J., & Quequezana, P. (2020). Brecha de infraestructura en el Perú. Estimación de la brecha de infraestructura de largo plazo 2019-2038. [Infrastructure gap in Peru. Estimation of the long-term infrastructure gap 2019-2038.] https://publications.iadb.org/publications/spanish/document/Brecha-de-infraestructura-en-el-Peru￾Estimacion-de-la-brecha-de-infraestructura-de-largo-plazo-2019-2038.pdf Callaway, Brantly and Pedro H.C. Sant’Anna (2020). Introduction to DiD with Multiple Time Periods. December 2020. Available at https://cran.r-project.org/web/packages/did/vignettes/multi-period￾did.html CAN (2017). Política Nacional de Integridad y Lucha Contra la Corrupción. [National Policy of Integrity and Fight Against Corruption.]. https://cdn.www.gob.pe/uploads/document/file/45986/Politica￾Nacional-de-Integridad-y-Lucha-contra-la-Corrupcio%CC%81n.pdf?v=1561501666 CGR (2021). La reforma del control gubernamental en el Perú. Balance al trienio de su implementación. Lima, Perú. CGR (2022, August 22). Contraloría: Existen 2346 obras públicas paralizadas por más de S/ 29 mil millones. [Comptroller: There are 2,346 paralyzed public construction projects for more than S/ 29 billion.] https://www.gob.pe/institucion/contraloria/noticias/643238-contraloria-existen-2346-obras￾publicas-paralizadas-por-mas-de-s-29-mil-millones. CGR (2022, January 17). Contraloría advierte que más de 2300 obras permanecen paralizadas a nivel nacional. [Comptroller warns that more than 2,300 public construction projects remain paralyzed nationwide.] https://www.gob.pe/institucion/contraloria/noticias/576893-contraloria-advierte-que￾mas-de-2-300-obras-permanecen-paralizadas-a-nivel-nacional Collective Minds (2021). What capacities do networks need to succeed? https://collectivemind.medium.com/what-capacities-do-networks-need-to-succeed-940c9b19b683 Defensoría del Pueblo (2019). Reporte La Corrupción en el Perú, Año I N° 4. Avances y retos en la implementación del Modelo de Integridad en el Poder Ejecutivo. [Progress and challenges in the implementation of the Integrity Model in the Executive Power.] https://www.defensoria.gob.pe/wp￾content/uploads/2020/08/REPORTE-4-Modelo-de-Integridad.pdf Defensoría del Pueblo (2021). Análisis y perspectivas del nuevo marco regulatorio de las comisiones regionales anticorrupción. Serie Informes Especiales N° 019-2021-DP. [Analysis and perspectives of the new regulatory framework of the regional anti-corruption commissions.] Primera Edición: Lima, Perú. Decreto Supremo N° 001-2001-PROMUDEH, January 2001. Constituyen Mesa de Concertación para la lucha contra la pobreza. [Constitution of the Roundtable for the Fight against Poverty.] https://mesadeconcertacion.org.pe/sites/default/files/decreto_supremo_001_-_2001-promudeh.pdf CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 56 Decreto Supremo N° 014-2001-PROMUDEH, July 2001. Establecen normas modificatorias y complementarias para el funcionamiento de la Mesa de Concertación para la Lucha Contra la Pobreza. [Establish modifying and complementary norms for the operation of the Roundtable for the Fight against Poverty.] https://www.mesadeconcertacion.org.pe/sites/default/files/decreto_supremo_014_0.pdf Leading Inside Out & Collective Impact Forum (2017). Community Engagement Toolkit. https://collectiveimpactforum.org/wp-content/uploads/2021/12/Community-Engagement-Toolkit.pdf Ley N° 27867 de 2002, Ley Orgánica de Gobiernos Regionales. [Regional Governments Organic Law N° 27867] https://cdn.www.gob.pe/uploads/document/file/2089900/Ley%20N%C2%B0%2027867%20Ley%20Org %C3%A1nica%20de%20Gobiernos%20Regionales.pdf.pdf?v=1629296173 MEF. (n.d.) Incentivos para el Gobierno Nacional, Gobiernos Regionales y Gobiernos Locales. [Incentives for the National, Regional and Local Governments.] Retrieved September 27, 2022, from https://www.mef.gob.pe/es/?option=com_content&language=es-ES&Itemid=101547&lang=es￾ES&view=article&id=2221 Observatorio de Integridad (2018). Veeduría Ciudadana. Reporte 1. November 2018. http://www.observatoriointegridad.pe/sites/default/files/2018-11/Reporte1-veeduria-ciudadana.pdf Observatorio de Integridad (2019a). Veeduría Ciudadana de la Reconstrucción. Reporte 2. Vigilancia ciudadana en instituciones educativas. [Citizen oversight in educational institutions] May 2019. https://www.scribd.com/document/425144858/REPORTE-2-Vigilancia-ciudadana-en-instituciones￾educativas#from_embed Observatorio de Integridad (2019b). Veeduría Ciudadana de la Reconstrucción. Reporte 3. Vigilancia ciudadana en establecimientos de salud. [Citizen oversight in health facilities.] August 2019. https://www.scribd.com/document/425145657/REPORTE-3-Vigilancia-ciudadana-en￾establecimientos-de-salud#from_embed Observatorio de Integridad (2020a). Reporte de Veeduría Ciudadana. Reporte 4. Operativo a Instituciones Educativas. [Operative to Educational Institutions] August 2020. https://www.scribd.com/document/476153659/Reporte-IIEE-Agosto-2020#download&from_embed Observatorio de Integridad (2020b). Reporte de Veeduría Ciudadana. Reporte 7. II Operativo Virtual Operativo Arranca Perú. Segunda fase: Acceso a la información pública. [Second phase: Access to public information] https://www.scribd.com/document/487839016/Operativo-Arranca-Peru-segunda￾fase?fbclid=IwAR06Y6cT8mXh6xZzbi3b5PQP2Ya8jW4dbknQ91oFR_Ohghga3IG6dxFGk7Q ORSIMPACT & Spark Policy Institute (2018). When Collective Impact has an Impact. https://www.orsimpact.com/DirectoryAttachments/10262018_111513_477_CI_Study_Report_10- 26-2018.pdf OSCE (2020). Diagnóstico y Estrategia para la Gestión de Riesgos en Contratación Pública. [Diagnosis and Strategy for Risk Management in Public Procurement.] https://cdn.www.gob.pe/uploads/document/file/1038474/Diagn%C3%B3stico_y_Estrategia_para_la_ Gestion_de_Riesgos_en_Contrataci%C3%B3n_P%C3%BAblica.pdf?v=1595541496 CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 57 OSCE (2021). Informe de resultados de línea base de gestión de riesgos. [Risk management baseline results report.] https://cdn.www.gob.pe/uploads/document/file/2211302/Informe%20de%20resultados%20de%20l%C 3%ADnea%20base%20de%20gesti%C3%B3n%20de%20riesgos%202.pdf?v=1632845490 Secretaria de Integridad Publica. Integrity reports – December 2021. https://reporteintegridad.servicios.gob.pe/ Secretaria de Integridad (2022). Reporte de Integridad. Evaluación del Modelo de Integridad. Gobiernos Regionales. July. https://reporteintegridad.servicios.gob.pe/doc/EVALUACION_GORES_ICP_JULIO_2022.pdf Shack, N. & Arbulú, A., (2021). Una aproximación a los mecanismos de participación ciudadana en el Perú. [An approach to citizen participation mechanisms in Peru.] Contraloría General de la República. Lima, Perú. https://cdn.www.gob.pe/uploads/document/file/1707155/Una%20aproximacio%CC%81n%20a%20los %20mecanismos%20de%20participacio%CC%81n%20ciudadana%20en%20el%20Peru%CC%81.pdf.pdf ?v=1630448457 Shack, N., Pérez, J., & Portugal, L., (2020). Cálculo del tamaño de la corrupción y la inconducta funcional en el Perú: una aproximación exploratoria. [Calculation of the size of corruption and functional misconduct in Peru: an exploratory approach.] Documento de política en control gubernamental. Contraloría General de la República. Lima, Perú. http://repositorio.contraloria.gob.pe/bitstream/handle/ENC/18/Calculo_de_la_Corrupcion_en_el_Pe ru2020.pdf?sequence=1&isAllowed=y USAID (2020). Transparent Public Investment Project. Monitoring, Evaluation, and Learning Plan. V.3. August 3, 2020. USAID (2020). Transparent Public Investment Project. Quarterly report, Q1. April 30, 2020. USAID (2020). Transparent Public Investment Project. Quarterly report, Q2. Year1. July 30, 2020. USAID (2020). Transparent Public Investment Project. Quarterly report, Q3. Year 1. December 18, 2020. USAID (2021). Transparent Public Investment Project. Annual Performance Report, 2020. March 10, 2021. USAID (2021). Transparent Public Investment Project. Quarterly report, Q1. Jan-March. May 21, 2021 USAID (2021). Transparent Public Investment Project. Reporte Trimestral, Q2. Año 2. August 20, 2021 USAID (2021). Proyecto Inversión Pública Transparente. Informe Anual de Desempeño 2021. [Transparent Public Investment Project. Annual Performance Report 2021] February 4, 2021. USAID (2021). Civil Society Engagement Assessment. August 2021. https://pdf.usaid.gov/pdf_docs/PA00Z663.pdf USAID (2022). Proyecto Inversión Pública Transparente. Proyectos de inversión Pública priorizados con contrapartes del Perú. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 58 ANNEX VIII. LIST OF KEY INFORMANT INTERVIEWS AND FOCUS GROUP DISCUSSIONS Number Name Title, Organization Orga￾nization Region Date DD/MM/YY KII/FGD 1 Dania Tesén Timaná Secretaria Técnica, Oficina Anticorrupción Gobierno Regional PS Piura 21/06/22 KII 2 Diandra Córdova García Coordinadora, Oficina Anticorrupción, Gobierno Regional PS Piura 21/06/22 KII 3 Berenice Adriano Olaya Secretaria Técnica, Red Mesa de Concertación para la Lucha contra la Pobreza (MCLCP) Pub-CSO Piura 20/06/22 KII 4 Víctor Palacios Córdova Coordinador Regional, Mesa de Concertación para la Lucha contra la Pobreza (MCLCP) Pub-CSO Piura 20/06/22 KII/FDG 5 Silvia Flores Rosillo Lideresa, A.H. Los Algarrobos CSO Piura 24/06/22 FGD 6 Cynthia Ipanaqué Lozada Vocera, Red Interquorum CSO Piura 22/06/22 KII Remote FDG 7 Milagros Puelles Huama Representante, Red Interquorum CSO Piura 24/06/22 FGD 8 Joan Ramírez Merino Gerente Regional, Contraloría General de la República (CGR) PS Piura 27/06/22 KII 9 Flor de Maria Infante Ramirez Presidenta, Junta Vecinal Comunal (JUVECO) Miraflores CSO Piura 23/06/22 KII/FDG 10 Juan Manuel Reyes Castillo Decano, Colegio de Periodistas Piura Municipalidad Querecotillo CSO Piura 22/06/22 KII 11 Gelly Rivera Coordinadora Regional, Asociación Civil Transparencia IP Piura 20/06/22 KII 12 Mariela Eliana Porras Delgado Jefa, Oficina de Integridad Institucional Gobierno Regional PS Lambayeque 21/06/22 KII 13 Elías Daniel Pinglo Risco Ex-Oficial de integridad, Colegio de Abogados CSO Lambayeque 20/06/22 KII 14 Anthony Peña Flores Integrante, Asamblea de Delegados de Organizaciones de la Sociedad Civil, (ADOSCIL) CSO Lambayeque 21/06/22 KII 15 Roberto Javier Lingán Guerrero Ex Gerente Regional, Contraloría General de la República (CGR) PS Lambayeque 22/06/22 KII 16 Carlos Antonio Balarezo Mesones Secretario Técnico Red, Colegio de Ingenieros CSO Lambayeque 20/06/22 KII/FDG 17 Neil William Saucedo Acosta Miembro, Colegio de Periodistas CSO Lambayeque 28/06/22 KII 18 Wendy del Carmen Paredes Castañeda Gerente General, Mancomunidades Municipalidades del Valle La Leche PS Lambayeque 20/06/22 KII CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 59 Number Name Title, Organization Orga￾nization Region Date DD/MM/YY KII/FGD 19 Sonia Arteaga Muñoz Ex-Secretario Técnico, Red Mesa de Concertación para la Lucha contra la Pobreza (MCLCP) Pub-CSO Lambayeque 23/06/22 KII 20 Pina Teresa Mundaca Estevés Coordinadora Comité de al Acuerdo de Gobernabilidad Vigilancia (COVAG) CSO Lambayeque 21/06/22 KII/FDG 21 Yulliana Nataly Tineo Santisteban Sub-Coordinadora, Centro de Formación y Promoción Humana Santa Ángela (Cefoproh) CSO Lambayeque 21/06/22 KII/FDG 22 Alex Juárez López Integrante, Asamblea de Delegados de Organizaciones de la Sociedad Civil de Lambayeque (ADOSCIL) CSO Lambayeque 24/06/22 FDG 23 José Carlos Sánchez Coordinador Regional, Asociación Civil Transparencia IP Lambayeque 19/06/22 KII 24 Néstor Roque Solís Asesor, Presidencia del Consejo Regional, Gobierno Regional PS Lima Provincia 8/06/22 KII/FDG 25 Abner Renojo Castillon Secretario General, Consejo Regional de la Juventud (COREJU) CSO Lima Provincia 9/06/22 KII Remote 26 José Arturo Pichilingue Velásquez Representante, Cámara de Comercio y Producción de la Provincia de Huaura Priv Lima Provincia 10/06/22 FDG 27 Nelson Janco Capajaña Secretario Técnico Red, Mesa de Concertación para la Lucha contra la Pobreza (MCLCP) Pub-CSO Lima Provincia 8/06/22 KII 28 Liliana Ruth Vega Guillen Integrante, Acción Corporativa Inspira up-Acción Juvenil Región Lima CSO Lima Provincia 10/06/22 FDG 29 Juan Arturo Mauricio Alor Presidente, Colegio de Ingenieros del Perú - Consejo Departamental de Lima Comité Local Huaura - Oyón CSO Lima Provincia 9/06/22 KII Remote 30 Eva Eustaquio Guardia Presidenta, Cámara de Comercio y Producción de la Provincia de Huaura Priv Lima Provincia 9/06/22 KII 31 Rocio Del Pilar Álvarez Garcia Coordinadora, Ministerio de Desarrollo Social y Inclusión (MIDIS) PS Lima Provincia 10/06/22 FDG 32 Jesús Ojeda Asesor, Gobierno Regional de Lima PS Lima Provincia 8/06/22 KII 33 Carmen Rosa Arevalo Salas Secretaria Técnica, Mesa de Concertación para la Lucha contra la Pobreza (MCLCP) Pub-CSO Loreto 14/06/22 KII/FDG 34 Debora Linares Promotora Regional, Mesa de Concertación para la Lucha contra la Pobreza (MCLCP) Pub-CSO Loreto 17/06/22 FDG 35 Alejandro Domínguez Verde Integrante Mesa de Concertación para la Lucha contra la Pobreza (MCLCP) Pub-CSO Loreto 17/06/22 FDG CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 60 Number Name Title, Organization Orga￾nization Region Date DD/MM/YY KII/FGD 36 Hillary Nohelia Cardama Freitas Representante, Red Regional Anticorrupción CSO Loreto 15/06/22 KII 37 Julian Soplin Comisionado de la Oficina Defensorial de Loreto, Defensoría del Pueblo PS Loreto 15/06/22 KII 38 Katia Nuria Rios Culquicondor Fiscal Provincial de la Segunda Fiscalía de Prevención del Delito de Loreto, Ministerio Público PS Loreto 14/06/22 KII 39 Jannina Del Águila Gallardo Co Secretaria Técnica, Coordinadora Regional del Consejo Directivo de Foro Salud CSO Loreto 14/06/22 KII/FDG 40 Walter Cambero Alva Ex-Decano, Colegio de Abogados CSO Loreto 17/06/22 KII 41 Arq. Maria Julia Guevara Rios Auditora, Contraloría General de la República PS Loreto 14/06/22 KII 42 María Adolfina Guevara Gomez Coordinadora, Asociación Civil Transparencia CSO Loreto 13/06/22 KII/FDG 43 José Díaz Coordinador TPI, Proética IP Loreto 13/06/22 KII 44 Smith Flores Vásquez Gerente Regional de Control de Cusco, Contraloría General de la República PS Cusco 16/06/22 KII 45 César José Cabrera Álvarez Sub-Secretario Técnico, Red Regional Anticorrupción CSO Cusco 16/06/22 KII/FDG 46 Amparo Vargas Flores Coordinadora Regional, Mesa de Concertación para la Lucha contra la Pobreza (MCLCP) Pub-CSO Cusco 16/06/22 KII 47 Isaias Barreto Oficina de Integridad Pública e Interventor-Preventor, Gobierno Regional PS Cusco 15/06/22 KII 48 Ana Cecilia Lucana Coordinadora, Red Regional Anticorrupción CSO Cusco 17/06/22 KII/FDG 49 Luz Sandoval Integrante, Red Regional Anticorrupción CSO Cusco 17/06/22 FGD 50 Lohuana Marycielo Antonio Jalixto Representante, Red Interquorum CSO Cusco 15/06/22 KII 51 Dagner Chicata Céspedes Representante, Red Interquorum CSO Cusco 17/06/22 KII/ FGD 52 Heiymmer Portella Coordinador Regional TPI, Proética IP Cusco 13/06/22 KII 53 Diana Chávez Directora de Proyectos, Asociación Civil Transparencia IP Lima ciudad 7/06/22 KII 54 Carlos Arroyo Director del Programa de Auditorías Ciudadana y Gobierno Abierto, Proética IP Lima ciudad 7/06/22 KII 55 Mayra Pérez Márquez Especialista de Programa Auditorias Ciudadanas, Proética IP Lima ciudad 7/06/22 KII CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 61 Number Name Title, Organization Orga￾nization Region Date DD/MM/YY KII/FGD 56 Fernando Hurtado Regalado Subsecretario de Monitoreo, Asesor, Secretaría de Integridad Pública (SIP), PCM PS Lima ciudad 28/06/22 KII Remote 57 Maria Cecilia Chil Subdirectora de Procesamiento de Riesgos, Organismo Supervisor de las Contrataciones del Estado (OSCE) PS Lima ciudad 27/06/22 KII Remote 58 Patricia Guillen Gerente de Control Social y Denuncias, Contraloría General de la República PS Lima ciudad 27/06/22 KII Remote 59 Paola Barragán COP/Directora, Proyecto Inversión Pública Transparente (TPI) IP Lima ciudad 7/06/22 KII 60 Marco Aspilcueta MEL Specialist, Proyecto Inversión Pública Transparente (TPI) IP Lima ciudad 6/06/22 KII 61 Carlo Mario Velarde CSO Oversight Specialist, Proyecto Inversión Pública Transparente (TPI) IP Lima ciudad 23/06/22 KII Remote 62 Rosa Urbina Coordinadora Regional, ex Directora Adjunta, Proyecto Inversión Pública Transparente (TPI) IP Lima ciudad 6/06/22 KII 63 Sobeida González Project Management Specialist, Democracy, Human Rights, and Governance Office USAID Peru Lima ciudad 6/06/22 KII Remote 64 Miriam Choy Development A Specialist USAID Peru Lima ciudad 22/06/22 KII Remote Note: • PS (Public Sector Agencies): 16 KII • CSOs: 20 KII • Pub/CSO (Mixed Public Sector-Civil Society Organizations, MCLCP): 6 KII • IP and USAID Peru: 13 KII CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 62 ANNEX IX. DATA COLLECTION INSTRUMENTS INTRODUCTION FOR INTERVIEWERS 1. Before you administer the instrument, make sure you have available the information of the IN in each region (such as name, membership structure, and date of creation), the information of the prioritized projects, and the ISs as well. 2. It is highly recommended that you review those materials in advance as you will need to recall or mention them during the KII or FGD. Each IN has a different name, which you will have to use correctly. 3. Also, be prepared to read the consent before the interviews and FGD. The instruments are: 1. CONSENT – KII 2. CONSENT – FGD 3. IN-DEPTH INTERVIEW TO BE ADMINISTERED TO USAID/PERU 4. IN-DEPTH INTERVIEW TO BE ADMINISTERED TO TPI COP AND ADJUNCT COP AND PARTNERS’ COORDINATORS 5. IN-DEPTH INTERVIEW TO BE ADMINISTERED TO TPI MONITORING AND LEARNING SPECIALIST 6. IN-DEPTH INTERVIEW TO BE ADMINISTERED TO STAFF FROM THE COMPTROLLER’S OFFICE AND INTEGRITY SECRETARIAT 7. IN-DEPTH INTERVIEW TO BE ADMINISTERED TO TPI REGIONAL COORDINATOR 8. IN-DEPTH INTERVIEW TO BE ADMINISTERED TO MEMBERS OF THE IN 9. IN-DEPTH INTERVIEW TO BE ADMINISTERED TO REGIONAL GOVERNMENT STAFF CONNECTED TO THE CONTROL UNIT 10. IN-DEPTH INTERVIEW TO BE ADMINISTERED TO RESEARCH JOURNALISTS 11. FOCUS GROUP GUIDE TO BE ADMINISTERED TO IN MEMBERS 12. GUIDE FOR EMBEDDED RESEARCHERS 13. SURVEY TO BE ADMINISTERED TO IN MEMBERS CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 63 CONSENT – KII Name: Date: IN (Code): Location: Interviewer: Interviewer, please read the following script to start the interview: Hello and thank you for agreeing to speak with us. My name is [researcher] and this is my colleague [researcher]. We are working with NORC at the University of Chicago on a study about the INs supported by the Transparent Public Investment Project (TPI) of USAID. The purpose of the study is to analyze the work of the network, the way the work is organized and how the IN teams apply the ISs looking for transparency and accountability of the regional government. In the context of this study, we would like to interview you about this project. This conversation will last between 45 and 60 minutes. Your participation in this interview is voluntary and you can choose not to answer a question and skip it. Moreover, you can stop the interview at any moment without providing a reason. Your perspective is very important to help USAID improve its programs. The information we will be collecting through this interview will be kept safe by our team. Your responses will be kept anonymous and not linked to your name – each person interviewed will be given a unique identification number. Your identity will be kept confidential and will not be shared outside of the ET. Other information that could identify you (e.g. title, district, etc.) will be excluded from the report and other documents produced by our team and shared with USAID and other institutions. If you have any questions, you may ask them now or later, even after the interview has started. If you wish to ask questions later, you may contact Ingrid Rojas Arellano at RojasArellano-Ingrid@norc.org. This proposal has been reviewed and approved by NORC’s Institutional Review Board, which is a committee whose task is to assure human subjects' protections and minimize respondent burden. Do you agree to participate in this interview today? Yes No Do you agree to have this conversation recorded today? Yes No CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 64 CONSENT – FGD Name: Date: Location: Moderator: Moderator, please read the following script to start the FGD: Hello and thank you for agreeing to speak with us. My name is [researcher] and this is my colleague, [researcher]. We are working with NORC at the University of Chicago on a study about the INs supported by the Transparent Public Investment Project (TPI) of USAID. The purpose of the study is to analyze the work of the network, the way the work is organized and how the IN teams apply the ISs looking for transparency and accountability of the regional government. In the context of this study, we would like to talk to you about this project. This FGD will last between 45 and 60 minutes. Your participation in this FGD is voluntary and you can choose not to answer a question and skip it. Moreover, you can stop your participation at any moment without providing a reason. Your perspective is very important to help USAID improve its programs. The information we will be collecting through this FGD will be kept safe by our team. Your responses will be kept anonymous and not linked to your name – each person interviewed will be given a unique identification number. Your identity will be kept confidential and will not be shared outside of the ET. Other information that could identify you (e.g., title, district, etc.) will be excluded from the report and other documents produced by our team and shared with USAID and other institutions. If you have any questions, you may ask them now or later, even after the FGD has started. If you wish to ask questions later, you may contact Ingrid Rojas Arellano at RojasArellano-Ingrid@norc.org This proposal has been reviewed and approved by NORC’s Institutional Review Board, which is a committee whose task is to assure human subjects' protections and minimize respondent burden. Do you agree to participate in this FGD today? Yes No Do you agree to have this conversation recorded today? Yes No CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 65 IN-DEPTH INTERVIEW TO BE ADMINISTERED TO USAID/PERU Introduction and consent will be inserted; likewise, the information of each IN, ISs and prioritized projects will be handed out to the interviewers. CONTEXT 1. In your opinion, what are the differences of the IN approach in comparison to other modalities of citizen’s oversight mechanisms supported by USAID, like Veedurias Ciudadana of Reconstrucción con Cambios? DEVELOPMENT PROCESS, MEMBERSHIP, AND OPERATIONS 2. Based on your experience, what factors or conditions have supported the INs’ achievements to date? • Probe: How about the internal structure with different stakeholders (public, private, and CSO)? • Probe: How about the support they received from the regional and central government? • Probe: How about the tools and training the members have received from TPI? • Probe: How about the type of projects the network is monitoring? • Probe: Have some IN been more successful? And in what areas? 3. Based on your experience, what factors have limited the INs’ development and work? • Probe: How about the structure and membership? • Probe: The citizens and public officials’ commitment and the time needed to invest in the network? • Probe: The complexities of the procurement process and the methodology of the ISs? • Probe: The training and support in the field? • Probe: Based on what you have observed, what network elements would you like to adjust? 4. Do you think the response of the regional governments has been positive? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 66 IN-DEPTH INTERVIEW TO BE ADMINISTERED TO TPI COP AND ADJUNCT COP, AND PARTNERS’ COORDINATORS (TRANSPARENCIA AND PROÉTICA) Introduction and consent will be inserted; likewise, the information of each IN, ISs and prioritized projects will be handed out to the interviewers. PURPOSE 1. In your view, what is the purpose of the networks? • Probe: Would you adjust the purpose in the future? MEMBERSHIP 2. In your opinion, has the structure intended for the networks been accomplished in the regions? • Probe: [For COP and Adjunt COP] Considering that the IN Cusco has a membership structure with a predominant public sector membership while Piura, Lambayeque and Lima have the CSO as a predominant member, in your opinion how have this structure affected the IN goals? • Probe: [For Proética] Considering that the IN Cusco has a membership structure with a predominant public sector membership while Loreto have the CSO as a predominant member, in your opinion, how have this structure affected the IN goals? • Probe: [For Transparencia] Considering that both the IN in Piura and Lambayeque have membership structure with a CSO as a predominant member, in your opinion, how has this structure affected the IN goals? RESOURCES AND OPERATIONS 3. In your opinion, have the networks been successful or partially successful in developing and implementing the action plans? • Probe: What differences do you see in the networks? Which ones have been more successful? • What factors or conditions do you think helped those networks to be more successful? 4. Considering your experience with the INs, have the networks been successful or partially successful to encourage the regional government to comply with the ISs? Specifically for the selected projects? • Probe: What differences do you see in the networks? Which ones have been more successful? • What factors or conditions do you think helped those networks to be more successful? CONTEXT AND DEVELOPMENT PROCESS 5. In your opinion, has the IN approach been considered as a possible model of collaboration by the central government agencies specializing in government procurement regulation, control and auditing in the region? [referring to the decentralized offices of CGR, SIP, and OSCE] • Probe: Have the coordination with CGR and the practice and content of the Control Simultáneo/Concurrente helped the network goals? How? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 67 RESOURCES AND OPERATIONS 6. Considering your experience with the network, have the members been able to use the tools provided by the central government agencies specializing in government procurement regulation, control and auditing? [referring to CGR, SIP, OSCE] like • Probe: How about the open data mechanism and the complaint portal/window? • Probe: How about the use and monitoring of the recommendations that the CGR prepared for the regional governments during the Control Simultáneo /Concurrente visits? • How about the monitoring of the improvement of the Preventive Capacity Index of regional governments? • Probe: Has the project been able to use central government tools and transfer the knowledge to the networks? How about in the training and coaching? • Are there other tools the project is currently working on? DEVELOPMENT PROCESS 7. Are there examples you can share with us of successful experiences of the IN? • Probe: What factors do you think helped that success story? 8. Based on your experience, what are the factors that limit the networks’ success? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 68 IN-DEPTH INTERVIEW TO BE ADMINISTERED TO TPI MONITORING AND LEARNING SPECIALIST, AND R1, R2, AND R3 SPECIALISTS Introduction and consent will be inserted; likewise, the information of each IN, ISs and prioritized projects will be handed out to the interviewers. DEVELOPMENT PROCESS 1. [For the MEL Specialist only] We would like to discuss the factors and conditions that explain the progress or delays in some performance indicators to date. Let’s begin with the “Number of selected public investment projects that exceed the compliance practices of the public procurement ISs.” ● Probe: Are there differences among INs? ● Probe: Do you expect that the indicator’s level [mention indicators which are behind expected value] will change? What would be the actions that the project needs to make to improve such indicators? [ Continue working covering the rest of the indicators listed below one by one] ● Number of joint public-private action plans implemented among IN actors to oversee prioritized investment projects. ● Number of public ISs applied to public investment projects and promoted by INs ● Number of women participating in INs and promoting anti-corruption actions. 2. We want to know about the IN’s contributions towards the TPI project’s broader goal. Based on your knowledge, what are the factors or conditions that explain the progress or the delay of the activities tracked by the following indicators: First, let’s talk about the indicator “1.3. Public investment processes, tools, and/or protocols that reduce corrupt practices become the government of Peru rules that are widely disseminated and adopted by Peru’s national and local public entities”? ● Probe: Are there differences among the INs? ● Probe: Do you expect that the indicator [mention indicators which are behind expected value] will change? What would be the actions that the project needs to make to improve such indicators? [ Continue working covering the rest of the indicators listed below one by one] ● 1.6. Emblematic public investments follow the procurement standards and identify lessons learned and best practices to be replicated in other similar procurement processes. ● 2.1. An IN composed of key government of Peru actors, CSOs, and business associations operational with clear plans to oversee and improve public investment integrity at all stages. ● 2.2. National research journalist network (infomediaries) trained to investigate and monitor emblematic public investment projects, report risks of corruption, and disseminate best practices. ● 2.3. INs that are aware of and use access to information mechanisms and best practices through an open government lens. ● 3.1. At least two major business associations (CONFIEP, CAPECO, and target region chamber of commerce) adopt and implement integrity compliance standards and publicize periodic reports, including on performance measurement and success stories. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 69 ● 3.2. Voluntary codes of ethics and other integrity initiatives are designed and adopted in prioritized sectors and regions. ● 3.3. The anti-bribery model developed by Peruvian business associations for big companies is adapted to medium and small businesses, particularly in prioritized sectors and regions. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 70 IN-DEPTH INTERVIEW TO BE ADMINISTERED TO STAFF FROM THE COMPTROLLER’S OFFICE, PUBLIC PROCUREMENT REGULATORY AGENCY AND INTEGRITY SECRETARIAT Introduction and consent will be inserted; likewise, the information of each IN, ISs and prioritized projects will be handed out to the interviewers. CONTEXT 1. [For the Comptroller’s Office] In your opinion, and based on your experience, what are the differences between the work of the INs supported by the TPI and the Monitores Ciudadanos supported by your organization? ● Probe: Do you think they can complement each other, or they duplicate the work? How? ● RESOURCES AND OPERATIONS 2. [For the Comptroller’s Office and OSCE] In your opinion, do the application of the ISs to the Public Investment Projects and the rules and regulations set and administered from your organization complement each other? ● Probe: Do you think the methodology to apply the ISs to the public investment projects could be duplicating efforts? ● Probe: Have you been able to share your work with the network to improve their understanding of the matter and advocacy for accountability? 3. Based on your experience and information available to you, do the INs’ work support your decentralized work… ● [For the Comptroller’s Office] … when they oversee the implementation of the Control Simultáneo/Concurrente recommendations? [Elaborate about the cases where CGR is part of the network.] ● [For OSCE[ … when they oversee the plan to reduce the procurement risks? ● [For SIP] … when they oversee the implementation of the plan to improve the Preventive Capacity Index? DEVELOPMENT PROCESS 4. Based on your knowledge, have some INs been more successful in supporting your work? Why do you think that? 5. In your opinion, what elements of the IN approach could be improved? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 71 IN-DEPTH INTERVIEW TO BE ADMINISTERED TO TPI REGIONAL COORDINATORS Introduction and consent will be inserted; likewise, the information of each IN, ISs and prioritized projects will be handed out to the interviewers. PURPOSE 1. In your view, what is the purpose or goal of the network [ name by region]? ● Probe: Is that purpose shared by all members? LEADERSHIP AND GOVERNANCE 2. In your view, how good or bad are the interactions among the members of the network? ● Probe: How are decisions made? Can you give an example? ● Probe: Is it a good space or environment for communication? How do you maintain contact? Are all members engaged? OPERATIONS 3. Tell me how you divide or organize the work among the network’s members; for example, discuss how you prepared and implemented the action plan or made a collective announcement? ● Probe: Are roles defined besides that of the Technical Secretariat? What is your role in the Network? ● Probe: Given that there are diverse groups in the network, are there individuals or organizations better at certain tasks than others inside the network? ● Probe: How do the network members monitor or apply the methodology of ISs to check the project’s compliance? Are there differences between public servants and CSO? 4. Can you describe how the training and coaching to the integrity members are implemented? [Try to get the number of days/hours] [Check if there is training material available] ● Probe: After the training, are all members capable of applying the ISs to the public investment projects? ● Probe: Do all members know how to use the open data from public agencies? Do you remember any circumstances or examples? ● Probe: Were all members able to use the complaint model portal prepared by SIP? Have you or other members used that option? Can you give me an example? ● Probe: How about the monitoring of the plan to reduce procurement risks following OSCE guidelines? Or the improvement of the Preventive Capacity Index following SIP guidelines? Or monitoring the implementation of the CGR recommendations after Control Simultáneo/ Concurrente Audit? DEVELOPMENT PROCESS, MEMBERSHIP 5. In your opinion, what are the achievements the network has reached so far? ● Probe: Can you give an example? 6. In your opinion, what factors or elements support the network’s accomplishments or are crucial to its success? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 72 ● Probe: How about membership commitment? How many hours do members contribute to the network? ● Probe: How about the structure of the membership, having a mixture of public, private and CSO representatives? [Except Cusco, where the public sector is predominant:] Probe: How about the structure of the current membership? ● Probe: What are the benefits of having a Technical Secretariat that belongs to a collaborative network with a mixed membership and is led by a CSO like the MCLCP? [For Cusco only] What are the benefits of having a technical secretariat that belongs to the public agency in charge of the control system? [we are referring to the regional office of the CGR] ● Probe: How about having a catalyst organization like USAID and the TPI project? CONTEXT 7. How is the relationship of the network with other agencies and institutions linked to its purpose, for example, transparency in the use of public funds? ● Probe: What are the benefits of having a member of the CGR regional office? ● [For Lima region, where there is no member from the CGR] Have you received information or training on the CGR’s Control Simultáneo/Concurrente audit of the projects you monitor? ● Probe: Do members have interactions with the media or have received support from them? 8. What is your opinion about the external elements that may affect positively the IN’s work? What would be the factors or issues that limit the network’s work? ● Probe: What do you feel about the political environment? Public sector rules and regulations? DEVELOPMENT PROCESS 9. Considering your experience with the network until now, what elements would you like to improve? And how? ● Probe: How about training and coaching? [the methodology, content, and duration] ● Probe: How about the structure of the network? Would you consider adding more or different members? ● Probe: How about the way the members collaborate to do the work, for example producing reports and tools for advocacy? ● Probe: How about the logistic support and resources? ● Probe: How about other topics that we have not mentioned? Do you want to share any information you consider important about your network’s work? 10. Before working for [differentiate between Transparencia, ProEtica and TPI project] have you worked in areas related to the INs, like civil society organizations that oversee the public sector or public sector agencies of control and regulation? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 73 IN-DEPTH INTERVIEW TO BE ADMINISTERED TO MEMBERS OF THE IN Introduction and consent will be inserted; likewise, the information of each IN, ISs and prioritized projects will be handed out to the interviewers. MEMBERSHIP 1. To begin the interview, please tell us what organization are you representing in the network?, or Are you a citizen or individual member? ● Probe: Are you currently, or in the past, a member of another network, or association or alliance? How about the organization you are representing in the network? ● Probe: [For all CSO members] Are you familiar with Monitores Ciudadanos and/or Veedurias Ciudadanas? Yes/No [If yes, continue to the next questions] ● For Piura and Lambayeque, Lima: [For CSO only] Have you participated in the Veedurias Ciudadanas monitoring Reconstrucción con Cambios program? ● [For Loreto] Have you participated in the veeduria ciudadana “Mirada Ciudadana”? ● [For CSO only in all INs] Have you participated in Monitores Ciudadanos? PURPOSE 2. In your view, what is the purpose or goal of the network [ name by region]? ● Probe: What do you want to accomplish as a member of the network? Do you think other members might have a different perspective or goal? 3. Do you think the IN can influence the regional government in matters of integrity and anticorruption policies and practices? LEADERSHIP AND GOVERNANCE 4. We would like to know Who are the members of the network? How are the relationships and communications among network members? ● Probe: How are decisions made? Can you give an example? In your view, do you think this can be improved? ● Probe: Have they listened to your ideas and opinions or questions and have you received feedback from them? In your view, do you think this can be improved? ● Probe: Is it a good space or environment for communication? How do you maintain contact? Are all members engaged? OPERATIONS 5. Can you tell me how you organize or divide the work among the network’s members? For example, please tell me how you prepared and implemented the action plan or made a collective announcement ● Probe: Are roles defined besides that of the Technical Secretariat? ● Probe: How do the network members monitor and/or apply the methodology of ISs to check the project’s compliance? Do you think the concept of the 15 ISs is friendly and easy to understand? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 74 ● Probe: Are you familiar with the Comptroller’s Office’s recommendations to the regional government when implementing the Control Simultáneo/Concurrente? Yes/ No [If yes] Do members of the network monitor the recommendations? No/Yes [If yes] How? ● Probe: Are you familiar with the results of the Preventive Capacity Index of the regional government following SIP guidelines? Yes/ No [If yes] Do members of the network monitor the regional government’s actions to improve the index? No/Yes [If yes] How? ● Probe: Are you familiar or heard about the regional government’s plan to reduce the procurement risks following OSCE guidelines? Yes/ No [If yes] Do members of the network monitor the implementation of the plan? No/Yes [If yes] How? ● Probe: There are diverse groups in the network; Are there some individuals or organizations that do certain tasks inside the network better than other individuals or organizations? ● How many times did the IN meet in the last 12 months? How many hours on average? What were the topics of the discussions or meetings and what do you think of them? 6. In your view, did the training and coaching to the network reach their goal? ● Probe: After the training, are all members capable of applying or overseeing the application of the ISs to the public investment projects? ● Probe: Are you familiar with the use of open data from public offices? Yes/No [If yes] Have you or other members of your network looked for data using the open data portal? Can you give me an example? ● Probe: Are you familiar with the use of the complaint model portal implemented by SIP or the Comptroller’s Office? Yes/No [If yes] Have you or other members used it? Can you give me an example? DEVELOPMENT PROCESS 7. In your opinion, what are the achievements the network has reached so far? ● Probe: Can you give an example? ● Probe: Do you have any experience interacting with the government to implement the ISs to their projects? [For regional government officers] Or direct experience in the government implementing the ISs? MEMBERSHIP 8. In your opinion, besides the factors already mentioned, what other factors or elements support the network’s accomplishments or are crucial to its success? ● Probe: How about membership commitment? How many hours do members contribute to the network? ● Probe: How about the structure of the membership, having a mixture of public, private and CSO representatives? ● [Except Cusco, where the public sector is predominant] Probe: How about the structure of the current membership? ● Probe: How about having a Technical Secretariat that belongs to a collaborative network like the MCLCP? [For Cusco only] What are the benefits of having a technical secretariat that belongs to the public agency in charge of the public sector audits and control [we are referring to the regional office of the CGR] and also as part of the Regional Anticorruption Commission? ● Probe: Are you familiar with or do you know that USAID and the TPI project are supporting the IN? What do you think about their support? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 75 CONTEXT 9. What is your point of view regarding the relationship of the network with other relevant agencies and institutions like the CGR, the media, and the regional government? ● Probe: What was the role of the member representing the CGR regional office in the network? Have you received information or training on the CGR’s Control Simultáneo/Concurrente audit of the projects you monitor? ● [For Lima region, where there is no member from the CGR] Have you received information or training on the CGR’s Control Simultáneo/Concurrente audit of the projects you monitor? ● Probe: Do you communicate with the media or have received support from them? ● Do you expect to use the IS methodology or knowledge you acquired in the network about public investment and procurement in other activities your work on? ● Probe: Can you give me examples? [personally, or institutionally] [If the member is part of the public sector, he/she should refer to his office work mostly, while the CSO member should refer to his/her organization and dissemination activities to other networks] DEVELOPMENT PROCESS 10. Besides the elements or factors already mentioned, what other external elements limit your work in the IN? What would be the factors or issues that limit the network’s work and impact in the fight against corruption? ● Probe: What do you feel about the political environment? ● Probe: Is the regional government available or open to hold communications with the network? ● Probe: Do you know if or think that the rules and practices of public procurement are well￾established, or do they allow for further input? 11. Considering your experience with the network until now, besides the elements already mentioned, what other elements or factors facilitate the network’s accomplishments? ● Probe: How about training and coaching? [the methodology, content, and duration] ● Probe: How about the structure of the network? Would you consider adding more or different members? ● Probe: How about the way the members collaborate to do the work, for example producing reports and tools for advocacy? ● Probe: How about the logistic support and resources? ● Probe: How about other topics that we have not mentioned? Do you want to share any information you consider important about your network’s work? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 76 IN-DEPTH INTERVIEW TO BE ADMINISTERED TO REGIONAL GOVERNMENT STAFF CONNECTED TO THE CONTROL UNIT, THE INVESTMENT OFFICE, AND INTEGRITY OFFICE (OR OFFICER) Review ahead which IN have this regional post as a member, and which networks do not. Introduction and consent will be inserted; likewise, the information of each IN, ISs and prioritized projects will be handed out to the interviewers. CONTEXT 1. In your opinion, does the work of the IN complement or support your work responsibilities? ● Probe: Can you give examples? ● Probe: Have you had access and reviewed their action plans or recommendations? What is your opinion about those reports? 2. Based on your experience, do the ISs and the regulations of the CGR and the OSCE complement each other? ● Probe: What is the value-added after applying the ISs? ● Probe: Are the ISs easy to follow and apply to the projects? ● Probe: Have you had the opportunity to apply the standards yourself? ● Probe: Have you applied the ISs or plan to use them in the future? In other projects? Yes/No Why? 3. What are the transparency initiatives implemented by the regional government? Do they apply to the management of public works? Are any of these actions done because of the network’s collaborative engagement with the government? DEVELOPMENT PROCESS 4. From your perspective, what measures or changes would you recommend to improve the network’s work and enhance their impact on the regional government’s transparency? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 77 IN-DEPTH INTERVIEW TO BE ADMINISTERED TO RESEARCH JOURNALISTS Introduction and consent will be inserted; likewise, the information of each IN, ISs and prioritized projects will be handed out to the interviewers. CONTEXT 1. Based on your experience, have you advanced your investigative pieces after TPI training or coaching? ● Probe: We would like to know about some investigative cases you conducted about public accountability. Can you give us an example? ● Probe: Can you elaborate on the news report you published about government transparency? 2. How relevant are TPI’s resources for your work these days? And in the future? 3. What are the limitations you encounter in your work when dealing with government transparency? ● Probe: Has the TPI project helped reduce those limitations? ● Probe: Are editorials and outlets owners interested in the topic of transparency and corruption? 4. Do you think you are going to continue working in this type of research? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 78 FOCUS GROUP GUIDE TO BE ADMINISTERED TO IN MEMBERS Introduction and consent will be inserted; likewise, the information of each IN, ISs and prioritized projects will be handed out to the interviewers. MEMBERSHIP 1. What do you think about the structure of the network? ● Probe: Do you think you would work better divided by groups [type of organization] or as you are organized now? ● Probe: Does this membership structure help you reach your goals? ● Probe: Who do you think another organization is missing and should be part of the network to make you stronger? 2. Is belonging to another organization that works collaboratively helpful to navigate this network and be more successful in your advocacy and dialogue with the government? ● Probe: Do you think you can have more impact on government accountability? ● [For Cusco] Do you think that having this network with such important control and justice agencies help to support government accountability? OPERATIONS 3. How do you feel about the IN’s goal of improving government accountability and transparency through the application of ISs? ● Probe: Have all of you been involved in the monitoring or application of the methodology or do you think some members are more equipped to do this job and others have other skills like advocacy and communications? ● Probe: Has it been a complex matter? ● Probe: Did you run into trouble while trying to apply the standards to the selected projects? ● Probe: Do you think there are other ways to influence the government and keep it accountable? DEVELOPMENT PROCESS 4. In your opinion, what are the factors blocking or limiting the IN’s work? 5. In your opinion, what are the factors that promote the IN’s achievements and what would you recommend doing to improve the efficacy of the network? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 79 GUIDE FOR EMBEDDED RESEARCHERS Introduction and consent will be inserted; likewise, the information of each IN, ISs and prioritized projects will be handed out to the interviewers. BACKGROUND INFORMATION IN’s objective. TPI’s IN approach aims to find novel ways for actors to collaborate in order to improve public investment processes. Specifically, they seek to promote greater transparency and integrity in the execution of public contracting and public investment processes of target public works. Its goals are to formalize actors’ commitments, incentivize their collaboration, and increase accountability through action plans focused on prioritized public investment projects. For that purpose, the TPI project has designed a list of ISs that will be used as benchmarks for public procurement and investment projects. Networks will use that reference when preparing their action plans focusing on two prioritized projects. Membership structure. The INs were structured to have public, private and CSO representatives collaborating on providing oversight to regional governments and ensuring they complied with ISs. Some commonalities could help structure the cases. Piura, Lambayeque and Lima Provincias are regions that belong to the Reconstrucción con Cambios Program, which called the attention of the CGR and CSO to monitor the public investments since 2017-2018. These networks’ structure is predominantly composed of CSO membership. On the other end is Cusco, whose network is predominantly composed of public agencies connected to government control and justice. No less important is Loreto, with balanced participation of public and CSO membership. Support. The TPI project supports Lima’s IN directly. The other networks are supported by one of the two NGOs67 specialized in promoting democratic values, citizen’s oversight and good governance: Asociación Civil Transparencia, and Proética (Transparency International’s Peruvian Chapter). Except for Cusco, all INs have a Technical Secretariat who belongs to the Mesa de Concertation de Lucha contra la Pobreza (MCLCP), a network that also participated in the oversight of Reconstruction con Cambios together with the CGR. Other initiatives from the public sector. Paralleling the TPI work, the Comptroller’s Office has been auditing the selected projects with the control simultáneo modality, It is expected that the information from the audits is considered in the discussions and training of the network, improving their understanding of public procurement rules and regulations and the fight against corruption. Likewise, the Integrity Secretariat from the Prime Minister’s Office has developed a guide to use the open data from public agencies, which is expected to enrich the network’s knowledge. PROTOCOL It is crucial to obtain available documentation ahead of the meeting if possible. This includes the documentation that might be utilized by the members in the meeting you are attending and reports or minutes from previous meetings. 67 TPI sub-contractors. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 80 At the beginning of the meetings and interviews, the embedded researchers should introduce themselves as researchers documenting case studies of the INs. QUESTIONS FOR NON-PARTICIPANT OBSERVATION MEMBERSHIP AND LEADERSHIP 1. How are decisions made in the network? Are the decision-making and discussions horizontal? Do CSOs participate as actively as public organizations? Observe if the environment is inclusive and if all members, including women, participate. 2. Are all members welcome to participate? Do you observe that information is only flowing in one direction, or do you observe a conversation where there is feedback to comments and opinions? 3. How do you see the role of the Technical Secretariat? Is the MCLCP bringing its experience supporting coordination to the network? Is there another member that supports coordination? 4. Who is attending the meetings? Who is absent? DEVELOPMENT PROCESS AND OPERATIONS 5. Has the network evolved from training and strengthening its members to a phase of collaboration, and it is translated to products? [This refers to the production of analysis, announcements, action plans] 6. Is the network able to discuss the application of the ISs to the prioritized projects? 7. Do you observe differences in the knowledge of the matter? 8. Are all members engaged in the work and production, independent of the role they have adopted or assigned? Some could be peer reviewers, researchers, writers, or civil works observers. 9. Do you observe that the discussion transcends the tools like the ISs, and covers a bigger picture like transparency and the fight against corruption? Are members having initiatives to propose new rules or regulations, or the application of the methodology to other projects? QUESTIONS FOR IN-DEPTH INTERVIEWS WITH TECHNICAL SECRETARIAT (ONE MEMBER FROM THE PUBLIC SECTOR AND ONE MEMBER FROM A CSO) MEMBERSHIP 1. Please tell me what institution you are representing? Or did you join the network as a citizen? PURPOSE 2. In your own words, what do you want to accomplish with the network? 3. Is that goal or purpose shared by all members? OPERATIONS 4. The network has a diverse group of representatives. Do you think some members apply the methodology of ISs in public investments better than others? DEVELOPMENT PROCESS 5. What are the main achievements of the network? Can you elaborate? [this is the main area you should focus on, as this will be an important section in your case study] ● Probe: Regarding its purpose, did the network make a difference in the advocacy for a more transparent government? [Focus on the projects following the ISs, use of open data and use of the portal of citizen’s oversight created by SIP] ● Probe: Has the network developed connections and collaboration among its members and with other institutions? ● Probe: Have the network members acquired new knowledge for future work? CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 81 6. How did you reach that goal? Did you feel that all members made it possible? 7. In your opinion, what factors support the work of the network? 8. In your opinion, what factors limit the work of the network? CLOSING ANALYSIS Based on your observation, interviews, and documents you collected; what level of maturity does the network have? 1. Members connect. The network begins to construct the membership and focus its work on strengthening the connections. 2. Members coordinate. The network is strengthening capacities (ISs and public procurement) and exchanging information. 3. Members collaborate and produce. The network works to produce the action plan and has initiatives to seek public information using open data, or other actions that support transparency and fight against corruption. Even though these three levels could be considered phases of a process, they can overlap sometimes. For example, while the team is working already in collaborative action, they could be welcoming new members and improving their connections and coordinating new projects. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 82 SURVEY TO BE ADMINISTERED TO IN MEMBERS Question Choices 1 Please provide the following information 1.1 Sex a Female b Male 1.2 DOB a Month b Year 1.3 Region of residence a Piura b Lambayeque c Lima d Loreto e Cusco 2 Are you currently a member of the IN? a Yes b No 3 Did you join the IN representing an organization or as a citizen? Please mark the one that reflects your membership a As an individual, citizen b As representative of a civil society organization c As representative of the regional government d As representative of the regional or national Comptroller Office e As representative of a governmental organization other than the regional government and regional Comptrollers’ Office , f As representative of the private sector g Other (Please specify) ____ 4 When did you join the network? a Month b Year [If Q2=Yes, option a, skip Q5] 5 When did you leave the IN? a Month b Year [If Q1=No, skip Q6, Q7, and Q8] CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 83 Question Choices 6a How many hours a month do you commit to the network, considering the meetings, reading and research, visits to civil works, analysis, and writing, among other tasks? a 4 hours or less b 5 to 9 hours c 10 to 15 hours d 16 or more hours 6b How many hours a month do/did you commit to the network, considering the meetings, reading and research, visits to civil works, analysis, and writing, among other tasks? a 4 hour or less b 5 to 9 hours c 10 to 15 hours d 16 or more hours 7a Are you the Technical Secretary or the Adjunct Technical Secretary? a Yes b No 7b Were you the Technical Secretary or the Adjunct Technical Secretary of the IN? a Yes b No 8a ABOUT DECISION MAKING When my IN needs to make a group decision and take a stand on issues regarding transparency and accountability, I find that: a The process is very democratic, and everybody has a voice b Not everybody’s opinions are considered c Other, (Please specify) _______________ d I do not know or have an opinion on the issue 8b ABOUT DECISION MAKING When my IN needed to make a group decision and took a stand on issues regarding transparency and accountability, I found that a The process was very democratic, and everybody had a voice b Not everybody’s opinions were considered CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 84 Question Choices c Other, (Please specify) _______________ d I do not know or have an opinion on the issue [Ask Q9 ONLY if Q3 = representative of the regional government” or representative of the Comptrollers’ regional Lima Office (option c and d) 9 SOBRE LA APLICACIÓN DE LOS ESTANDARES DE INTEGRIDAD When my institution, the regional government, has applied the ISs to the prioritized projects, I have found that: a All the responsible staff have been able or know how to apply the ISs b Not all the responsible staff have been able or know how to apply the ISs c I am not sure how the responsible staff have applied the ISs [Ask Q10 ONLY if Q3 = As representative of the regional government or the regional Comptrollers’ Office (option c and d) and Q9= “Not all the responsible staff have been able or know how to apply the ISs” (option b)] 10 Why do you think some of the responsible staff have been unable to apply the ISs? a The public procurement process is complex b The ISs are difficult to understand or apply c One need working experience in the public sector or public procurement d The selected projects were not appropriate to apply the ISs e Other reasons: (Please specify) [Skip Q11 AND Q12 if Q3 = “As representative of the regional government” (option c)] CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 85 Question Choices 11 SOBRE EL MONITOREO DE LOS ESTANDARES DE INTEGRIDAD When the members of my IN have monitored or overseen the application of the ISs to the prioritized projects, I have found that: a All the members have been able or know how to monitor or oversee the application of the ISs b Not all the members have been able or know how to monitor or oversee the application of the ISs c I am not sure how members monitor or oversee the application of the ISs [Skip Q12 if Q11 = option a or c] 12 Why do you think some of the members have been unable to monitor or oversee the application of the ISs? Mark all the reasons that apply a The public procurement process is complex b The ISs are difficult to understand or apply c One needs working experience in the public sector or public procurement d The selected projects were not appropriate to apply the ISs e Other reasons: (Please specify) 13 TRAINING AND COACHING As a result of TPI’s training and coaching on public investment projects’ public procurement, I believe that: a My knowledge about public procurement in public investments is about the same b My knowledge about public procurement in public investments improved c My knowledge and competences to analyze the process of public procurement in public investments improved 14 Regarding the use of the ISs tool and methodology, I think that a I will not use the ISs in my work or in my advocacy activities promoting public sector accountability b I will use the ISs in my work or advocacy activities promoting public sector accountability CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 86 Question Choices c I am not sure if I will or will not use the ISs in my work or advocacy activities promoting public sector accountability d Other (Please specify) ___ [End interview if Q3 = “As representative of the regional government" (option c)] [If Q2 = No, skip Q15 AND Q16] 15 Out of all the tools and methodology reviewed with TPI staff and trainers which tools or methodology has helped you monitor the application of the ISs? Please mark up to three options a Use of open data platforms from public agencies to find information from regional government investments and procurement b Use of the complaint portal windows set by the Integrity Secretariat and/or the Comptroller’s Office c Monitoring the improvements of the Preventive Capacity Index of the regional government d Monitoring the recommendations of the Control Simultáneo/ Concurrente reports of the Comptroller’s Office to the regional governments e Monitoring the reduction of the risks related to the regional government’s procurement processes f None of the above helped me understand the ISs g Other, (Please specify) _______________ 16 FUTURE TRAINING Out of all the tools and methodologies reviewed with TPI staff and trainers, which tools or methodologies you would like to have more training and practice with. Please mark up to three options a Use of open data platforms from public agencies to find information from regional government investments and procurement b Use of the complaint portal windows set by the Integrity Secretariat and/or the Comptroller’s Office c Monitoring the improvements of the Preventive Capacity Index of the regional government CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 87 Question Choices d Monitoring the recommendations of the Control Concurrente reports of the Comptroller’s Office about the prioritized projects e Monitoring the reduction of the risks related to the regional government procurement processes f None of the above g Other (Please specify) ___________ 17 ABOUT REGIONAL GOVERNMENT ACCOUNTABILITY My network has worked to improve the regional government integrity in public procurement and I have found that: a The regional government has not been interested on providing information or making any improvement b The regional government has provided information and has intended to improve the integrity of its procurement process c The regional government has adopted the recommendations from the red and has taken actions to improve its procurement process 18 ENABLING FACTORS What are the three most important elements that you consider have contributed to the IN success? Please mark up to three options a Membership is diverse b TPI project and USAID as catalyst organizations c Training and coaching received d Technical Secretariat with experience e Democratic and cordial dialogue among members f ISs tool and methodology is clear and feasible to apply and monitor g Good relationships with the regional government h None of the above. Nothing has contributed to the success of the IN. i Other (Please specify) _______________ CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 88 Question Choices 19 LIMITING FACTORS What are the three most important elements that have limited the IN’s success? Please mark up to three options a Not enough time to participate in meetings and activities b Members ‘different levels of experience with public procurement and investment c Complexities of the ISs methodology d Not enough training and coaching e Lack of regional government engagement with the IN f COVID -19 limited the coordination g Members have different interests h None of the above. Nothing has limited the IN’s success. l Other __________________ CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 89 ANNEX X. TEAM MEMBERS BETTY ALVARADO DE LOZAN, TEAM LEAD Ms. Betty Alvarado is a senior specialist in public finance, with more than 20 years of experience in the areas of education, health and governance. She is an expert in project cycle from early assessments, implementation, monitoring, and ex-post evaluations. She has worked on quantitative and qualitative research analysis and in the design and administration of tools for data collection: surveys, in-depth interviews, focus groups. She has executed strategies for advocacy and constructive engagement with authorities and public officials from the local, regional, and national governments, academia, nonprofit and private sectors. She possesses extensive knowledge of Peru’s public administration processes and tools (procurement and investment). While working as faculty member of Universidad del Pacífico, she evaluated an important number of infrastructure projects for the Ministry of Economy in Peru in the areas of Health, Urban Development and Housing. Ms. Alvarado has applied the Public Expenditure Tracking Survey to the Health Sector and has studied the Result Based Budget in Peru among other fiscal tools. While working in the SUMMA education project in Peru, she analyzed the procurement processes of human resources contracting and purchase of educational materials of the Ministry of Education. She was a NORC consultant supporting the Performance Evaluation of Amazonia Lee project in Peru. She holds a Master of Public Administration (Fulbright Scholar 1986) and a Master of Philosophy from New York University, and a bachelor’s degree in Economics from Universidad de Lima. She is fluent in both Spanish and English. INGRID ROJAS ARELLANO, TASKING MANAGER AND QUANTITATIVE ANALYSIS LEAD Ms. Ingrid Rojas is a Research Scientist at NORC at the University of Chicago. She has nine years of experience conducting and managing research projects in a wide range of sectors, including labor, education, health, agriculture, transport, telecommunications, electricity, water and sanitation, governance, security, justice, and human trafficking. Ms. Rojas’ work has included research and evaluation projects in the Caucasus region, Latin America, South Asia, West Africa, and East Africa. A native Spanish speaker, Ms. Rojas is also fluent in English. She holds a Master of International Development Policy from Georgetown University, a Master of Public Administration from Syracuse University, and a B.S. in Economics from Universidad del Pacífico, in Lima, Peru. Ms. Rojas has expertise designing data collection instruments, programming ODK surveys, training enumerators, overseeing data collection, performing data quality review, and analyzing large datasets to carry out performance and impact evaluations. She also has experience drafting evaluation reports and editing them for 508 compliances, as well as de-identifying data and preparing supporting documentation for public release. When assessing the effects of development interventions, she has used both qualitative and quantitative methods; Ms. Rojas has experience conducting key informant interviews and focus group discussions, as well as using experimental and-quasi experimental techniques. In her role of project manager, she supervises the work of research analysts, writes quarterly progress reports and annual work plans, monitors the financial health of projects, and keeps good communication with stakeholders, among others. CONTRACT No. GS-10F-0033M / ORDER No. 7200AA18M00016 USAID.GOV USAID/PERU TPI INTEGRITY NETWORKS EVALUATION: MIDLINE REPORT | 90 JANET PORRAS-MENDOZA, LOCAL SUBJECT-MATTER EXPERT Dr. Janet Porras is a mixed-methods researcher and public sector specialist with experience evaluating and analyzing the operational and budgetary performance of social programs and subnational finances. She possesses extensive experience in creating and managing databases to analyze quantitative data. She is also skilled in qualitative methods, including designing, conducting, and analyzing focus group discussions and key informant interviews. Much of her professional life has been dedicated to reducing corruption in public investment and procurement. As a Public Sector Specialist for World Bank, she has helped analyze and provide recommendations for strengthening government institutions’ financial systems across Latin America. In particular, she contributed to a report on the reform of the regional governments’ financing system in Peru, identifying the main limitations of the fiscal structure and designing an equalization transfer. She holds a Ph.D. in Public Management and Policy and a Master’s in Economics, both from Georgia State University, and she is fluent in Spanish and English. OMAR NARREA RIVAS, LOCAL SUBJECT-MATTER EXPERT Mr. Omar Narrea is a specialist in public management and technical expert in quantitative research. He possesses strong skills in monitoring public policy instruments for investment at both national and territorial levels. Mr. Narrea is committed to promoting public and private actors’ engagement in transparent public investment. As an Affiliate Consultant for the Universidad del Pacífico’s School of Public Management, he served as a data specialist for a Ministry of Economy and Finance project related to the balance of the implementation of the new National System of Public Investment (Invierte.pe). Additionally, as a Specialist in the National Budgetary System of Peru for the Ministry of Economy and Finance, Mr. Narrea was responsible for a team of specialists to carry out 18 budgetary evaluations with a 12% coverage of the National Public Budget. He received his MSc in Mega Infrastructure Projects at the Bartlett School of University College London and an MSc in Social Policy from the London School of Economics. He is fluent in Spanish and English.