PROREP (PROGRAM REPRESENTASI) PERFORMANCE MANAGEMENT PLAN (PMP) REVISED JUNE 2015 Contract No. AID-497-C-11-00002 June 3, 2015 This document has been produced for review by the United States Agency for International Development. It has been prepared by Chemonics International Inc. Project Contact Information: Chemonics International Inc., John K. Johnson, Chief of Party, Gedung BRI II, Suite 2806, Jl. Jend. Sudirman No. 44-46, Jakarta 10210 jkjohnson@chemonics.com Chemonics International Inc., Veenita Kaushik, Director, 1717 H Street, NW, Washington, D.C. 20006, USA vkaushik@chemonics.com Picture on cover: Infant receives checkup at community health center (Puskesmas) in Sumedang, West Java. ProRep’s health policy community is working on health policy initiatives to formalize citizen participation and multi-stakeholder collaboration in monitoring the quality of health services. The views expressed in this document do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS Introduction ................................................................................................................ 1 1. PMP design........................................................................................................1 2. ProRep Results Framework ...............................................................................2 3. Monitoring and Evaluation Design......................................................................4 4. Revisions to Results Framework and PMP Indicators over time ........................9 5. Updated performance data table......................................................................24 Appendix A: Summary of Comments from ProRep’s Mid-Term Evaluation..............35 Appendix B: Schedule of M&E Key Activities for Year 5 ..........................................38 PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 1 Introduction ProRep’s Performance Mangement Plan (PMP) has been updated twice in the life of the project. The first update was in response to (a) changes suggested in the project’s mid-term evaluation (MTE) in 2013, (b) USAID’s new 2014-2019 Country Development Cooperation Strategy (CDCS) and (c) the start of ProRep’s new “policy cluster,” program focus. Minor additional changes were made in 2015, as ProRep added, at USAID’s request, a new Democratic Rights and Governance Cluster. These adjustments, however, were close enough to ProRep’s original design that the PMP did not need to be completely restructured. Some additions to and expansion of the ProRep Results Framework were sufficient, in addition to changes to definitions of indicators. For ease of following the changes made to the originally approved PMP, sections 1 – 3 include the language from our original PMP in plain text, with new language in this document in italics. - original Introduction begins - This PMP (Performance Management Plan) presents an overview of the monitoring and evaluation (M&E) functions supporting the work of ProRep. It includes indicators, measures of qualitative and quantitative achievement over the duration of the project, and data sources. It also establishes the role of Chemonics International (along with its M&E subcontractor, Social Impact) in gathering data and submitting performance reports. The PMP is an update of the version included in Chemonics’ original proposal for carrying out ProRep. It is based partly on the ProRep component assessments done in July 2011, and partly on program discussions with USAID that have helped clarify desired indicators and reporting requirements. 1. PMP design The PMP is designed to involve all ProRep’s technical team members and relevant project stakeholders. This design enables: Efficiency. Technical team members have first-hand knowledge of activities and immediate results in their areas of work, and are best suited to collect and verify basic PMP data in their respective technical areas. Ownership. Technical team members appreciate that the PMP system belongs to the entire project team. This will ensure that the information generated is relevant and consistent with the interests of the project. Feedback. Having collected and analyzed M&E information, technical team members will have first-hand information on project progress and will be able to use M&E data to guide project implementation. Capacity building. By being involved in M&E, ProRep team members will able to transfer skills to project participants, helping them build data spreadsheets, analyze data, and monitor results. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 2 Adaptability. The PMP reflects project plans as currently conceived, but we have borne in mind that project tools and activities will likely continue to be refined during one year and indeed the life of the project. This being the case the PMP is a living document that will be updated as needed to reflect changes in ProRep strategy and project activities. A sense of importance. ProRep regards performance measurement as important. The PMP will give technical team members back-up support on data collection and quality analysis, and maintain auditable records of indicator progress and quality. Full details of performance measurement (including scoring schemes or survey scales, for example) will be laid out in performance indicator reference sheets that will be completed during Quarter 2. These sheets spell out the precise definition of each indicator, the management utility of tracking the information, the unit of measure, the method of acquisition, the frequency of collection, the data source, baseline and target information, data analysis, data quality assessment plans, and who among the ProRep technical staff is responsible for collecting the data. 2. ProRep Results Framework As a planning, communications and management tool, the ProRep Results Framework (RF) embodies the cause-effect relationships between ProRep’s Program Objective, Program Intermediate Results (PMIs) and Key Result Areas (KRAs). The Program Objective is at the pinnacle of the ProRep Results Framework; it is ‘Better informed and more representative legislative and policy-making processes’. ProRep’s Program Intermediate Results (PIRs), contribute to the realization of the Program Objective, and are three in number. They are:  ‘Representative capacity of membership- and constituency-based CSOs strengthened’  ‘Research institutions’ capacity to conduct and disseminate policy-relevant research and analysis on key policy and governance issues strengthened’  ‘More effective, responsive and transparent legislative processes’ These three PIRs are, in turn, realized by means of results achieved in 11 key result areas (KRAs). Details of these 11 KRAs are given below, while the whole Results Framework are given in Exhibit A: Amended ProRep Results Framework, below. Please see Table 1: Summary of Revions to ProRep Results Framework, for a summary of these changes, including the addition of a KRA, bringing the total KRAs to 12. PIR 1: Representational capacity of membership and constituency-based CSOs strengthened Under PIR 1, ProRep is to strengthen the effectiveness of 16 to 20 CSOs with well-defined members and/or constituencies that aim to: 1) expand their membership/constituencies and/or 2) better represent the views, interests and aspirations of their members/constituents to government at the national or regional level. PIR 1 will be achieved in three Key Result Areas (KRAs): KRA 1.1: CSO Membership, constituencies and networks significantly expanded PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 3 KRA 1.2: Partner CSOs engaging more frequently and effectively with legislators and policymakers. KRA 1.3: Partner CSOs more frequently using media for advocacy and outreach PIR 2: Research institutions’ capacity to conduct and disseminate policy research on key policy and governance issues strengthened Under PIR 2, ProRep is to provide training, TA, grants and other forms of support to 15 to 20 research institutions (universities, think tanks or CSOs) to build their capacity to do timely, high quality and sustainable policy-relevant research and analysis and to effectively disseminate their findings to legislators, policymakers, the media and other stakeholders. PIR 2 will be achieved in three KRAs: KRA 2.1: Improved institutional effectiveness and institutional policy research capabilities KRA 2.2:Increased volume and quality of policy research produced and disseminated KRA 2.3: Institutional research more relevant to and having more influence on legislative policy making. KRA 2.3 changed to Increased relevance of policymaking and oversight of research produced by Research Institutions. See Table 1 regarding the change to KRA 2.3. PIR 3: More Effective, Responsive and Transparent Legislative Processes (PIR 3 was changed to “More Effective, Responsive, and Transparent Legislative and Policymaking Processes). Under PIR3, ProRep will support efforts to make the DPR (and possibly the DPD) more effective, responsive and transparent. This will be done by providing training, short- and long-term technical assistance, study tours and other support to various groups. PIR3 is divided into five KRAs: KRA 3.1: DPR members more aware of and better able to represent constituency interests KRA 3.2: More informed, efficient, and responsive lawmaking KRA 3.3: More effective and transparent budget analysis and oversight KRA 3.4: DPR management and procedural reforms adopted and implemented KRA 3.5: Policymakers and Other Actors Collaborate on Key Policy Issues KRA 3.5 was added in 2014. In addition, with ProRep no longer providing parliamentary assistance after project year 3, targets for KRAs related to this program (KRAs 3.1, 3.2., 3.3., and 3.4) were changed to zero for Years 4 and 5. With this PIR’s broader focus, it covers both the original – legislative – project work, and ProRep’s new policy work. Three performance indicators were added to help measure progress in this new KRA. See Table 2: Summary of Revisions to ProRep Performance Data Table. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 4 PIR 4: Assistance for special initiatives needed to protect or advance democratic governance. Legislative and policymaking processes are always somewhat unpredictable; inevitably there are unanticipated changes, problems and opportunities. Therefore, USAID needs to preserve a degree of flexibility and resources in order to be able to respond to new and changing needs. Under this PIR timely but typically short-term assistance will be provided to government agencies and independent bodies to address unanticipated needs and opportunities associated with protecting or advancing democratic governance. PIR 14will be achieved in one Key Result Area (KRA): KRA 4.1: Indonesian efforts to address strategic needs and opportunities affecting democratic governance improved 3. Monitoring and Evaluation Design 3.1. Methodology, Analysis and Communication Monitoring progress and evaluating results are key management functions in performance￾based project implementation. Performance monitoring will allow the ProRep team to determine whether or not an activity is making progress towards the intended results. The strength of ProRep’s M&E will lie in its ability to provide timely performance information that enables ProRep to manage for results and to improve project performance. ProRep’s approach to monitoring and evaluation will focus on collecting information that can be corroborated and verified by documentation obtained from project participants and stakeholders. As noted earlier, the whole ProRep technical team will be involved, as the quality of data requires the input and work of not only the M&E officer, but also the component Specialists. This approach is reliable and cost-efficient since the component Specialists liaise regularly with project counterparts and make field visits to their locations. Therefore, they can collect data for analysis during their regular activities. Analysis and communication are important elements of performance management. The ProRep project team will not only collect performance and impact data; it will also add value to the raw data by analyzing the data and putting it in context, thus turning data into information. This transformation must then be communicated to have an impact. ProRep will communicate the information as appropriate to USAID, and (with USAID’s agreement) as appropriate to the Government of Indonesia (GOI), project participants, other stakeholders, and the public. 3.2. Indicators The ProRep M & E officer will conduct the analysis of performance indicator data, provide the comprehensive coverage needed to review project progress, and also provide inputs for project management. Every quarter the PMP system will track two main types of performance indicators: output and outcome. Output indicators will track project activities as they take place, and will account for how project resources are spent. An example of an output indicator is the number of persons trained in a workshop. Outcome indicators will PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 5 account for the impact of project activities, measuring their effects or results and providing feedback to managers so that they can identify areas where implementation strategies may need to be adjusted. An example of an outcome indicator is the quality of draft legislation as assessed by an expert in a case where the project had trained the legislative drafters. Quantitative and qualitative indicators. Quantitative indicators will be used to record and report results where project outcomes and outputs are easily measurable in dimensions or amounts. These will be expressed in numeric values, either numbers or percentages. We have also included several qualitative indicators in order to convey more complex processes or concepts. This data will usually be measured through expert assessment and opinion. Disaggregation of data. Where possible, indicators will be disaggregated by gender and other factors as appropriate to specific indicators. Timing and duration. The ProRep M&E plan is for the full five-year duration of the project. At the end of each year, the M&E plan will be reviewed and adjustments made as needed for the following years. Any changes will be documented in ProRep’s M&E files. The table in Section 5 of this updated PMP details performance indicators in each PIR, with definition of indicator (including unit of measurement), data disaggregation, data source(s), method(s) and scope of collection, frequency of data collection, baseline(s) and designated targets for the 1st and 5th years. As reference information, in the performance data table of this updated PMP we include an additional column with achievement information to date, based on original indicators for Years 1-3, and revised indicators for Year 4. 3.3. Baselines and Targets Initial baselines and targets for Year 1 and Year 5 have been identified for those indicators that are already supported by program-relevant information sufficient to allow such identification. During the coming months, ProRep will focus on further baseline data collection and verification, along with discussion of performance targets with USAID and ProRep partner organizations. Over the life of the project there have been changes regarding baselines, targets and to some indicator definitions, and these are noted in the Updated Performance Data Table. With regard to baselines, given that many indicators we are measuring were new (for example, ProRep partners were not producing research products for parliament – KRA 2.2.a), and the difficulty and expense of collecting some of this information (ProRep had virtually no access to the DPR during much of the first year of the project, for example) ProRep assumed a “0” baseline for indicators. As per the ProRep contract (p. 41) the project conducted a number of assessments at the start-up phase of the project which served to establish an initial baseline regarding the capacity and needs of the potential beneficiaries. In accordance with the original approved PMP the project also conducted a Knowledge, Attitudes, and Practices (KAP) survey of CSO partners, and subsequently conducted periodic assessments of supported partners to capture and assess organizational changes. Under PIR 3, above, we note that we added some indicators and stopped collecting information on others – corresponding with changes to project direction and focus. In PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 6 addition, following ProRep’s mid-term evaluation, and comments from evaluators that Indonesia’s political situation, and the realization that effecting policy changes of any kind through the parliament was a much slower and more difficult process than USAID or the project implementers had realized, we changed the definition of some indicators (see Appendix A for the evaluator’s comments regarding project indicators). In addition, given that a great of deal ProRep’s work was with local-level CSOs which were attempting to help bring about policy change at the local level, we amended some indicators to capture local￾level changes as well as national. Please see Sections 4:Revisions to Results Framework and PMP Indicators over time, and 5: Updated performance data table, for detail. 3.4. Roles and Responsibilities in Data Management The ProRep M&E Specialist will take the lead in developing the system, using a participatory approach to ensure that all members of the technical team understand both the importance of and their role in contributing to performance-based management. As mentioned earlier, The PMP is designed to involve all technical team members and relevant project stakeholders. 3.4.1. Quality Reviews Technical team members are conducting initial quality reviews for various M&E data. Upon completion of the data entry spreadsheets, each technical team member concerned will examine the quantitative data to identify weaknesses in the data or errors such as logical inconsistency, out-of-range values, or significant departures from trends. Should any problem be identified, the component Specialist will be responsible for working with the M&E Specialist to verify data against original sources and if necessary cross-verify them against data from alternate sources, and also for identifying indicators and data collection methods that might replace those found to be unreliable. 3.4.2. Data Sources and Collection Methods The M & E Specialist will obtain data on project indicators from a variety of sources in collaboration with the ProRep technical teams. Sources will include technical team members, CSOs, universities, think tanks, and DPR members and staff, as well as documents such as internal project records, R & A, legislation drafted and passed, DPR procedural manuals, and media reports. ProRep understands the importance of collecting both quantitative and qualitative data in order to capture the complexity of project outcomes. The specific data source for each indicator is identified in the indicator table. Primary data from project records. Some of the indicators presented in this PMP directly measure outputs of project activities, so data for such activities (such as numbers trained) can easily be attained from project records. Secondary data from project partners or public records. Data collection on other project indicators requires more complex collaboration with project partners and participants, particularly the DPR. ProRep staff will work with these partners and participants to establish a means of regularly collecting this data, through performance measurement tools, instructions, and consultations, so as to serve our collective purposes for the duration of the project and beyond. Qualitative indicators. As mentioned, for some indicators we will use established experts. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 7 Gender. In all possible and relevant instances, the PMP features collection, analysis and reporting of data disaggregated by gender. The M&E Specialist will ensure that project indicators address gender differences where they exist, are free from underlying discriminatory assumptions, and where relevant, include indicators that measure differential impact on men and women. Service providers / grantees will also be trained to take gender considerations into account, such as with whom and under what circumstances women may speak freely when the collection method is an interview. 3.4.3. Data Storage We will create a database to track key legislative and policy reforms that will be used for M&E purposes. The technical teams will have primary responsibility for this database. The M & E Specialist will conduct regular spot checks for accuracy. In addition, ProRep will create a database giving information on grants, technical assistance, training, and other support given during project implementation. 3.5. Reporting and Review ProRep will provide M&E updates in its quarterly reports to USAID. The reports will include a summary of activities undertaken to control, verify, and validate the M&E data being reported, as well as any anomalies discovered and the corrective measures taken to resolve them. These reports will also provide contextual analysis when factors beyond the project’s control affect M&E information. ProRep’s annual report to USAID will contain in-depth analysis of annual progress, an update of annual targets, discussions of progress and hurdles, and a presentation of success stories, lessons learned, and best practices. In addition to providing quantitative data, the technical staff will also provide written narratives covering major achievements during the reporting period and/or major obstacles hampering progress. Success stories. ProRep will be looking out for success stories. Routine and special assessments of legislative and policy processes in Indonesia will be included as part of project reporting. As they take place they will be included in quarterly and annual reports. Project information will be available as requested to provide liaison support to USAID, including audio-visual presentations, speeches, talking points, and briefings. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 8 Exhibit A: Amended ProRep Results Framework Note: Revisions given in blue/lighter font. PROGRAM OBJECTIVE: Better Informed and More Representative Legislative and Policymaking Processes P.I.R 1 Representational Capacity of Membership and Constituency Based CSOs Strengthened PIR 4 Assistance for Special Initiatives Needed to Protect or Advance Democratic Governance. PIR 3 More Effective, Responsive and Transparent Legislative and Policymaking Processes P.I.R 2 Research Institution Capacity to Conduct and Disseminate Policy￾Relevant Research and Analysis on Key Policy and Governance Issues Strengthened KRA 1.1 CSO Membership, Constituencies and Networks Significantly Expanded KRA 4.1 Indonesian Efforts to Address Strategic Needs and Opportunities Affecting democratic Governance Improved KRA 3.1 DPR Members Better Represent Constituent Interests KRA 2.1 Research Institutions Improve Institutional Effectiveness and Applied Policy Research and Analysis Capabilities KRA 1.2 Partner CSOs Engage More Frequently and Effectively with Legislators and Policy Makers KRA 3.4 DPR Management and Procedural Reforms Adopted and Implemented KRA 3.2: DPR Staff Research, Analysis and, Drafting Improved KRA 2.2 Research Institutions Increase Volume and Quality of Policy￾Relevant Research and Analysis Produced and Disseminated KRA 1.3 Partner CSOs More Frequently Use Media for Advocacy and Outreach KRA 3.5 Policymakers and Other Actors Collaborate on Key Policy Issues KRA 3.3 DPR Institutional Capacity to Do Budget Analysis and Oversight Strengthened KRA 2.3 Increased Relevance to Policymaking and oversight of Research Produced by Research Institutions PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 9 4. Revisions to Results Framework and PMP Indicators over time The two tables in this section summarize the changes to ProRep’s Results Framework, and PMP indicators. Table 1: Summary of Revisions to ProRep Results Framework Original Modification Rationale KRA 2.3 Increased Relevance to and Influence On Legislative Policy Making of Research Produced by Research Institutions KRA 2.3 Increased Relevance to Policymaking or Oversight of Research Produced by Research Institutions To capture the broader scope of our policy cluster program, not limited to legislative process. PIR 3 More Effective, Responsive and Transparent Legislative Processes PIR 3 More Effective, Responsive and Transparent Legislative and Policymaking Processes To capture the broader scope of our policy cluster program, not limited to legislative process. None KRA 3.5 Policymakers and Other Actors Collaborate on Key Policy Issues To elaborate new indicators to measure progress in our new policy cluster approach. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 10 ProRep revised the PMP in response to feedback received in the mid-term evaluation, and the project’s change in scope (i.e. ending support to the Indonesian legislature, and the shift to policy clusters). Table 2, below summarizes the revisions to the performance indicators data table and the rationale for these revisions. Table 2: Summary of Revisions to ProRep Performance Data Table Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale PO.a. Definition: Definition: Definition: Modification #1 Installation of a new government and parliament in 2014 is expected to make policy changes more difficult (historically, Indonesian parliaments make very little policy change until later in their tenure). Modification #2 The definition is revised to adjust to the new policy cluster approach, which focuses not solely on central government regulation and law but also focus on local government regulation. Number of national bills, amendments or laws positively incorporating concepts or language recommended by one or more ProRep partner CSOs or CSo coalitions Number of national or local bills, amendments laws or policies positively incorporating concepts or language recommended by one or more ProRep partner CSOs or CSO coalitions Policies may include bills, amendments, laws, decrees, government and parliament regulations and procedures Number of national or local bills, amendments laws or policies positively incorporating concepts or language recommended by one or more ProRep partner CSOs or CSO coalitions Policies may include bills, amendments, laws, decrees, government and parliament regulations and procedures at the national or local/regional level. Disaggregation: Disaggregation: Bills, amendments or laws relevant to concern of a. gender b. disadvantaged groups Woman focused Data Source: Data Source: Grantee reports; interviews with selected MPs or their staff Grantee reports; interviews with selected MPs or their staff. Also verified through response sheets from policymakers or reference to research in policy document. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 11 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale Target Target Yr 1: 2 Yr 2: 2 Yr 3: 2 Yr 4: 2 Yr 5: - Yr 1: 2 Yr 2: 2 Yr 3: 2 Yr 4: 2 Yr 5: 2 PO.b. Definition Definition Modification #1 Installation of a new government and parliament in 2014 is expected to make policy changes more difficult. Modification #2 Research group in each cluster will also focus on region/local government. Number of national bills, amendments or laws incorporating concepts or language recommended by a participating university, think tank or CSO Number of national bills, amendments, laws or local government regulation/decree incorporating concepts or language recommended by a participating university, think tank or CSO Target: Target: Target Cumulatively: Yr 1: 2 (Women-related: 1) Target to be determined based on year 1 data Yr 1: 2 Yr 2: 1 Yr 3: 1 Yr 4: 1 Yr 5: 1 Yr 1: 2 Yr 2: 1 Yr 3: 1 Yr 4: 1 Yr 5: 7 PIR 1.a Definition: Definition: To expand the indicator definition to cover both membership-based organizations and constituent￾based organizations. ProRep desk assessment of the extent to which partner CSOs use various methods of receiving information and/or opinions on issues from individual members. Methods may include Face-to-face meetings; the CSO’s website; social networking; and text ProRep desk assessment of the extent to which partner CSOs use various methods of receiving information and/or opinions on issues from individual members. For non- membership organizations, this will include collecting information from constituents. Methods may include Face-to-face meetings; the CSO’s website; social PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 12 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale messaging. networking; and text messaging. Data source Data Source ProRep will carry out a rapid assessment, utilizing a short checklist and based on data in the CSO KAP survey (Yr 1, 3 and 5) and CSO reports Grantee reports with documentation of methods used. Frequency: Frequency: Bi-annual: Years 1, 3 and 5 Annual Target Target Yr 1: 2 Yr 2: 2 Yr 3: 3 Yr 4: 3 Yr 5: 2 To be determined based on the result analysis of KAP survey Yr 1: 2 Yr 2: 2 Yr 3: 3 Yr 4: 3 Yr 5: 3 PIR 1.b Data source Data source Improvements in technology make internet and SMS gateway surveys possible and a cost effective option to gathering data. Modification #1: The percentage is per year, but an average during the project period. Survey of CSO members (probability sample to margin of error of +/- 5%) Survey of CSO members via in￾person interviews, Internet and/or SMS gateway. Frequency: Frequency: Bi-annual: Years 1, 3 and 5 1 time during the life of project. Target: Target: Yr 1: 10% (women: 20%) Yrs. 1-5: 30% (women:25%) PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 13 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale Yr 2: 20% (women: 25%) Yr 3: 30% (women: 25%) Yr 4: 35% (women: 30%) Yr 5: 40% (women: 35%) It requires initial inputs/information of current practices which will be gathered by KAP. KRA 1.1. Definition Definition: Modification #1 To expand the indicator definition to cover both membership-based organizations and constituent￾based organizations. The policy cluster focus for ProRep’s Years 4 and 5 will result in decreased focus on membership expansion, with an increased focus on achieving policy improvements Modification #2 Initially ProRep counted only membership-based CSOs and numbers were low because of most ProRep CSO partners we non-membership based With ProRep’s shift in focus to policy clusters in the 4th project year, project focus is less on expanding memberships and more on building policy clusters Number of ProRep-supported CSOs experiencing an increase in membership of 10 % or more Number of ProRep-supported CSOs experiencing an increase in membership of 3% or more, or with increased numbers of constituents. For non-membership organizations this must be demonstrated by an increase in number of documented constituents (through a database, for example) or in an increase in the number of organizations represented. Data source Data Source ProRep count based on data from KAP survey in years 1, 3 and 5, and CSO reports in years 2 and 4. Grantee reports with reference to member database Target Target Target Yr 1: 2 Yr 2: 4 Yr 3: 6 Yr 4: 8 Yr 1: 2 Yr 2: 4 Yr 3: 6 Yr 4: 2 Yr 1: 2 Yr 2: 4 Yr 3: 6 Yr 4: 2 PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 14 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale Yr 5: 10 Yr 5: 2 Yr 5: 3 and policy change. Consequently, targets for years 4 & 5 are reduced. KRA 1.2.a Definition Definition Installation of a new parliament in 2014 is expected to result in fewer opportunities for hearings and other formal sessions. Number of ProRep partner CSOs that report that their representatives have formally provided, during the reporting period, information or policy perspective to MPs or Parliament staff during committee hearings and other formal sessions addressing legislation, oversight or budget. Number of ProRep partner CSOs that report that their representatives have provided, during the reporting period, information or policy perspective to MPs or Parliament staff, and representatives of government, including at the local/regional level addressing legislation, oversight or budget. Disaggregation Disaggregation: Number of organization participating in legislative proceedings Number of organizations Data Source Data Source CSO grantee records MP or related policy makers response sheet KRA 1.2.b Definition Definition Modification #2 New policy cluster approach will not focus solely on parliament and will involve government representatives. Installation of a new government and parliament in 2014 is Number of written policy advocacy products submitted to MPs or staff by ProRep partner CSOs with content that is also found, in substance, in subsequent legislation or legislative, oversight or budget sessions or summaries. Legislative, oversight or budget Number of written policy advocacy products submitted to MPs /staff or national/local government representatives by ProRep partner CSOs with content that is also found, in substance, in subsequent legislation or legislative, oversight or budget sessions or PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 15 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale response may include the defeat of content that is counter to CSO policy interests. summaries. Legislative, oversight or budget response may include the defeat of content that is counter to CSO policy interests. expected to make policy changes more difficult. Target Target: Target Modification #1 reduced targets for years 4 & 5 because ProRep, which had focused on parliament, was no longer working with the institution. Modification #2 reflects the new policy cluster approach and changes the definition so that it includes national and local government representatives. Therefore, the year 5 target is increased. Yr 1: 2 Yr 2: 3 Yr 3: 5 Yr 4: 6 Yr 5: 8 Yr 1: 2 Yr 2: 3 Yr 3: 5 Yr 4: 2 Yr 5: 2 Yr 1: 2 Yr 2: 3 Yr 3: 5 Yr 4: 2 Yr 5: 5 KRA 1.3.b Target Target: Target Modification #1 Achievement of year 2-4 exceed the target. ProRep cooperated with journalist partners with excellent access to the media (i.e. AJI, Seknas Fitra and IBC). New policy cluster focus will involve substantive media engagement and outreach. Targets for Years 4 and 5 have increased due to our ability to meet higher than expected targets in Years 1-3. Yr 1: 3 (Women-related: 1) Yr 2: 5 (Women-related: 1) Yr 3: 7 (Women-related: 2) Yr 4: 9 (Women-related: 2) Yr. 5 : 11 (Women-related: 3) Yr 1: 3 (Women- related: 1) Yr 2: 5 (Women- related: 1) Yr 3: 7 (Women- related: 2) Yr 4: 20 (Women- related: 3) Yr. 5 : 20 (Women- related: 3). Yr 1: 3 (Women- related: 1) Yr 2: 5 (Women- related: 1) Yr 3: 7 (Women- related: 2) Yr 4: 20 (Women- related: 3) Yr. 5 : 36 (Women- related: 3). PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 16 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale Modification #2 Targets for Years 4 and 5 have increased due to our ability to meet higher than expected targets in Years 1-3. Also, our new policy cluster focus will involve substantive media engagement and outreach. PIR 2.a Definition Definition Modification #1 It is more practical and cost effective to conduct this survey one time during the project period. On an ongoing basis, grantees provide ProRep with “response sheets” documenting policy makers responses to the research they provide. Modification #2 It proved more practical and cost effective to conduct this survey one time during the project period. On an ongoing basis, grantees provide ProRep with “response sheets” documenting policy makers response to research provided. The definition is revised to adjust to the new policy cluster approach, which targets not just parliament members and staff but also national and local government representatives. Percent of surveyed MPs and Parliament staff who report they are “satisfied” or “very satisfied” with at least one product of at least one ProRep-supported research institution. Respondent ratings will be counted only when the respondent can identify or recall a particular research product or study. [See KRA2.3.c for specification of universe of survey] Percent of surveyed MPs/ Parliament staff or national/local government representatives who report they are “satisfied” or “very satisfied” with at least one product of at least one ProRep-supported research institution. Respondent ratings will be counted only when the respondent can identify or recall a particular research product or study. [See KRA 2.3.a for specification of universe of survey]. Frequency Frequency: Yrs 1, 3 and 5 1 time during the life of project. Target Target Yr 1: 10% Yr 2: 10% Yr 3: 10% Yr 4: 10% Yr 5: 10% [Targets to be determined Years 1-5: 30% PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 17 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale based on Yr 1 data.] KRA 2.1 Data source Data source Modification #1 Expanding data sources enables collection of additional data. The policy cluster focus for ProRep’s Years 4 and 5 will result in a decreased focus on organizational capacity building, and a greater focus on achieving policy improvements. Modification #2 The policy cluster focus for ProRep’s Years 4 and 5 will result in a decreased focus on organizational capacity building and a greater emphasis on policy change. Organizational Effectiveness Assessment Organizational Effectiveness Assessment and Grantee reports with documentation of new organizational activities/achie vements. Target Target: Yr 1: 3 Yr 2: 6 Yr 3: 5 Yr 4: 9 Yr 5:10 Yr 1: 3 Yr 2: 6 Yr 3: 5 Yr 4: 5 Yr 5: 3 KRA 2.2.a Definition Definition: Definition: Year 2 and 3 achievements exceed targets because some ProRep partners distributed research products to working groups, such as the women’s caucus or through several special committees on particular subjects. New policy cluster approach will focus on parliament AND government representatives both national and local. Number of copies (in hard-copy or electronic format) of research study products disseminated from ProRep partner research institutions to Parliament members or their staff Number of copies (in hard- copy or electronic format) of research study products disseminated from ProRep partner research institutions to Parliament members or their staff, and representatives from government. Number of copies (in hard- copy or electronic format) of research study products disseminated from ProRep partner research institutions to Parliament members or their staff, and representatives from government – including national and local government representatives. KRA 2.2.b Definition Definition Definition Modification #1 PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 18 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale Number of target Parliament members and staff (e.g., members of selected commissions, BAKN, BALEG, etc.) who report they have received written research products from ProRep partner research institutions. Number of target Parliament members and staff (e.g., members of selected commissions, BAKN, BALEG, etc.) or national and local government representatives who report they have received written research products from ProRep partner research institutions Achievement did not hit targets because most ProRep partners have never worked wth MPs. They had little background producing policy documents for parliament or interacting with MPs. New policy cluster approach will not focus solely on parliament and will involve government representatives. Modification #2 ProRep partners had difficulty reaching targets because the ProRep program provided their first support to work explicitly with MPs on policy issues. New policy cluster approach will not focus solely on parliament and will involve government representatives either national or local. Data source Data source ProRep survey of target Parliament members ProRep survey of target Parliament members Response sheets from policy makers provided by grantees. Target Target Cumulatively: Yr 1: 50 Yr 2: 80 Yr 3: 110 Yr 4: 140 Yr. 5: 184 Yr 1: 50 Yr 2: 80 Yr 3: 110 Yr 4: 20 Yr. 5: 10 KRA 2.3.a Data source Data Source New policy cluster approach will not focus solely on parliament and will involve government representatives so targets for years 4 & 5 are lower Survey of selected Parliament members Response sheets from policy makers provided by grantees. Target Target Yr 1: 15 Yr 2: 15 Yr 1: 15 Yr 2: 15 PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 19 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale Yr 3: 15 Yr 4: 15 Yr 5: 15 Yr 3: 15 Yr 4: 10 Yr 5: 5 PIR 3.a Target Target Modification #1 Installation of a new government and parliament in 2014 is expected to make policy changes more difficult. Modification #2 In Years 4 and 5 ProRep will no longer provide assistance to parliament, and the installation of a new government and parliament in 2014 will mean fewer legislative amendments, so targets for parliamentary achievements are reduced. Yr 1: Deliberated: 2, Passed : 1 Yr 2: Deliberated: 2, Passed: 1 Yr 3: Deliberated: 3, Passed: 1 Yr 4: Deliberated: 3, Passed: 2 Yr 5: Deliberated: 2, Passed: 1 Yr 1: Deliberated:2, Passed: 1 Yr 2: Deliberated:2, Passed: 1 Yr 3: Deliberated:3, Passed: 1 Yr 4: Deliberated: 1, Passed: 0 Yr 5: Deliberated: 0, Passed: 0 See Rationale KRA 3.1 Data source Data Source Modification #1 Projec achievements did not hit targets in early years as ProRep’s JABAT (parliamentary outreach program) began implementation later. In Years 4 and 5 ProRep will no longer be providing direct assistance to parliament. Modification #2 ProRep used JABAT reports for an accurate measure of consultations with constituents. Early targets were not reached DPR records, confirmed by ProRep contacts with a sample of DPR members/ staff MP Response sheets/ Interviews Media coverage Target Target Yr 1: 100 (By women: 50) Yr 2: 100 (By women: 50) Yr 3: 150 (By women: 75) Yr 4: 200 (By women: 100) Yr 5: 100 (By women: 50) Yr 1: 100 (By women: 50) Yr 2: 100 (By women: 50) Yr 3: 150 (By women: 75) Yr 4: 0 (By women: 0) Yr 5: 0 (By women: 0). See Rationale PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 20 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale because the JABAT program began during Year 2. The Year 3 target was exceeded because of JABAT program success. ProRep program support to Parliament ends in year 4, so targets are reduced to 0. KRA 3.2.a Target Target ProRep’s Mid-Term Evaluation (MTE) notes that targets were unrealistically high, noting that, “The process of influencing Parliament takes much longer.” Achieving changes in multiple formal laws takes longer than the project period. Targets were reduced. Yr 1: 2 Yr 2: 5 Yr 3: 8 Yr 4: 3 Yr 5: 2 Yr 1: 2 Yr 2: 5 Yr 3: 8 Yr 4: 1 Yr 5: 1 KRA 3.3.a Target Target ProRep support to the BAKN (public accounts committee) led to several oversight actions, so targets for Years 2 and 3 were exceeded. In Years 4 and 5 ProRep will no longer be providing direct assistance to parliament, so targets are reduced to zero. Yr 1: 1 (gender-relevant: 0) Yr 2: 2 (gender relevant: 1) Yr 3: 3 (gender relevant: 2) Yr 4: 2 (gender relevant 1) Yr 5: 2 (gender-relevant: 1) Yr 1: 1 (gender- relevant: 0) Yr 2: 2 (gender relevant: 1) Yr 3: 3 (gender relevant: 2) Yr 4: 0 (gender relevant 0) Yr 5: 0 (gender- relevant: 0) KRA 3.4 Definition Definition: In Years 4 and 5 ProRep will no longer be providing direct assistance to parliament , so targets are reduced to zero. Number of DPR procedural reforms proposed, passed and implemented. A reform must be accompanied by some evidence of implementation in order to be counted. Institutional reforms including Standard Operating Procedures, internal rules and procedure in DPR. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 21 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale Target Target Yr 1: Deliberated: 1, Passed: 1 Yr 2: Deliberated: 1, Passed: 1 Yr 3: Deliberated: 2, Passed: 2 Yr 4: Deliberated: 1, Passed: 1 Yr 5: Deliberated: 1, Passed: 1 Yr 1: Deliberated: 1, Passed : 1 Yr 2: Deliberated: 1, Passed: 1 Yr 3: Deliberated: 2, Passed: 2 Yr 4: Deliberated: 0, Passed: 0 Yr 5: Deliberated: 0, Passed: 0 KRA 3.5.a Indicator: Modification #1 Adding new indicators of multi￾stakeholder forum due to policy cluster approach Modification #2 Target of achievements increased with the addition of democratic rights as a policy cluster. KRA.3.5.a Number of multi￾stakeholder forums convened with ProRep support to discuss key policy issues. Definition Number of forums bringing together at least three of the following stakeholders to discuss issues related to identified policy clusters: CSOs, research organizations, MPS and MP staff. constituents Number of forums bringing together at least three of the following stakeholders to discuss issues related to identified policy clusters: CSOs, research organizations, policymakers and constituents. Data Source Documentation of meetings which may include photographs or press reports. Frequency: Annual Target Target Yr.4: 6 Yr.5: 10 Yr.4: 6 Yr.5: 14 KRA 3.5.b Indicator: Modification #1 PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 22 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale KRA.3.5.b Number of multi￾stakeholder forums convened with ProRep support that develop formal policy recommendations. Adding new indicators of multi￾stakeholder forum due to policy cluster approach. Definition Number of multi-stakeholder forums (defined in KRA 3.5.a) that create formal, written recommendations on key policy changes or policies to be addressed. Data Source: Written policy recommendation Frequency Annual Target Yr.4: 0 Yr.5: 8 Yr.4: 0 Yr.5: 8 KRA 3.5.c Indicator: Modification #1 Adding new indicators of multi￾stakeholder forums with the new policy cluster approach Modification #2 KRA.3.5.c Number of multi￾stakeholder forums convened with ProRep support that develop formal action plans for influencing policy. Definition Number of multi-stakeholder forums PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 23 Indicator Original Modification #1 (August 2014) Modification #2 (April 2015) Rationale (defined in KRA 3.5.a) that create formal, written action plans for addressing policy change in key policy cluster areas. Achievement targets increased with the addition of democratic rights as a policy cluster. Data Source: Written action plans Frequency Annual Target Target Yr.4: 0 Yr.5: 4 Yr.4: 0 Yr.5: 5 KRA 4.1 Indicator Indicator: Modification #1 JABAT contributes a large number of achievements since we counted each MP/ grantee as an achievement by considering different method and approach that used by MP/grantee in engaging with their consitituents/members Several different JABAT activities during years 2 & 3 contributed to the project exceeding year 2 and 3 targets. Number of special initiatives undertaken by USAID through ProRep that result in significant policy improvements or reform. KRA.4.1. Number of special initiatives undertaken by USAID through ProRep Definition Definition: Number of initiatives undertaken whose aims are achieved as agreed and defined in consultation with USAID Number of initiatives undertaken by USAID to respond flexibly and rapidly to address other unanticipated needs and opportunities pertaining to the protection and advancement of democratic governance in Indonesia. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 24 5. Updated performance data table - Language from 2014 revision begins – The PMP year definitions presented below are aligned with the Federal Fiscal Year and ProRep’s annual reports to date. To cover the expected five calendar years of the project, PMP Year one covers a six month period, and PMP Year five covers 18 months. Therefore, to adjust the PMP to fit the program approach in Year 5, ProRep has slightly revised the following parts of PMP. Table 4 below presents the updated performance data table for ProRep Year 5. Some of the indicators have been revised to fit with ProRep’s new focus from components to clusters. In addition, since the parliament is no longer the focus of ProRep, indicators under Component 3 will also have a target of “zero”. As ProRep has broadened the targeted stakeholders, the definitions and related notes for select indicators have been slightly revised to accommodate this broadened set of stakeholders. The rationale for each revised indicator is provided in the notes section of the indicator table. As noted in Section 3.3, baselines are considered to be ‘0.’ ProRep Performance Management Plan Years: Year 1: April 19, 2011 – September 30, 2011 Year 2: October 1, 2011 – September 30, 2012 Year 3: October 1, 2012 – September 30, 2013 Year 4: October 1, 2013 – September 30, 2014 Year 5: October 1, 2014 – April 18, 2016 PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 25 Perrmance Indicator Definition of Indicator (including unit of measurement) Disaggregation (s) Data Source(s), Method(s) and Scope of Collection Frequency of Data Collection Baseline(s) Targets (Yr 1; Yr 5) Achievement (orig. indicator Ys. 1-3 & revised Indicators Ys. 4-5 Notes PROGRAM OBJECTIVE: BETTER INFORMED AND MORE REPRESENTATIVE LEGISLATIVE AND POLICYMAKING PROCESSES PO.a. Number of laws, legislative amendments or Parliamentary oversight proceedings influenced by CSO advocacy Number of national or local bills, amendments laws or policies positively incorporating concepts or language recommended by one or more ProRep partner CSOs or CSO coalitions Policies may include bills, amendments, laws, decrees, government and parliament regulations and procedures at the national or local/regional level. Women-focused Grantee reports, interviews with selected MPs or their staff, response sheets from policymakers or reference to research in policy documents Annually Zero baseline Informed by Comp 1 Assessment, KAP Survey, and subsequent assessments of competence Yr 1: 2 Yr 2: 2 Yr 3: 2 Yr 4: 2 Yr 5: 2 Yr 1 : 0 of 2 Yr 2 : 1 of 2 Wf – 0 Yr 3 : 2 of 2 Wf-1Yr 4: 2 of 2Wf-1 The definition is revised to adjust to the new policy cluster approach, which focuses not only on central government regulation, laws and policies, but also focuses on local government policies. PO.b. Number of bills, amendments or laws influenced by research and analysis provided by a supported research institution Number of national bills, amendments, laws or local government regulation/decree incorporating concepts or language recommended by a participating university, think tank or CSO Laws relevant to concerns of : a) Gender b) Disadvantaged group Grantee reports Survey of selected Parliament members and staff Legislation for verification of grantee reports Quarterly Zero baseline Informed by Comp 2 Assessment Yr 1: 2 Yr 2: 1 Yr 3: 1 Yr 4: 1 Yr 5: 7 Yr 1 : 0 of 2 Yr 2 : 1 Yr 3: 1 Yr 4: 1 Research group in each cluster will also focus on region/local government. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 26 PIR 1: REPRESENTATIONAL CAPACITY OF MEMBERSHIP- AND CONSTITUENCY-BASED CSOS STRENGTHENED PIR1.a. Extent of CSO use of diverse mechanisms to receive information and opinions from their members ProRep desk assessment of the extent to which partner CSOs use various methods of receiving information and/or opinions on issues from individual members. For non￾membership organizations, this will include collecting information from constituents. Methods may include Face-to￾face meetings; the CSO’s website; social networking; and text messaging. Number of new mechanism used on selected issues and issue orientation of : (a) women (b) disadvantaged groups Grantee reports with documentation of methods used. Annual Zero baseline Informed by Comp 1 Assessment, KAP Survey, and subsequent assessments of competence Yr 1: 2 Yr 2: 2 Yr 3: 3 Yr 4: 3 Yr 5: 3 Yr 1 : 0 of 2 Yr 2 : 2 of 2 (1 wf-1 Yr 3 : 3 of 3 (1 Wf - 1 Yr 4: 3 of 3 Wf-1 PIR1.b Percent of CSO members and constituents who feel their interests have been effectively advanced by the CSO Percent of respondents to a survey of individual CSOmembers and constituent who indicate that CSO promotion of their interests on one or more key issues has been “effective” or “very effective.” Gender; disadvantaged groups Survey of CSO members via in￾person interviews, or Internet and/or SMS gateway interviews. 1 time during the life of project. Zero baseline Yrs. 1-5: 30% (women: 25%) SPPN Sergai: >80% YSKK: > 85% SPI Mamuju: <30% InProSula: 79 % KRA 1.1: CSO Members, Constituencies and Networks Significantly Expanded KRA1.1. Number of CSOs with significantly expanded membership Number of ProRep-supported CSOs experiencing an increase in membership of 3% or more, or with increased numbers of constituents. For non￾membership organizations this must be demonstrated by an increase in number of documented constituents (through a database, for example) or in an increase in the number of organizations represented. Number of organizations Grantee reports referring to member databases and numbers of constituents Annually Zero baseline Informed by Comp 1 Assessment, KAP Survey, and subsequent assessments of competence Yr 1: 2 Yr 2: 4 Yr 3: 6 Yr 4: 2 Yr 5: 3 Yr 1 : 0 of 2 Yr 2 : 2 of 4 Yr 3 : 5 of 6 Yr Yr 4: 6 of 2 Initially ProRep counted only membership-based CSOs and numbers were low because of most ProRep CSO partners were non￾membership based. With ProRep’s shift in focus to policy clusters in the 4th project year, project focus is less on expanding memberships and more on building policy clusters and policy change. Consequently, targets for PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 27 years 4 & 5 are reduced. KRA 1.2: Partner CSOs Engage More Frequently and Effectively with Legislators and Policy-makers KRA1.2.a. Number of ProRep￾supported CSOs that participate in legislative proceedings Number of ProRep partner CSOs that report that their representatives have, during the reporting period, provided information or policy perspective to MPs or Parliament staff, and representatives of government - including at the local/regional level - addressing legislation, oversight or budget. Number of organizations CSO grantee records or MP or related policy maker response sheets Quarterly Zero baseline Informed by Comp 1 Assessment, KAP Survey, and subsequent assessments of competence Yr 1: 2 Yr 2: 4 Yr 3: 6 Yr 4: 8 Yr 5: 10 Yr 1 : 0 of 2 Yr 2 : 4 of 4 Yr 3 : 6 of 6 Yr 4: 8 of 8 KRA 1.2.b. Number of policy briefs brought to Parliament by CSOs and substantively reflected in responsive legislation, oversight or budget proceedings Number of written policy advocacy products submitted to MPs /staff or national/local government representatives by ProRep partner CSOs with content that is also found, in substance, in subsequent legislation or legislative, oversight or budget sessions or summaries. Legislative, oversight or budget response may include the defeat of content that is counter to CSO policy interests. Issue focus of policy briefs: gender issues; issues of concern to other disadvantaged groups CSO activity records; content analysis of policy briefs, draft legislation and oversight and budget proceedings; interviews with MPs and MP and CSO staff Quarterly Zero baseline Informed by Comp 1 Assessment, KAP Survey, and subsequent assessments of competence Yr 1: 2 Yr 2: 3 Yr 3: 5 Yr 4: 2 Yr 5: 5 Yr 1 : 0 of 2 Yr 2 : 2 of 3 Yr 3 : 9 of 5 Yr 4: 2 of 2 Lower targets for years 4 and 5 reflect new policy cluster approach, which focuses not solely on parliament but will also include national and local government representatives. In addition, Installation of a new government and parliament in 2014 is expected to reduce numbers of policy changes those years. KRA 1.3: Partner CSOs More Frequently Use Media for Advocacy and Outreach KRA 1.3.b. Number of times CSO advocacy positions are featured in media Mass media; social network media; traditional media a) Type of issue b) Number of times advocacy are featured in media Contracted media analysis firm compiles media records, clippings, etc. Items may be found in products of mass media as well as traditional Quarterly Zero baseline Informed by Comp 1 Assessment, KAP Survey, and subsequent assessments of competence Yr 1: 3 (Women￾related: 1) Yr 2: 5 (Women￾related: 1) Yr 3: 7 (Women￾related: 2) Yr 4: 20 Yr 1: 0 Yr 2 : 55 of 5 Yr 3 : 72 of 7 Yr 4 : 68 of 9 Achievements for Years 2- 4 exceed targets because a number of ProRep either include journalists (such as AJI – the Indonesian Association of Independent Journalists) or work closely with journalists, such as Seknas Fitra and IBC. Targets for Years 4 and 5 have increased due to our PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 28 media. Partner CSOs will assist in identifying source material. (Women￾relate: 3) Yr. 5 : 36 (Women￾related: 3). ability to meet higher than expected targets in Years 1-3. Also, our new policy cluster focus will involve substantive media engagement and outreach. PIR 2: RESEARCH INSTITUTIONS’ CAPACITY TO CONDUCT AND DISSEMINATE POLICY RESEARCH ON KEY POLICY AND GOVERNANCE ISSUES STRENGTHENED PIR2.a. Level of selected MP or Parliament staff satisfaction with policy research Percent of surveyed MPs/ Parliament staff or national/local government representatives who report they are “satisfied” or “very satisfied” with at least one product of at least one ProRep-supported research institution. Respondent ratings will be counted only when the respondent can identify or recall a particular research product or study. [See KRA 2.3.a for specification of universe of survey]. G Gender Survey of MP and MP staff satisfaction with Qstar and long term grantee products 1 time during the life of project. Zero baseline Yrs 1-5: 30% Data collection ongoing It proved more practical and cost effective to conduct this survey one time during the project period. On an ongoing basis, grantees provide ProRep with “response sheets” documenting policy makers response to research provided. The definition is revised to adjust to the new policy cluster approach, which targets not solely the parliament member and staff but also national and local government representatives. KRA 2.1: Research Institutions’ Effectiveness and Policy Research Capabilities Improved KRA 2.1.a. Number of organizations in key areas achieve organizational improvements Number of organizations with qualitative improvement in development (including policy research capabilities), operations and financial management capacity Number of Organization Organizational Effectiveness Assessment and Grantee reports documenting new organizational improvements Annually Zero baseline Informed by Component 2 assessment Yr 1: 3 Yr 2: 6 Yr 3: 5 Yr 4: 5 Yr 5: 3 Yr 1 : 0 of 3 Yr 2 : 3 of 6 Yr 3 : 5 of 5 Yr 4: 5 of 5 The policy cluster focus for ProRep’s Years 4 and 5 will result in a decreased focus on organizational capacity building and a greater emphasis on policy change PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 29 KRA 2.2: Quality, Volume and Dissemination of Research Institutions’ Policy Research Increased KRA 2.2.a. Number of copies of written research products disseminated to Parliament members and policy makers Number of copies (in hard￾copy or electronic format) of research study products disseminated from ProRep partner research institutions to Parliament members or their staff, and representatives from government – including national and local government representatives. (a) Policy theme of document content; (b) Type of product; (c) Institutional source; (d) Recipient (DPR Member or other) DPR research institution partner reports; DPR research institution partner reports; verified by spot￾check survey of a sample of intended recipient Parliament members Annual Zero baseline Informed by Component 2 assessment Yr 1: 8 (Women￾related: 3) Yr 2: 6 (Women￾related: 2) Yr 3: 12 (Women￾related: 4) Yr 4: 6 (Women￾related: 2) Yr. 5 : 12 (Women￾related: 4). Yr 1 : 0 of 8 Yr 2 : 37 of 6 Yr 3 : 29 of 12 Yr 4: TBD Year 2 and 3 achievements exceed targets because some ProRep partners distributed research products to working groups, such as the women’s caucus or through several special committee on particular subjects. New policy cluster approach will focus on parliament AND government representatives either national or local. KRA2.2.b. Number of target Parliament Members and staff who report receipt of written research products and/ or verbal presentation from research institutions. Number of target Parliament members and staff (e.g., members of selected commissions, BAKN, BALEG, etc.) or national and local government representatives who report they have received written research products from ProRep partner research institutions (a) Policy theme of document content; (b) Type of product (full report or other); (c) Recipient (DPR Member or other) ProRep survey of target Parliament members and response sheets from policy makers provided by grantees. Annual Zero baseline Yr 1: 50 Yr 2: 80 Yr 3: 110 Yr 4: 20 Yr. 5: 10 Yr 1: 0 of 50 Yr 2: 37 of 80 Yr 3: 29 of 110 Yr 4: 20 of 20 ProRep partners had difficulty reaching targets because the ProRep program provided their first support to work explicitly with MPs on policy issues. New policy cluster approach will not focus solely on parliament and will involve government representatives either national or local. KRA 2.3: Research Institutions' Research More Relevant to, and Having More Influence on, Legislative Policymaking KRA 2.3.a. Number of Parliament members reporting use of products of Number of selected Parliament members (or their staff) who report that they have used one or more products of at least one ProRep-supported research (a) Policy theme of document content; (b) Type of product (full Grantee reports, surveys of MPs, and response sheets from MPs and staff Yrs 1, 3 and 5 Zero baseline Yr 1: 15 Yr 2: 15 Yr 3: 15 Yr 4: 10 Yr 1 : 0 of 15 Yr 2 : 18 of 15 Yr 3 : 23 of 15 Yr 4: 23 of 10 New policy cluster approach will not focus solely on parliament and will involve government representatives, so targets PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 30 research institutions institution in the last year. Selected Parliament members include members active in issues relevant to research conducted by partner institutions (e.g., members of BALEG, BAKN, etc.). report or other); (c) Recipient (DPR Member or other) Yr 5: 5 for Years 4 & 5 are lower. PIR 3: MORE EFFECTIVE, RESPONSIVE AND TRANSPARENT LEGISLATIVE PROCESSES PIR3.a. Number of pieces of legislation positively affecting democratic governance deliberated and passed Number of Parliamentary bills and amendments deliberated and passed that positively affect democratic governance (for example, bills/amendments on the election law) Bills or amend￾ments deliberated versus passed Parliament records Annual Zero baseline Yr 1: Deliberated:2 Passed: 1 Yr 2: Deliberated:2 Passed: 1 Yr 3: Deliberated: 3 Passed: 1 Yr 4: Deliberated: 1 Passed: 0 Yr 5: Deliberated: 0 Passed: 0 Yr 1: Deliberated:0 Passed: 0 Yr 2: Deliberated:1 Passed: 1 Yr 3: Deliberated: 2 Passed: 1 Yr 4: Deliberated: 1 Passed: 0 Targets for Years 4 and 5 are lower because ProRep has ended its program focus on the parliament. In addition, Installation of a new government and parliament in 2014 is expected to make policy changes more difficult. KRA 3.1: DPR Members More Aware of and Better Able to Represent Constituent Interests KRA3.1.a. Number of consultations with Number of face-to-face meetings, phone calls, or text messages held between ProRep-engaged DPR Gender of MP MP Response sheets/ Interviews Annual Zero baseline Infomred by Yr 1: 100 (By women: 50) Yr 2: 100 Yr 1: 0 of 3 Yr 2: 37 of 100 Yr 3: 161 of 150 ProRep used JABAT reports for an accurate measure of consultations with constituents. Early PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 31 constituents held by ProRep￾engaged DPR members members and their constituents Media coverage JABAT program reports Component 3 assessment (By women: 50) Yr 3: 150 (By women: 75) Yr 4: 0 (By women: 0) Yr 5: 0 (By women: 0). targets were not reached because the JABAT program began during Year 2. Year 3 target was exceeded because of JABAT program success. ProRep program support to Parliament ends in Year 4, so targets are reduced to 0. KRA 3.2: More informed, efficient and responsive lawmaking KRA 3.2.a. Linkages of ProRep￾supported legislative amendments to the amendment process Number of identified paths of influence tracked to legislative amendments. A linkage is counted for each of the following: When a ProRep-proposed amendment or change to budget is discussed in committee; appears on a committee agenda; discussed in committee oversight hearings. None DPR records; reports of ProRep grantees across components Annual Zero baseline Yr 1: 2 Yr 2: 5 Yr 3: 8 Yr 4: 1 Yr 5: 1 Yr 1 : 0 of 2 Yr 2 : 2 of 5 Yr 3 : 1 of 8 ProRep’s Mid-Term Evaluation (MTE) notes that targets were unrealistically high, noting that, “The process of influencing Parliament takes much longer.” Achieving changes in multiple formal laws takes longer than the project period. In Years 4 and 5 ProRep will no longer be providing direct assistance to parliament, and the installation of a new government and parliament in 2014 will mean fewer legislative amendments, so targets are reduced. KRA 3.3: More effective and transparent budgetary analysis and oversight KRA3.3.a. Number of U.S.- assisted actions contributing to Number of DPR actions (including issuing reports) Number of DPR actions (including issuing reports) Gender relevance of actions DPR records, supplement ed by Annual Zero baseline Yr 1: 1 (gender￾relevant: Yr 1 : 0 of 1 Yr 2 : 5 of 2 ProRep support to the BAKN (public accounts committee) led to several oversight actions, so targets PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 32 better budget or program oversight undertaken by the DPR contributing to better budget or program oversight selected ProRep interviews Infomred by Component 3 assessment 0) Yr 2: 2 (gender relevant: 1) Yr 3: 3 (gender relevant: 2) Yr 4: 0 (gender relevant 0) Yr 5: 0 (gender￾relevant: 0) Yr 3 : 9 of 3 for Years 2 and 3 were exceeded. In Years 4 and 5 ProRep will no longer be providing direct assistance to parliament, so targets are reduced to zero. KRA 3.4: DPR Management and Procedural Reforms Adopted and Implemented KRA3.4.a. Number of DPR institutional reforms proposed, passed and implemented. Institutional reforms including Standard Operating Procedures, internal rules and procedure in DPR. Reforms deliberated versus passed DPR records, supplement ed by selected ProRep interviews Annual Zero baseline Infomred by Component 3 assessment Yr 1: Deliberated : 1 Passed : 1 Yr 2: Deliberated: 1 Passed: 1 Yr 3: Deliberated: 2 Passed: 2 Yr 4: Deliberated: 0 Passed: 0 Yr 5: Yr 1: Deliberated : 1 Passed : 1 Yr 2: Deliberated: 2 Passed: 1 Yr 3: Deliberated: 4 Passed: 2 Yr 4: Deliberated: 2 Passed: 0 In Years 4 and 5 ProRep will no longer be providing direct assistance to parliament, so targets are reduced to zero. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 33 Deliberated:0 Passed: 0 KRA 3.5: Policy Makers and Other Actors Collaborate on Key Policy Issues KRA.3.5.a Number of multi￾stakeholder forums convened with ProRep support to discuss key policy issues. Number of forums bringing together at least three of the following stakeholders to discuss issues related to identified policy clusters: CSOs, research organizations, policy makers, constituents. Documentati on of meetings which may include photographs or press reports Annual Zero baseline Yr.4: 6 Yr.5: 14 Yr 4: 4 Yr. 5 TBD New indicator with addition of policy cluster program in 2014 and targets adjusted with with the addition of democratic rights as policy cluster. KRA.3.5.b Number of multi￾stakeholder forums convened with ProRep support that develop formal policy recommendatio ns. Number of multi-stakeholder forums (defined in KRA 3.5.a) that create formal, written recommendations on key policy changes or policies to be addressed. Written policy recommend ation Annual Zero baseline Yr.4: 0 Yr.5: 8 Yr 4: 0 Yr. 5: TBD New indicator with addition of policy cluster program in 2014. KRA.3.5.c Number of multi￾stakeholder forums convened with ProRep support that develop formal action plans for influencing policy. Number of multi-stakeholder forums (defined in KRA 3.5.a) that create formal, written action plans for addressing policy change in key policy cluster areas. Written action plans Annual Zero baseline Yr.4: 0 Yr.5: 5 Yr 4: 0 Yr. 5: TBD New indicator with addition of policy cluster program in 2014 and targets adjusted with with the addition of democratic rights as policy cluster. PROREP PERFORMANCE MANAGEMENT PLAN (PMP) – REVISED JUNE 2015 34 PIR 4: ASSISTANCE FOR SPECIAL INITIATIVES NEEDED TO PROTECT OR ADVANCE DEMOCRATIC GOVERNANCE KRA 4.1. Indonesian Efforts to Address Strategic needs and Opportunities Affecting Democratic Governance Improved KRA.4.1. Number of special initiatives undertaken by USAID through ProRep Number of initiatives undertaken by USAID to respond flexibly and rapidly to address other unanticipated needs and opportunities pertaining to the protection and advancement of democratic governance in Indonesia. a. Project component b. Number of organizations Contractor report Quarterly Zero baseline Yr 1: 4 Yr 2: 5 Yr.3: 4 Yr.4: 5 Yr.5: 5 Yr 1: 0 Yr 2: 10 Yr.3: 22 Yr.4: 9 Several different JABAT activities during Years 2 & 3 contributed to the project exceeding Year 2 and 3 targets. ProRep PMP 35 Appendix A: Summary of Comments from ProRep’s Mid-Term Evaluation In accordance with contract requirements, a mid-term evaluation of ProRep was conducted in April 2013. The objective of the evaluation was to identify which elements of the project were having impacts, which were not, and which aspects of the project design might need to be adjusted. The midterm evaluation provided feedback on M&E indicators and targets. The tables below summarize the evaluators’ comments that were considered when revising the PMP. Table 3a: Summary of comments on Project-level Indicators Indicator Number Original Indicator Notes PO a. Number of laws, legislative amendments or Parliamentary oversight proceedings influenced by CSO advocacy Indicator needs revising. The process of influencing Parliament takes much longer PO b. Number of bills, amendments or laws influenced by research and analysis provided by a supported research institution Indicator needs revising. The process of influencing Parliament takes much longer Table 3b: Summary of comments on Component 1 Civil Society Indicators PIR. I a. Extent of CSOs use of diverse mechanisms to receive information and opinions from their members Indicator needs revising Diverse mechanisms are used to provide information to a wider audience, not all of whom are their members. PIR. 1 b. Percent of CSO members and constituents who feel that their interests have been effectively advanced by the CSO This information was intended to be the result of a survey, but no survey has yet been done KRA 1.1.b Number of CSOs experiencing with significantly expanded membership Indicator needs revising Many CSOs do not have members in a formal sense. They have a combination of alliances to networks, alliances to other CSOs, and beneficiaries KRA 1.2.a Number of ProRep supported CSOs that participate in legislative proceedings Indicator needs revising Making representation to MPs (which is reported by ProRep) is valuable, but is not the same as participating in formal legislative proceedings KRA.1.2.b Number of policy briefs brought to Parliament by CSOs and substantively reflected in responsive legislation, oversight or budget proceedings CSOs have usually only been able to bring issues to the attention of MPs in public discussions KRA 1.3.b Number of times CSO advocacy positions are featured in the media 39 were for AJI. Women related also relates to issues important to women – in this case School Operational Fund Table 3c: Summary of comments on Component 2 Research Institution Indicators Indicator Number Indicator Notes PIR 2.a. PIR2.a. Level of selected MP or Parliament To be collected through a survey with ProRep PMP 36 staff satisfaction with policy research members of BALEG, BANGGAR, and Komisi KRA 2.1.a Number of organizations in key areas achieve organizational improvements Supposed to come from an Organizational Effectiveness Assessment KRA 2.2.a KRA 2.2.a. Number of copies of written research products disseminated to Parliament members and policy makers Through QStaR grants KRA 2.2.b Number of target Parliament Members and staff who report receipt of written research products and/ or verbal presentation from research institutions to be collected through a survey with members of BALEG, BANGGAR, and Komisi KRA 2.3.a Number of Parliament members reporting use of products of research institutions Data was gather from reports by partners and were verified through interviews with selected MPs by ProRep Table 3d: Summary of comments on Component 3 Legislative Bodies Indicators Indicator Number Indicator Notes PIR 3.a. Number of pieces of legislation positively affecting democratic governance deliberated and passed This is MD3 Law; the proposed amendments to the Law from BAKN have been submitted to BALEG KRA 3.1.a Number of consultations with constituents held by ProRep-engaged DPR members This is from JABAT KRA 3.2.a Linkages of ProRep-supported legislative amendments to the amendment process Yr 1: Activities with BALEG (round table meeting for preparation of BALEG workshops on MD3 Law). Yr 2: public hearings on amending MD3 Law in three provinces) KRA.3.3.a Number of U.S.-assisted actions contributing to better budget or program oversight undertaken by the DPR Yr 1: BAKN annual report, investigative report and hearing with ministry of education. Yr 2: hearing in 3 universities on higher education budget; verification of BPK audit findings and recommendations with 5 state-owned companies KRA 3.4.a Number of DPR institutional reforms proposed, passed and implemented This is related to the amendments of MD3 Law ProRep PMP 38 Appendix B: Schedule of M&E Key Activities for Year 5 Activities 2015 2016 Milestones and Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr Deliverables PMP Revision Revised PMP Indicator data collection and verification (using database system) Data verification collected Quarterly Report development Quarterly Report Annual Report development Annual Report Lesson Learned Assessment (In￾depth Interview) Lesson learned (for final report) Final Report development Final Report Reflection and learning workshop with Education Cluster - Success story - Lesson learned Reflection and learning workshop with Health Cluster Reflection and learning workshop with DRG Cluster Reflection and learning workshop with Environment Cluster Reflection and learning workshop with PRN Follow up with former partners to identify success stories (in-depth interview) Success stories Finalize and disseminate Organizational Assessment Survey Report of Organizational Improvement