Final Report Final Assessment—Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice— Phase II Project. 12 August 2019 Bu Wilson, Team Leader/International Consultant Abel Amaral, National Consultant Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. i THIS PAGE IS LEFT BLANK INTENTIONALLY. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. ii Final Report Final Assessment—Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice—Phase II Project. 12 August 2019 Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. iii Acronyms and Abbreviations AES Australian Evaluation Society AJAR Asian Justice and Rights APF Asia Pacific Forum of National Human Rights Commissions CA Capacity Assessment CAC Anti-Corruption Commission (Comissão Anti Corrupção) CFP Civil Service Commission (Comissão da Função Pública) CSO Civil Society Organisation DEC USAID’s Development Experience Clearing House e-CMS Electronic Complaints Management System ESC Economic, Social and Cultural F-FDTL Timor-Leste Defence Force (Falintil-Forsa Defesa de Timor-Leste) G2G Government to Government (Governu ba Governu) GoTL Government of Timor-Leste HAK HAK Association for Law, Human Rights and Justice (Hukum, Hak Asasi dan Keadilan) JSMP Judicial System Monitoring Program KAQ Key Assessment Question KMS Knowledge Management System LGBT Lesbian, Gay, Bisexual and Transgender MOU Memorandum of Understanding NGO Nongovernment Organisation NHRI National Human Rights Institutions NZMFAT New Zealand Ministry of Foreign Affairs and Trade OECD Organization for Economic Cooperation and Development OHCHR Office of the High Commissioner for Human Rights PDHJ Ombudsman for Human Rights and Justice (Provedoria dos Direitus Humanus no Justisa) PME School Feeding Program (Programa Merenda Eskolar) PNTL National Police of Timor-Leste (Polísia Nasional Timor-Leste) RAEOA Special Administrative Region of Oé-Cusse Ambeno (Rejiaun Administrativa Espesiál Oé-Cusse Ambeno) RDTL Democratic Republic of Timor-Leste (República Democrática Timor￾Leste) RMT Results Management Table SEANF South East Asian National Human Rights Institutions Forum SMR Standard Minimum Rules SOP Standard Operating Procedures SOW Statement of Work UN United Nations UNDP United Nations Development Program Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. iv Table of Contents ACRONYMS AND ABBREVIATIONS.............................................................................................III TABLE OF CONTENTS......................................................................................................................IV 1. EXECUTIVE SUMMARY .............................................................................................................1 2. BACKGROUND .............................................................................................................................6 3. FINAL ASSESSMENT PURPOSE...............................................................................................7 4. KEY ASSESSMENT QUESTIONS (KAQ).................................................................................8 5. ASSESSMENT METHODOLOGY ..............................................................................................8 6. LIMITATIONS............................................................................................................................10 7. FINDINGS....................................................................................................................................10 7.1 HUMAN AND INSTITUTIONAL CAPACITY .....................................................................................10 7.2 PUBLIC KNOWLEDGE, MONITORING AND INVESTIGATION........................................................19 7.3 ADVOCACY FOR POLICY CHANGE ..................................................................................................26 7.4 GOTL COMMITMENT .....................................................................................................................26 7.5 INFLUENCE AND RELATIONSHIP WITH OTHER GOVERNMENT INSTITUTIONS...........................27 8. CONCLUSION.............................................................................................................................28 9. RECOMMENDATIONS.............................................................................................................28 10. ANNEXES ..................................................................................................................................30 ANNEX 1 STATEMENT OF WORK (SOW)...............................................................................................I ANNEX 2 LIST OF CONSULTATIONS ...................................................................................................VIII ANNEX 3 REFERENCES ............................................................................................................................X ANNEX 4 SIGNED DISCLOSURES OF INTEREST ................................................................................. XIV ANNEX 5 DATA COLLECTION AND ANALYSIS TOOLS........................................................................XVI ANNEX 6 ASSESSMENT TEAM ..........................................................................................................XXIV Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 1 1. Executive Summary This is a final assessment of assistance provided to the “Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice (PDHJ) Phase II Project” in Timor-Leste. The assistance took the form of Government to Government (G2G) assistance provided over a period of 36 months for the sum of $523, 443 through United States Agency for International Development (USAID) (38%) and New Zealand Ministry of Foreign Affairs and Trade (NZMFAT) (62%). The project commenced in September 2015 and was due to finish in September 2018, but a no-cost extension saw the project extended to January 2019. The goal of the G2G assistance is to improve PDHJ’s ability to advocate for human rights and good governance by increasing community uptake and benefit from PDHJ services and improving the performance of the PDHJ. Specifically, this assistance seeks to (1) build the PDHJ’s ability to increase public knowledge of human rights, good governance, and the PDHJ mandate; (2) enhance the capacity of PDHJ staff, particularly in its four regional offices, to monitor and investigate good governance and human rights issues across the country, and; (3) improve PDHJ’s overall capacity and business processes. The PDHJ is an independent constitutional authority with a mandate to promote and protect human rights and good governance in Timor-Leste, serving as a National Human Rights Institution (NHRI) as well as a traditional Ombudsman. It reports to the National Parliament on its activities. The PDHJ has been operating in Timor-Leste since March 2006. The PDHJ has a central office in Dili and four regional offices in Oecusse, Baucau, Maliana (Bobonaro Municipality) and Same (Manufahi Municipality). The PDHJ deals with complaints and cases of human rights and good governance violations, implements education and promotion activities, and monitoring of human rights and good governance issues. PDHJ is mandated to integrate human rights and good governance principles into the legislation and policy of Timor-Leste. In addition to providing recommendations to promote accountability, the PDHJ also recommends how to strengthen systems and create an environment which will prevent future violations. PDHJ is one of a suite of oversight institutions in Timor-Leste which support more transparent and accountable governance, and in doing so also support Sustainable Development Goal 16 “building accountable institutions”. This assessment is intended to identify the short- and long-term effectiveness of USAID and NZMFAT support for the Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project for stakeholders from USAID Timor-Leste, NZMFAT and the Government of Timor-Leste (GoTL). This will include, but not be limited to, the impact of this assistance for the PDHJ to advocate for policy changes in Timor-Leste; GoTL’s expected long-term commitment to PDHJ; the PDHJ’s influence to and relationship with other government institutions; and lessons learned for any future assistance by USAID, NZMFAT, or other donors. The assessment will complement the recent NZMFAT Evaluation of New Zealand’s Country Programme in Timor-Leste Parts 1-4, has been presented to in-country stakeholders and the report will be posted to USAID’s Development Experience Clearing House (DEC). In answering the five assessment questions the team will utilise Organization for Economic Cooperation and Development (OECD) criteria. For the purposes of this assessment, the OECD definitions of relevance, effectiveness, efficiency, impact and sustainability are utilised. The assessment has applied a gender lens throughout the assessment, noting where outcomes or benefits of the assistance may be different for women and men, if there are any unexpected outcomes that may be different for women and men, and to what extent PDHJ advocated on issues of particular relevance to women with government institutions. The assessment was framed around five key assessment questions (KAQ) contained in the Statement of Work (SOW): 1. What were the key outcomes or substantive changes, positive or negative, to the human and institutional capacity of the PDHJ especially at the regional offices? Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 2 2. To what extent has the assistance achieved its goals to build the regional offices’ ability to increase public knowledge of human rights, good governance, and the PDHJ mandate, and to monitor and investigate good governance and human rights issues? What are the key factors that support and/or constrain the achievements? 3. To what extent has this assistance influenced PDHJ’s ability to advocate for policy changes within the GoTL? 4. To what extent has this assistance influenced the GoTL’s expected long-term commitment to PDHJ demonstrated by the current and future budgets allocation to PDHJ? How and to what extent can PDHJ sustain activities without this assistance in the future? 5. To what extent has this assistance impacted the PDHJ’s influence on and relationship with other government institutions? How have the other government institutions received and acted upon PDHJ findings and recommendations? In addition to the KAQ the assessment team utilised the 2015 Capacity Assessment (CA) of PDHJ as a partial baseline to enable assessment of changes in human and institutional capacity over the period of the G2G assistance. This CA was requested by the Provedor and was conducted in January 2015 and the final report was provided in June 2015, three months before the commencement of the G2G assistance. The methodology for this evaluation was based upon document review, fieldwork consultations, and stakeholder feedback. A mixed-method approach was employed, as it is best suited to meeting the objectives of the SOW. Verification and triangulation of evaluation findings occurred through data derived from a variety of sources, the use of multiple perspectives to interpret the data, and the use of a team of assessors, each of whom comes to law and justice development from a unique standpoint. Document review was conducted between 4—10 June, 2019, an in-brief was held with USAID in Dili on 10 June 2019 and an evaluation plan was provided on the same day. Fieldwork was conducted between 10—29 June in Dili, as well as in the regions of Baucau and Same, including also the municipalities of Manatuto and Ainaro. The purpose of the fieldwork was to ask the evaluation questions and compile information and evidence to inform the assessment. The five main assessment questions were used to derive sub-questions and check lists for semi-structured interviews and some limited small group discussions. Questions were reviewed and adapted as required through the fieldwork process. A process of purposive sampling was used in order to cover relevant stakeholders including PDHJ staff at national and regional level, former PDHJ staff, Office of the High Commissioner for Human Rights (OHCHR), donors, relevant state institutions at national and sub-national level, and civil society organizations at national and sub-national level. Interviews were conducted in Tetum, English and Bahasa Indonesia. The assessment team interviewed a total of 55 people. Of these, 36% were women, reflecting that fewer women hold positions in state institutions including the Polísia Nasional Timor￾Leste (PNTL). The team stayed in close contact with USAID through the course of the assessment in order to update them on challenges faced with the assessment, clarify issues and seek assistance in obtaining additional information and access to respondents. The team held daily debriefs and analysis was undertaken on an ongoing basis. At the end of the fieldwork three presentation/debriefs were conducted for USAID, NZMFAT and PDHJ respectively on 26, 27 and 28 June 2019. This provided an opportunity for the assessment team to receive useful feedback and further refine their analysis. The first draft of the written report was presented to USAID for review and comment on 10 July 2019, followed by a second draft on 26 July 2019 and a final report on 12 August 2019. The first draft of the report was reviewed by staff from Hau Meni & Associates and Fundasaun Mahein as part of a quality control process. The International consultant is a member of the Australian Evaluation Society (AES) and, as a member, agrees to abide by the AES Code of Ethics and the Guidelines on Ethical Conduct of Evaluation (Revised July 2013). These Guidelines and Code of Ethics were adhered to during the evaluation. The evaluation team was comprised of: • Dr Bu Wilson, Director/Principal Consultant with Hau Meni & Associates Pty Ltd, an Australian limited liability company specialising in design, evaluation and research. She has a PhD in law, Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 3 extensive experience in Timor-Leste, and subject matter expertise in evaluation, governance, law and justice sector development and ending violence against women; and • Mr Abel Amaral, Interim Deputy Director with Fundasaun Mahein, a well-established Timor￾Leste NGO specializing in monitoring the development of the security sector. He has a law degree and extensive experience in human rights, governance and security sector monitoring and reform. As with any study of this nature there were several limitations. These included significant delays in interviewing the Provedor, which additionally meant other national level PDHJ staff were not interviewed until after the first two debriefs had been completed; significant delays in obtaining agreement to be interviewed from some state institutions; inability to meet with all selected state institutions (despite concerted efforts), possible selection bias due to a purposive sampling process;1 possible response biases (inherent in all self-reporting methods including interviews, focus group discussions and surveys); possible measurement error due to respondents not understanding what was being asked, failing to remember relevant information or not giving an accurate response; and limited participation of women, reflecting the much greater number of men employed in PNTL and civilian administration. Findings The project was implemented at a time of considerable political and budgetary uncertainty and this impacted adversely on implementation. Regional offices have been consolidated with increased investigation capacity and human resources as a result of the G2G project, although this is not yet optimal. While the Knowledge Management System (KMS) is reported as being designed and implemented it is not accessible on a regular basis at the regional offices. This is due in part to an unreliable internet connection and a shortage of qualified IT staff. While training on various aspects of the KMS including the electronic Case Management System (e-CMS) appears to have gone well during the project, it is not possible to put this into practice, particularly in the regional offices, without a better functioning system. Telephone and transport resources in the regional offices are currently inadequate to carry out the mandate of the PDHJ, although the initiative of the current Provedor to address IT and telephone communications issues is to be commended. While a range of regular meetings now occurs within the PDHJ, not all regional offices are conducting staff meetings, and there is potential to improve communications between regional offices. The PDHJ does not yet have a gender policy and strategy which would facilitate mainstreaming of the perspectives and experiences of women in human rights and good governance throughout the PDHJ’s work. However, the initiative of the current Provedor to develop such a policy and strategy, with assistance from UN Women is to be commended. Training as part of the G2G project does not appear to have been accessed proportionally by women staff in PDHJ, for a variety of reasons including that training was focused on the regional offices and investigators, all of which already had more male than female staff; and reportedly there was a reluctance of existing Dili-based female staff to be deployed to regional offices for family reasons. While the recent appointment (2019) of a legal advisor within the PDHJ will go some way to providing the legal expertise required for the PDHJ to fulfil their mandate and demonstrate their independence, it does not fulfil the requirements of Article 13 of Decree Law 25/2011. This Article establishes a Legal Counsel Office with wide ranging responsibilities including “conducting technical analysis for monitoring and verifying the compatibility of any law, regulation, administrative decision, policy in force or any legislative proposal with customary International Law and the international human rights treaties in force”. The staffing of regional offices does not currently appear to be adequate, although the current Provedor is to be commended for addressing this issue. PDHJ met or exceeded its G2G project targets for the percentage of people in targeted communities who agree that PDHJ services help to reduce human rights violations and maladministration in Timor￾1 Purposive sampling (also known as judgment, selective or subjective sampling) is a sampling technique in which a researcher relies on his or her own judgment when choosing members of a population to participate in the study. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 4 Leste; the percentage of people in targeted communities who are aware of basic human rights issues; and the percentage of PDHJ regional staff who have good awareness of human rights and governance issues which is a prerequisite for conducting public outreach activities. Over the life of the G2G project there was a significant increase in the percentage of people who had heard of the PDHJ, and a significant increase in the percentage of people who were aware of the PDHJ regional offices. There was also a consistently high percentage of people who could correctly identify the mandated activities of the PDHJ and a significant decrease in people wrongly believing that PDHJ still has an anti-corruption mandate. PDHJ met or exceeded its G2G project targets for the number of PDHJ staff capable of conducting monitoring and investigations on human rights and good governance issues affecting communities, and the number of monitoring reports produced and issued. PDHJ met the targets on production and issuance of monitoring reports on detention facilities, prisons and the school feeding program. The quality of recommendations in some reports would benefit from better focus and targeting. PDHJ reports meeting or exceeding targets on implementation of recommendations from monitoring but no information is provided to substantiate this. Monitoring and/or annual reports would benefit from providing details of uptake of recommendations from monitoring. A review of the effectiveness of all aspects of the complaints mechanism would be beneficial in addressing lack of trust in the system. Collaboration between civil society and PDHJ would greatly increase the effectiveness and reach of PDHJ’s mandate, although this currently occurs on a largely ad hoc basis. While PDHJ has had some important wins in advocating on specific cases, it has to date had less success in advocacy for more systemic or policy level change. In particular, provided there is adequate legal capacity present, PDHJ is well placed to more actively and publicly review and recommend amendments to existing and draft legislation and other regulations, comment on government policy relevant to human rights, and consider holding inquiries into specific economic, cultural and social rights. While state institutions are surprisingly positive about PDHJ’s role this has not translated into sufficient state budgetary allocations for PDHJ to fulfil its mandate. The G2G assistance has increased the contact between PDHJ and other government institutions, particularly as a result of monitoring activities. This is especially notable in relation to the regional offices. The current Provedor is very active in renewing or concluding Memorandums of Understanding (MoU) with other GoTL institutions but it is too early to say if this will translate into tangible outcomes. These MoUs should be made public. The nature of PDHJ reporting makes it difficult to ascertain what recommendations have and have not been acted upon by state institutions. Recommendations For donors 1. Donors may wish to consider funding specific aspects of support to the PDHJ which would capitalise on, and consolidate, the achievements already realised during the G2G assistance. This could include support to comprehensively reviewing the effectiveness of the complaint mechanism, ensuring that IT functionality is established and maintained including through the development of local staff capacity, and/or supporting the establishment of a more effective and comprehensive legal support unit. For Government and the National Parliament 2. GoTL and the national parliament should ensure that PDHJ receives adequate funds to perform all the functions set out in its mandate. 3. Parliament should more actively engage with PDHJ, in particular through the annual reporting process and the parliamentary committees in order to make government more accountable and ensure that PDHJ’s recommendations are properly considered by government authorities. For relevant state authorities 4. Relevant state authorities should be proactive in seeking engagement with PDHJ, ensuring that their own staff are aware of their rights and obligations, and responding in a timely manner to PDHJ recommendations. For relevant CSOs 5. Relevant CSOS should consider in what ways they can support a better human rights culture by partnering with or supporting PDHJ for improved advocacy, outreach, monitoring, research, and better understanding of the complaints' mechanism. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 5 For PDHJ 6. The PDHJ should consider the benefits of ensuring that the IT environment within PDHJ allows for all staff to access the internet portal and associated KMS, e-CMS, internet and email. 7. PDHJ should pursue the updating and maintenance of a professional website. 8. PDHJ should consider the benefits of improved telephone and vehicle access for staff within the available budget. 9. PDHJ should ensure increased effectiveness of regional offices through promotion of good communications, teamwork and collaboration across functions. 10. PDHJ should pursue its plans to develop and implement a gender policy and strategy that will encompass both the internal functioning of the organization, and its external relations. 11. PDHJ should pursue the establishment of a fully staffed legal advice unit to enable it to fulfil its mandate. 12. PDHJ may wish to consider options for increasing its coverage and dosage for outreach and monitoring activities, within its available budget. This may include an increased emphasis on mass media for outreach and an increased emphasis on collaboration with civil society for monitoring. 13. PDHJ should review the effectiveness and functioning of the current complaints’ mechanism, with the intention of making it simple, accessible, speedy and transparent. 14. PDHJ should more actively and publicly review and recommend amendments to existing and draft legislation and other regulations according to international human rights standards, comment on government policy relevant to human rights, and consider holding inquiries into specific economic, cultural and social rights. 15. PDHJ should explore how it can be more accessible to the public, particularly in cases of emergency. 16. PDHJ should consider ways to make recommendations in reports better focused and targeted, setup a regular timetable for following up, and report regularly and publicly on implementation of recommendations. 17. PDHJ should consider development of a strategy for collaborating and partnering more fully with civil society, in order to improve advocacy, outreach, monitoring, research, and better understanding of the complaints' mechanism. 18. PDHJ should consider development of a strategy for collaborating and partnering more fully with prioritised state institutions. 19. PDHJ should make its MoUs with state institutions and civil society public. 20. PDHJ should continue to promote its achievements and advocate with GoTL for improved and adequate funding. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 6 2. Background This is a final assessment of assistance provided to the “Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice (PDHJ) Phase II Project” in Timor￾Leste. The assistance took the form of G2G assistance provided over a period of 36 months for the sum of $523, 443 through USAID (38%) and New Zealand Aid (62%). The project commenced in September 2015 and was due to finish in September 2018. A no-cost extension saw the project extended to January 2019. Under the VII Constitutional Government there was an extended period of budgetary uncertainty including failure to pass the 2017 program and budget; and from January 2018 a limited operation under the duodecimal system, with one twelfth of the previous year’s budget allocated monthly. This affected service delivery, policy development and policy implementation. The G2G project is a follow-on activity to previous USAID assistance from 2013. New Zealand Aid has also previously supported PDHJ through the United Nations Development Program (UNDP)/OHCHR Project that ran from 2011 till 2014, with a three-month extension into 2015, with an assessment conducted in 2013 (See Kluyskens et al, 2013). The goal of the G2G assistance is to improve PDHJ’s ability to advocate for human rights and good governance by increasing community uptake and benefit from PDHJ services and improving the performance of the PDHJ. Specifically, this assistance seeks to (1) build the PDHJ’s ability to increase public knowledge of human rights, good governance, and the PDHJ mandate; (2) enhance the capacity of PDHJ staff, particularly in its four regional offices, to monitor and investigate good governance and human rights issues across the country, and; (3) improve PDHJ’s overall capacity and business processes. The PDHJ is an independent constitutional authority with a mandate to promote and protect human rights and good governance in Timor-Leste. Until 2009 PDHJ also had an anti￾corruption mandate which was repealed following the establishment of an independent Anti￾Corruption Commission. PDHJ serves as a National Human Rights Institution (NHRI) as well as a traditional Ombudsman. It reports to the National Parliament on its activities. The PDHJ has been operating in Timor-Leste since March 2006. The PDHJ has a central office in Dili and four regional offices in Oecusse, Baucau, Maliana (Bobonaro Municipality) and Same (Manufahi Municipality). The PDHJ is a full member of the Asian Ombudsman Association and the International Ombudsman Institute. The PDHJ deals with complaints and cases of human rights and good governance violations, implements education and promotion activities, and monitoring of human rights and good governance issues. PDHJ is mandated to integrate human rights and good governance principles into the legislation and policy of Timor-Leste. In addition to providing recommendations to promote accountability, the PDHJ also recommends how to strengthen systems and create an environment which will prevent future violations. PDHJ’s Strategic Plan 2011 – 2020 priorities include ensuring state compliance with human rights and good governance principles, ensuring that PDHJ has the capacity and independence to guarantee the implementation of its mandate, protecting vulnerable groups against human rights abuses, and ensuring equitable access to public services. Section 27 of the 2002 Constitution of Timor-Leste established the Ombudsman for Human Rights and Justice as an independent organ. The Statutes of the PDHJ were approved through Law 7/2004, amended by Law 8/2009 and the Organic Structure of the PDHJ was approved through Decree Law 25/2011. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 7 PDHJ is one of a suite of oversight institutions in Timor-Leste which support more transparent and accountable governance, and in doing so also support Sustainable Development Goal 16 “building accountable institutions”. In addition to PDHJ other oversight institutions include the Commission on Anti-Corruption (CAC), the Civil Service Commission (CFP), a suite of Parliamentary Committees, the Inspector General’s office, the Prosecutor General’s office, the Court of Appeal and Audit and the National Procurement Commission. Paper 9: Human Rights which forms part of the 2017 NZMFAT Evaluation of New Zealand’s Country Programme in Timor-Leste notes that there has been recent solid progress by PDHJ against its objectives including significant public awareness events in the municipalities; monitoring of key institutions including prison and schools; strengthening of PDHJ’s internal capacity and knowledge management system; and improved capacity of PDHJ regional staff to monitor, assess and report on human rights and good governance—as well as conduct engagement with communities. However, this assessment also notes challenges for PDHJ including reduced budget, donor dependence, dwindling political support, some cultural resistance to PDHJ’s independence and procedures, and concerns about reporting to PDHJ for fear of retribution. 3. Final Assessment Purpose This assessment will identify the short- and long-term effectiveness of USAID and MFAT support for the Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project for stakeholders from USAID Timor-Leste, NZMFAT and the GoTL. This will include, but not be limited to, the impact of this assistance for the PDHJ to advocate for policy changes in Timor-Leste; GoTL’s expected long-term commitment to PDHJ; the PDHJ’s influence to and relationship with other government institutions; and lessons learned for any future assistance by USAID, NZMFAT, or other donors. The assessment will complement the recent NZMFAT Evaluation of New Zealand’s Country Programme in Timor-Leste Parts 1-4, has been presented to in-country stakeholders and the report will be posted to USAID’s DEC. In answering the five assessment questions the team will utilise OECD criteria, noting that the assessment questions in the SOW have a focus on effectiveness, with a lesser focus on impact and sustainability. For the purposes of this assessment, the OECD criteria are defined as follows: • relevance is the extent to which the objectives of a development intervention are consistent with beneficiaries’ requirement, country needs, global priorities and partners’ and donors’ policies; • effectiveness is the extent to which the desired programmatic outcomes are achieved or expected to be achieved from delivered outputs; • efficiency refers to how resources (funds, expertise, time, etc.) have been used and translated into outputs; • sustainability is the probability that the changes attributable to programmatic activities persist after donor assistance has ended; and • Impact refers to the primary and secondary long-term effects produced by a development intervention (OECD 1991, 2000). The assessment has applied a gender lens throughout the assessment, noting where outcomes or benefits of the assistance may be different for women and men, if there are any unexpected outcomes that may be different for women and men, and to what extent PDHJ advocated on issues of particular relevance to women with government institutions. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 8 4. Key Assessment Questions (KAQ) This assessment was framed around five key assessment questions contained in the SOW: 1. What were the key outcomes or substantive changes, positive or negative, to the human and institutional capacity of the PDHJ especially at the regional offices? 2. To what extent has the assistance achieved its goals to build the regional offices’ ability to increase public knowledge of human rights, good governance, and the PDHJ mandate, and to monitor and investigate good governance and human rights issues? What are the key factors that support and/or constrain the achievements? 3. To what extent has this assistance influenced PDHJ’s ability to advocate for policy changes within the GoTL? 4. To what extent has this assistance influenced the GoTL’s expected long-term commitment to PDHJ demonstrated by the current and future budgets allocation to PDHJ? How and to what extent can PDHJ sustain activities without this assistance in the future? 5. To what extent has this assistance impacted the PDHJ’s influence on and relationship with other government institutions? How have the other government institutions received and acted upon PDHJ findings and recommendations? In addition to the KAQ the assessment team used, where possible, the 2015 CA of PDHJ as a partial baseline to enable assessment of changes in human and institutional capacity over the period of the G2G assistance. This CA was requested by the Provedor and was conducted in January 2015 and the final report was provided in June 2015, three months before the commencement of the G2G assistance. The CA was conducted by PDHJ itself, the Asia Pacific Forum of National Human Rights Institutions (APF), UNDP Asia Pacific Regional Hub (APRH) and OHCHR. The process identified five issues with the largest gaps in capacity: • the capacity to ensure effective and efficient internal communications and coordination within PDHJ • the capacity to ensure that the perspectives and experiences of women in human rights and good governance are mainstreamed throughout PDHJ’s work • the capacity of the leadership to build and lead PDHJ as a strong independent institution for human rights and good governance • the capacity to apply legal expertise in human rights and good governance in PDHJ’s work; and • the capacity of PDHJ’s leadership to build the quality and effectiveness of PDHJ staff. 5. Assessment Methodology The methodology for this evaluation was based upon three processes: (a) document review and desk analysis; (b) fieldwork consultations; and c) stakeholder feedback. A mixed-method approach was employed, as it is best suited to meeting the objectives of the SOW. Verification and triangulation of evaluation findings occurred through: • data derived from a variety of sources; • the use of multiple perspectives to interpret the data; and • the use of a team of assessors, each of whom comes to law and justice development from a unique standpoint (see Roles and Responsibilities of Team Members below). Using a mixed-method approach and multiple data sources permits the use of various analysis techniques, including induction (discovery of patterns); deduction (testing of theories); and Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 9 abduction (uncovering explanations), the combination of which will produce more valid and reliable assessment findings. The assessment team reviewed the documents provided by USAID and PDHJ, and sought out other relevant documents related to the planning and performance of PDHJ. These documents included: • G2G project design and progress reports; • PDHJ strategic plan, monitoring reports and annual reports to the National Parliament; • GoTL Budget Books; • previous independent assessments and reviews; • news items; • financial and budgetary statements; and • academic and scholarly articles and studies. These documents were analyzed between 4—10 June, 2019 in order to learn directly about the work of PDHJ and the context in which it operated; informed directions of inquiry to pursue further; established linkages to other sources of information; and enabled verification and triangulation of data. This complemented the in-brief with USAID/Timor-Leste’s Contracting Officer’s Representative (COR) and other USAID/Timor-Leste staff. An Evaluation Plan was provided to USAID on 10 June 2019. Fieldwork was conducted between 10—29 June in Dili, as well as in the municipalities of Baucau and Same. These municipalities were chosen in consultation with USAID as these regional offices receive the most cases outside of Dili. Within each region, consultations took place in the municipality that serves as the regional centre, as well as in one other municipality. In the case of Baucau this meant consultations were conducted in both Baucau and Manatuto; and in Manufahi consultations took place in Same and Ainaro. Where possible this occurred down to the administrative post or postu level. The purpose of the fieldwork was to ask the evaluation questions and compile information and evidence to inform the assessment. The five main assessment questions were used to derive sub-questions and check lists for semi-structured interviews and some limited small group discussions. Questions were reviewed and adapted as required through the fieldwork process. A process of purposive sampling was used in order to cover relevant stakeholders including PDHJ staff at national and regional level, former PDHJ staff, OHCHR, donors, relevant state institutions at national and sub-national level, and civil society organizations at national and sub-national level. Interviews were conducted in Tetum, English and Bahasa Indonesia. The assessment team interviewed a total of 55 people. Of these, 36% were women, reflecting that fewer women hold positions in state institutions including the PNTL. 56% of interviews took place in Dili reflecting the national representation of state institutions, donors, and civil society organisations. 16% of interviews took place in Baucau, 13% in Same, and 1% in Manatuto and Ainaro. Two people who are resident overseas were interviewed by phone. The team stayed in close contact with USAID through the course of the assessment in order to update them on challenges faced with the assessment, clarify issues and seek assistance in obtaining additional information and access to respondents. The team held daily debriefs and analysis was undertaken on an ongoing basis. At the end of the fieldwork three presentation/debriefs were conducted for USAID, NZMFAT and PDHJ Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 10 respectively on 26, 27 and 28 June 2019. This provided an opportunity for the assessment team to receive useful feedback and further refine their analysis. The first draft of the written report was presented to USAID for review and comment on 10 July 2019, followed by a second draft on 26 July 2019 and a final report on 12 August 2019. The first draft of the report was reviewed by staff from Hau Meni & Associates and Fundasaun Mahein as part of a quality control process. The International consultant is a member of the AES and, as a member, agrees to abide by the AES Code of Ethics and the Guidelines on Ethical Conduct of Evaluation (Revised July 2013). These Guidelines and Code of Ethics were adhered to during the evaluation. 6. Limitations As with any study of this nature there were several limitations. These included: • significant delays in interviewing the Provedor, which additionally meant other national level PDHJ staff were not interviewed until after the first two debriefs had been completed; • significant delays in obtaining agreement to be interviewed from some state institutions; • inability to meet with all selected state institutions (despite concerted efforts), including Parliamentary committees; • possible selection bias due to a purposive sampling process;2 • possible response biases (inherent in all self-reporting methods including interviews, focus group discussions and surveys); • possible measurement error due to respondents not understanding what was being asked, failing to remember relevant information or not giving an accurate response; and • limited participation of women, reflecting the much greater number of men employed in PNTL and civilian administration 7. Findings 7.1 Human and Institutional Capacity (KAQ 1): What were the key outcomes or substantive changes, positive or negative, to the human and institutional capacity of the PDHJ especially at the regional offices? FINDING 1: The project was implemented at a time of considerable political and budgetary uncertainty and this impacted adversely on implementation. FINDING 2: Regional offices have been consolidated with increased investigation capacity and human resources as a result of the G2G project, although this is not yet optimal. FINDING 3: While the KMS is reported as being designed and implemented it is not accessible on a regular basis at the regional offices. This is due in part to an unreliable internet connection and a shortage of qualified IT staff. FINDING 4: While training on various aspects of the KMS including the e-CMS appears to have gone well during the project, it is not possible to put this into practice, particularly in the regional offices, without a better functioning system. FINDING 5: Telephone and transport resources in the regional offices are currently inadequate to carry out the mandate of the PDHJ. 2 Purposive sampling (also known as judgment, selective or subjective sampling) is a sampling technique in which a researcher relies on his or her own judgment when choosing members of a population to participate in the study. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 11 FINDING 6: The initiative of the current Provedor to address IT and telephone communications issues is to be commended. FINDING 7: While a range of regular meetings now occurs within the PDHJ, not all regional offices are conducting staff meetings, and there is potential to improve communications between regional offices. FINDING 8: The PDHJ does not yet have a gender policy and strategy which would facilitate a mainstreaming of the perspectives and experiences of women in human rights and good governance throughout the PDHJ’s work. However, the initiative of the current Provedor to develop such a policy and strategy, with assistance from UN Women, is to be commended. FINDING 9: Training as part of the G2G project does not appear to have been accessed proportionally by women staff in PDHJ. While this partly reflects the greater number of male staff this needs to be addressed in future so that inequalities within PDHJ are not magnified. FINDING 10: While the recent appointment (2019) of a legal advisor within the PDHJ will go some way to providing the legal expertise required for the PDHJ to fulfil their mandate and demonstrate their independence, it does not fulfil the requirements of Article 13 of Decree Law 25/2011 which establishes a Legal Counsel Office with wide-ranging responsibilities. FINDING 11: The staffing of regional offices does not currently appear to be adequate, although the current Provedor is to be commended for addressing this issue. This project was implemented at a time of considerable political and budgetary uncertainty and this impacted adversely and significantly on the project, by both slowing and restricting implementation. The program logic for the G2G project anticipated that through the implementation of capacity building and business process improvements, in the short term the human and organizational capacity of PDHJ would be improved and that in the medium term this would lead to improved performance of PDHJ. Activities for Output 4 were focused on designing and implementing a knowledge management system, and developing management structures and coordination mechanisms for the regional offices (PDHJ, 2015). The assessment team is of the view that this particular KAQ asks for a broader assessment of key outcomes and substantive changes in the human and institutional capacity of the PDHJ over the project period than is strictly confined to the G2G assistance, and that this is done in order to provide a snapshot in time that can be useful for any future assistance. For this reason, the assessment team has also used some of the priority issues identified in the 2015 CA assessment 3 as a baseline (PDHJ et al 2015). These issues relate substantively to the building of human and institutional capacity as envisaged by KAQ1, including at the regional offices; however, the issues also relate to other KAQ discussed below. These issues focus on internal communications, mainstreaming of the perspectives of women, application of legal expertise, and adequacy of human resources. 4 Where the development of human and institutional capacity relates to public outreach, monitoring, and investigation, this is addressed below in Section 7.2. It is of note that the current Provedor is planning to undertake another Capacity Assessment for the PDHJ in 2020. 3 These priorities were determined by the CA through identifying issues where the gap between the current and required capacity of the organisation was well above other issues identified. 4 Adequacy of human resources is covered by the 2015 CA under “The capacity of the leadership to obtain and manage the human and financial resources to implement the strategic plan and annual plan”. The question of financial resources will be addressed under KAQ4. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 12 This section utilises PDHJ reporting to donors for the G2G Project (Annual and Quarterly), including against the Results Management Table (RMT); PDHJ Annual Reporting to Parliament; and interviews and observations of the assessment team. 7.1.1 Results Management Table PDHJ reported meeting or exceeding targets each year for the following relevant indicators (PDHJ 2019b, 2018b, 2016b): • Knowledge Management System (KMS)5 designed and implemented by 2016; • percentage of staff using KMS on a regular monthly basis; and • number of monthly results-based reports produced by Regional offices. 7.1.2 The capacity to ensure effective and efficient internal communications and coordination within PDHJ This section assesses improvements in the capacity to ensure effective and efficient internal communications and coordination within PDHJ. The 2015 CA was of the view that this would need to include: [a] functioning internet system, with full access and usage across central and regional offices and all units; databases, record and report keeping; appropriately regular and relevant meetings; and follow up and feedback on programs, activities and complaints (PDHJ et al 2015). At the time of the CA in January 2015 these internal communication and coordination systems were described as “totally inadequate”, highlighting that “the internet and telephone network currently installed in the PDHJ was not sufficient to meet the current needs and future plans, within the institution”. The CA went on to note that one consequence of this situation was the very limited use of the Electronic Complaints Management System (e-CMS), in particular in regional offices that at that time had no internet access. Additionally, telephone communications were described as “highly problematic”, with regional offices only having one office mobile—kept by the Director—and no landlines, leaving staff reliant on their own personal mobiles. Finally, the CA identified that staff meetings were not conducted regularly, affecting planning, coordination and information exchange (PDHJ et al, 2015). Issues related to follow up will be covered in Section 7.2 below. The activities of the G2G project sought to address some of these internal communications and coordination issues through hiring an international IT Consultant to design and implement the KMS, design and install an internet portal interface, and provide an extensive suite of training for staff on the e-CMS and use of the KMS. The project also included training for the Regional Directors on the monthly coordination system for Results Based Management (RBM); the quarterly reporting system; and training for management and staff on annual action planning. Reporting on pre- and post-testing associated with these trainings is positive. The assessment team notes that while the KMS is reported as having been designed and implemented (Aryal, 2016; PDHJ, 2019b), it was not functioning on a regular basis at the regional offices, due at least in part to an unreliable internet connection and a shortage of qualified IT staff. This meant that regional offices frequently did not have access to internet, email, the e-CMS, reports, or templates; this reportedly also occurred on occasion at the national office. Consequently, staff cannot put IT-related training into practice, which has significant flow on effects for broader communication and coordination, and impacts the 5 Different project documents ascribe different properties to the KMS. This assessment uses the more comprehensive description implied in the RMT. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 13 ability of the regional offices to fulfil their monitoring, investigation and reporting requirements. Regional offices are still submitting their reports to the national office by travelling to Dili, both due to poor internet capacity and perceived lack of security in sending reports. Moreover, the PDHJ regional offices sometimes submit reports through private email accounts rather than the e-CMS system, as interruptions in the e-CMS system need to be reactivated by IT personnel even when a working internet connection is available. This has significant time and money implications. The poor state of the PDHJ website was raised by a number of respondents, highlighting that it is difficult to navigate, out of date, and does not contain the PDHJ publications that one would expect in order to create a sense of transparency and professionalism. During the period of fieldwork for this report the website was inaccessible on a number of occasions. Telephone communications remain as challenging for the regional offices as when the CA was undertaken. Each Regional office still has only one mobile phone—which is kept with the Director—and no landlines. If the Director is out or busy, the other staff have no access to a work phone and must rely on their personal mobile phones and credit. It is hard to see how it is possible to effectively plan, coordinate, monitor, investigate or deliver outreach without accessto a phone. There are also significant safety implications for staff out of the office, often in remote locations, without access to a mobile phone. When these issues were raised by the assessment team in a meeting with the Provedor she acknowledged the severity of the problem and informed us that she had recently hired two IT consultants (one national and one international) to resolve the significant IT issues within PDHJ; she had also authorised the installation of landlines in the regional offices. These are important and promising initiatives for PDHJ, but it may be worth also considering the costs and benefits of having additional mobile phones, or phone credit allocations, available to staff away from the office. Subsequent discussions with PDHJ’s Executive Secretary indicate that in addition to establishing landlines, there will be an allocation of $50/month per office for telephone costs, but there will be no additional mobile phones provided. The reason given for this is that the Director of each office will always accompany staff to the field. This would seem to be an unusual allocation of staff resourcing. Early annual and quarterly reports noted (PDHJ 2016b, 2016c, 2016d, 2016e) that there were challenges related to coordination between the national office in Dili and the four Regional Offices in Baucau, Maliana, Same and Oecusse, as well as with coordination of different activities within any one regional office due to the need to coordinate training, monitoring and outreach activities. These challenges resulted in delays in project implementation. The first Annual report ascribed coordination and communication issues to still needing to find ways to cooperate between the Director-General, Project Coordinator, National Departments and Regional Offices, as well as to a reorganization within PDHJ which resulted in altered management structures and confusion about individual responsibilities (PDHJ, 2016b). The 3rd quarterly report identified having only one vehicle per regional office as presenting coordination challenges (PDHJ, 2016e) when attempting to implement multiple activities, and this is reportedly still an issue. It appears that there are now a range of staff meetings throughout the organization, and Directors of Regional Offices regularly travel to Dili for meetings. However, it was noted of the two regional offices visited that whereas one office conducts regular staff meetings and encourages interaction across staff functions, it is reported that the other regional office never conducts regular staff meetings and there is little interaction between staff across the Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 14 functions. There appears to be no communication across regional offices by function—which is unfortunate, as the potential for mutual support and learning by monitoring or investigation officers would appear to be substantial. 7.1.3 The capacity to ensure that the perspectives and experiences of women in human rights and good governance are mainstreamed throughout the PDHJ’s work The Statement of Work (SOW) for this assessment (Annex 1) asks the assessment team to examine questions through a gendered lens, including reporting any gender biases improved or worsened via assistance to PDHJ. This section considers the previously identified need for a gender mainstreaming policy and strategy; women’s participation in PDHJ’s staffing; percentage of women accessing PDHJ training; percentage of women complainants; and advocacy on gender-related issues. In 2015 the CA recommended that: The PDHJ should develop and implement a gender mainstreaming policy and strategy, in consultation in particular with women staff of the PDHJ and with civil society organisations specialising in issues concerning the rights of women. The policy and strategy should have internal and external dimensions, dealing with both the participation and roles of women within the PDHJ and the work of the PDHJ for and with women. They should deal with both human rights and good governance issues affecting women. They should respond to the particular perspectives and experiences of women in relation to human rights and good governance (PDHJ et al 2015). The Provedor informed the assessment team that although there is currently no gender mainstreaming policy and strategy within PDHJ she has initiated the process of developing one in conjunction with UN Women. A number of other positive recent initiatives indicate that gender is now “on the agenda”. This includes recent workshops on gender inclusion (APF, 2019) and other initiatives in support of lesbian, gay, bisexual, transgender and intersex (LGBTI) rights (PNTL, 2019). PDHJ’s Strategic Plan 2011—2020 seeks to achieve gender balance within PDHJ staffing by 2020 (PDHJ, 2011). The percentage of female staff in the Provedor provides a useful snapshot of the institution, as well as the foundation for assessing the access of female staff to PDHJ training.6 It would appear that the percentage of female staff in PDHJ in 2019 is approximately 41%, as it was in 2015. Over these years it has ranged between 37% (2016) and 44% (2018) (RDTL 2019, 2018, 2017, 2016, 2015). There were also changes in the percentage of female and male permanent staff at each staffing level between 2015 and 2018, with an increase from zero at levels 4 and 5 in 2015 for women to ten women at level 4 and one at level 5 in 2018, as further illustrated in the figures below (PDHJ 2019a, 2016a). 6 Analysing the PDHJ staffing data between 2015—2018 is not straight forward, as there are a number of inconsistencies in data, both within and between sources. For these percentages Volume 2 of the Budget Books for each year have been used rather than the Annual Reports to Parliament as the former have a more consistent approach to presentation of data. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 15 In 2015 the Provedor was male, one of two Deputy Provedors was female, four of seven National Directors/Coordinators were female and all four regional directors were male. In 2018 the Provedor is female, one of two Deputy Provedors is female, four of eight National Directors/Coordinators are female and one of four regional directors is female. Training for staff as part of the G2G program was principally carried out in Year 1 and to a lesser extent in Year 2. While it is not possible to reconcile the training figures in the quarterly and annual reports,7 the annual reports indicate that in Year 1 training was attended by 70% male staff, and 30% female staff; whereas in Year 2 training was attended by 75% male staff, and 25% female staff (PDHJ 2018b, 2017b, 2017c, 2017d, 2016b, 2016c, 2016d, 2016e). While this does not align with the overall M/F staffing ratios in PDHJ, it may indicate the greater number of male staff occupying roles such as investigators and regional directors at this time, and who were thus prioritised for training. However, it is of concern that the significantly 7 For both Years 1 and Years 2 reporting of the number of staff trained in the first three quarters exceeded that reported annually. 0% 20% 40% 60% 80% 100% Level 1 Level 2 Level 3 Level 4 Level 5 Percentage men and women at each Staffing Level in PDHJ 2015 2015 M 2015 F 0% 20% 40% 60% 80% 100% Level 1 Level 2 Level 3 Level 4 Level 5 Percentage men and women at each Staffing Level in PDHJ 2018 2018 M 2018 F Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 16 greater access of male PDHJ staff to training under this program could potentially exacerbate existing inequities. Although comprehensive data is not available, and data is not always sex-disaggregated, it appears that issues of women’s access to PDHJ training may be broader than that covered just by staff training through the G2G project. By way of example, while it is variously reported that between 9% (APF, 2018) and 13% (PDHJ, 2019a) of participants in PDHJ training for F￾FDTL 8 were women, 30% of F-FDTL are women, indicating that women may not be getting appropriate access to training. It is acknowledged that the organization receiving training may have more control over who attends, but this is still something that PDHJ is able to address. It is reported that in 2018 Members of Parliament also recommended that there needed to be increased participation of women in trainings of public employees on human rights protection and promotion (JSMP, 2018). Early annual and quarterly progress reports (PDHJ 2016b, 2016c) indicated there were challenges for PDHJ staff in ensuring attendance and active participation of women at socialization sessions. With concerted efforts, by the end of Year 3 attendance of women at socialization sessions had increased from 38% to 52% (PDHJ 2019b). This was achieved by meeting with community leaders ahead of time to communicate an expectation of equal numbers of men and women, and by arranging some sessions just for women. Active participation in socialization sessions proved much more difficult to achieve, both for men and women, with little improvement over time, although the disparity in active participation between men and women decreased from 11 percentage points to four percentage points over the course of the project (PDHJ, 2019b). This suggests that overall approaches may need to be reviewed, together with more emphasis on encouraging both women and men’s participation. 8 This training was supported by OHCHR. 62 56 48 38 44 52 0 10 20 30 40 50 60 70 Y1 Y2 Y3 Percentage Male and Female Attendance in Socialisation Sessions M% F% Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 17 During the monitoring of prisons PDHJ carried out advocacy on separating female prisoners from male prisoners, and the need for female prisoners to have access to female doctors and gender specific health services (PDHJ 2016b, 2018b, 2019b). Based on data available in the 2018 Annual Report to the National Parliament it appears that the percentage of complaints from women each year has increased from 20% to 27% between 2016 and 2018 (PDHJ, 2019a). The Perception surveys conducted by PDHJ in 2016, 2018 and 2019 asked respondents if they agreed or disagreed with the statement “La'en sira nunka iha direitu atu baku sira nia fen” (Husbands never have the right to hit their wives). The percentage of respondents agreeing with this statement jumped from 36% in 2016 to 96% in 2019, indicating a decline in acceptance of spousal beating. This is supported by a decline in acceptance of other kinds of violence. While this may indicate changes in cultural attitudes, the phrasing of the question is not optimal as it is both confusing and implies the “correct” answer; and assumes that “hitting” and intimate partner violence are interchangeable concepts. There is also some confusion in how responses are reported between years (PDHJ 2019c, 2018c, 2016c). A figure of 4% of respondents accepting intimate partner violence would seem a rather low figure given the ongoing prevalence of this kind of violence in Timor-Leste (See e.g. The Asia Foundation, 2016). While it is reasonable to conclude that the socialization campaigns of PDHJ may have contributed to attitudinal change, it is not possible to quantify this given the wide range of other campaigns focused on intimate partner violence in Timor-Leste. Nevertheless, this is important messaging for the National Human Rights Institution to maintain in their future work. PDHJ may wish to consider rephrasing this question in line with more contemporary thinking on exploring social norms through surveys if future perception surveys are carried out. 7.1.4 The capacity to apply legal expertise in human rights and good governance in Timor-Leste While the G2G project did not specifically aim to build the specialist legal unit within PDHJ required by Decree Law 25/2011, the development of legal expertise is implicit in the assistance provided through the suite of training activities seeking to improve monitoring and investigation within PDHJ, all of which has a legal basis. Concerns were raised by a number of respondents that the lack of legal expertise within PDHJ has meant that mistakes have been 0 5 10 15 20 Year 1 Year 2 Year 3 Percentage of Men and Women who participated actively in PDHJ socialization sessions M F Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 18 made in the process of monitoring, investigation, and making recommendations to state institutions, with specific examples cited by respondents from PNTL, Ministry of Justice and OHCHR. By way of example one respondent noted that in 2016 they needed to give feedback to PDHJ that they were quoting an UNTAET regulation (23/2001) that had already been repealed and replaced with Decree Law 14/2014. PNTL also provided examples of incorrect use of legislation in framing complaints, claiming that they had raised this with PDHJ. Another respondent noted that the 2016 and 2017 Prison Monitoring reports still quote the above UNTAET regulation when there is no reason to do so. Furthermore, the 2016, 2017 and 2019 prison monitoring reports all incorrectly cite the Standard Minimum Rules for the Treatment of Prisoners of 1955 (SMR). These SMRs were completely revised in 2015 and rebranded as the “Mandela Rules” (United Nations, 2015). This is a lost opportunity for PDHJ to assess how Timor-Leste is complying with the new rules. Concerns were also raised by respondents about the level of legal knowledge amongst education/outreach staff, pointing out the necessity of being able to answer legal queries, particularly when training police and military personnel, and when questions arise related to issues such as use of force and states of emergency/states of siege. A fully resourced legal unit would be in a better position to facilitate improved performance by staff in these roles. Other respondents interviewed by the assessment team were concerned that since its establishment in 2004, the Provedor has not made full use of their constitutional powers to request an abstract review of constitutionality (Article 150 Constitution) or to request the Supreme Court of Justice to review the unconstitutionality by omission of any legislative measures (Article 151 Constitution). This concern has previously been raised by a range of commentators including JSMP (JSMP 2018, 2013), in the Capacity Assessment (Provedor et al 2015), and most recently by Members of Parliament following presentation of the PDHJ 2018 Annual Report (JSMP, 2019). Interviews conducted with the Provedor and Deputy Provedor by JSMP in 2018 suggested that the main reason for the PDHJ’s failure to call for a review of constitutionality was the lack of human resources, in particular the absence of a qualified legal expert on legislative and policy analysis. This echoes the recommendation of the Capacity Assessment in 2015 that: PDHJ should obtain and develop access to high level legal expertise to enable it to interpret and implement its constitutional and legislative responsibilities comprehensively, including through initiating proceedings in the Supreme Court under sections 150 and 151 of the Constitution (Provedor et al 2015). The CA went on to emphasise that this legal expertise for PDHJ is also central to the analysis and resolution of complaints, and ensuring that monitoring and reporting has a firm legal basis in human rights and good governance standards. Elsewhere in the CA report the connection is made that the exercise of its legal powers is an important way that PDHJ can assert and demonstrate its independence (PDHJ et al 2015). While the Provedor advised the assessment team in June 2019 that PDHJ has recently recruited an international legal adviser, she also noted that recruiting and retaining Timorese legal personnel has not been possible as the current salary scale, tied to the Civil Service Commission, is not competitive. She hoped that with amendments to the PDHJ Organic Law currently being scoped, PDHJ would be able to establish its own salary regime, and thus pay enough to recruit and retain suitable personnel. This, of course, assumes that PDHJ is also allocated adequate budget to do this. It is the view of the assessment team that while the Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 19 recruitment of one legal adviser is a good start it will not be adequate to meet the current and future needs of the PDHJ. 7.1.5 Adequate human resources PDHJ’s Strategic Plan 2011-2020 and the staffing chart attached to the Organic Law mandates a staffing complement of 134 persons. The 2018 Annual PDHJ Report to Parliament records 133 staff. The apparent inadequacy of personnel to carry out the mandate of the PDHJ has been raised previously by a number of commentators (JSMP, 2018, 2016, 2013; PDHJ et al 2015; United Nations, 2017), and this was frequently raised by a wide range of respondents both internal and external to PDHJ during the course of this assessment. The G2G project focused on building the capacity of PDHJ staff, rather than on total numbers of staff in the PDHJ. However, the assessment team noted some specific issues related to the adequacy of staff numbers, and the allocation of positions, in the regional offices which is pertinent to this report. It was noted in general that it would be difficult for regional office staff to carry out a sufficiently comprehensive program of outreach, monitoring or investigation with the available human (and transport) resources. It was also noted that one of the Investigators in Same regional office had responsibility not only for investigations but also for Finance, Administration and Outreach. It was the view of the assessment team that this was not a good use of human resources, considering her intensive training as an investigator as part of the G2G project. This issue was raised in a meeting with the Provedor who informed the assessment team that she recognised staffing of the regional offices was problematic, and was in the process of recruiting six additional staff, including two for Same and Maliana, and one each for Baucau and Oecusse. The team believes this is a good development and it now needs to be ensured that there are adequate transport resources for the larger teams, who already find it difficult to manage their responsibilities with only one vehicle per office. It may also be useful to assess whether the mandated number of staff contained in the Organic Law is still sufficient, and whether those staff are allocated in the most effective manner. 7.2 Public Knowledge, Monitoring and Investigation (KAQ2) To what extent has the assistance achieved its goals to build the regional offices’ ability to increase public knowledge of human rights, good governance, and the PDHJ mandate, and to monitor and investigate good governance and human rights issues? What are the key factors that support and/or constrain the achievements? FINDING 12: PDHJ met or exceeded its G2G project targets for the percentage of people in targeted communities who agree that PDHJ services help to reduce human rights violations and maladministration in Timor-Leste; the percentage of people in targeted communities who are aware of basic human rights issues; and the percentage of PDHJ regional staff who have good awareness of human rights and governance issues which is a prerequisite for conducting public outreach activities. FINDING 13: Over the life of the G2G project the PDHJ Perception Surveys indicated there was a significant increase in the percentage of people who had heard of the PDHJ, and a significant increase in the percentage of people who were aware of the PDHJ regional offices. The baseline for people hearing of the PDHJ was 31% in 2014 and 2016, increased to 40% in 2018 and 52% in 2019. There was also a consistently high percentage of people who could correctly identify the mandated activities of the PDHJ and a significant decrease in people wrongly believing that PDHJ still has an anti-corruption mandate. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 20 FINDING 14: PDHJ met or exceeded its G2G project targets for the number of PDHJ staff capable of conducting monitoring and investigations on human rights and good governance issues affecting communities, and the number of monitoring reports produced and issued. FINDING 15: PDHJ met the targets on production and issuance of monitoring reports on detention facilities, prisons and the school feeding program. The quality of recommendations in some reports would benefit from better focus and targeting. FINDING 16: PDHJ reports meeting or exceeding targets on implementation of recommendations from monitoring but no information is provided to substantiate this. Monitoring and/or annual reports would benefit from providing details of uptake of recommendations from monitoring. FINDING 17: A review of the effectiveness of all aspects of the complaints mechanism would be beneficial in addressing lack of trust in the system. FINDING 18: Collaboration between civil society and PDHJ would greatly increase the effectiveness and reach of PDHJs mandate, although this currently occurs on a largely ad hoc basis. The program logic for the G2G project anticipated that through the identification, design and implementation of outreach programs, the development and distribution of promotional resources and materials, and the development and piloting of investigation and monitoring systems in the regional offices, in the short term this would result in increased community awareness of PDHJ’s roles and mandate, increased PDHJ awareness of governance and human rights issues affecting communities, and it would contribute to improved human and organizational capacity of PDHJ. In the medium term this would lead to improved performance of PDHJ, and increased community uptake and benefits from PDHJ services. Activities for Output 1 were focused on designing and implementing outreach programs, including establishing partnerships with relevant local civil society organisations (CSOs). Activities for Output 2 were focused on development and distribution of promotional resources and materials. Activities for Output 3 were focused on developing and piloting investigation and monitoring systems with an emphasis on decentralization of the e-CMS and monitoring to the regional offices (PDHJ, 2015). 7.2.1 Results Management Table PDHJ reported meeting or exceeding targets each year for the following relevant indicators (PDHJ 2019b, 2018b, 2016b): • percentage of people in targeted communities who agree that PDHJ services help to reduce human rights violations and maladministration in Timor-Leste; • percentage of PDHJ’s recommendations from monitoring that are implemented; • number of investigations reports produced by regional staff; • number of monitoring reports produced by staff; • percentage of people in targeted communities who are aware of basic human rights issues; • percentage of PDHJ regional staff who have good awareness of human rights and governance issues; • number of PDHJ staff capable of conducting monitoring and investigations on human rights and good governance issues affecting communities; • number of public outreach events conducted; • number of promotional materials printed and distributed at regional levels • number of films produced and distributed; • number of radio programs produced and broadcast, • number of monitoring activities on human rights and good governance conducted by PDHJ; Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 21 • number of public monitoring reports issued; • number of preliminary assessments carried out by regional offices; and • and number of full investigations conducted by regional offices. Although there was a slow start on the following indicator it was achieved or exceeded by the end of the project: • number of participants at public outreach events. This was 96% achieved in Year 1, 91% achieved in Year 2, and by Year 3 the total number of participants exceeded the target by 9%. The percentage of female participants at public outreach events increased from 39% in Year 1, to 44% in Year 2, and met the target of 50% by Year 3. PDHJ only partially achieved the following indicators: • number of human rights and good governance complaints received by regional offices - between 56-60% of the target was achieved over the three years and the 2014 baseline remained higher than for any year of the G2G assistance; • number of complaints submitted to PDHJ during socialization sessions – 60% of the target was met in Year 1 but no complaints were collected in this way in Year 2 or Year 3; • percentage of people in targeted communities who have increased knowledge of PDHJ mandate – the targets for Years 1,2, and 3 were 40%, 50% and 60% but 31%, 40% and 50% was achieved respectively. The percentage of women in targeted communities who have increased knowledge of PDHJ mandate (of the total who have increased knowledge) targeted 50% of the total for Years 1, 2, and 3 but only 42%, 47% and 45% was achieved respectively. 7.2.2 Public knowledge of human rights, good governance and the work of PDHJ Over the life of the G2G project PDHJ conducted an extensive public outreach campaign aimed at improving knowledge of the role and mandate of PDHJ through socialization sessions, distribution of promotional materials, and production of film and radio programs. Activities supported by other donors included: • collaboration with Asian Justice and Rights (AJAR) Timor-Leste on producing the 20- episode television series Laloran Justisa with support from the European Union. This was broadcast on national television networks and downloaded from YouTube. In 2018 AJAR and PDHJ also ran workshops together focused on how to use the episodes together to educate in four regions of Timor-Leste, with participation from GoTL police and civil society. (AJAR 2018); • cooperation between PDHJ and AJAR to reunify children taken during the Indonesian occupation; and • production of a human rights training manual and provision of training for F-FDTL with support from OHCHR. The capacity of the PDHJ to undertake activities aimed at increasing public knowledge was improved by targeted trainings for PDHJ staff, with the benefit of this demonstrated in pre￾and post-testing knowledge scores. Further evidence of improved capacity to increase public knowledge of human rights, good governance, and the PDHJ mandate is provided by the PDHJ Perception Surveys conducted in 2016, 2018, and 2019 (PDHJ 2016c, 2018c, 2019c). The Perception Surveys show that over the life of the project there was a continuing increase in the percentage of people who had heard of the PDHJ from 31% in 2014 and 2016, to 40% in 2018, and 52% in 2019, with slightly less women than men who had heard of the PDHJ. Of respondents that had already heard of the PDHJ there was an increase in awareness of the PDHJ regional offices from 70% in 2016 to 84% in 2018 and 85% in 2019. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 22 Sources of information about PDHJ changed over time. The percentage of people who knew about PDHJ as a result of attending a public meeting or workshop declined from 47% in 2016 to 39% in 2019. In 2016 only 32% of people knew about PDHJ through the mass media (combined TV, radio, newspapers). However, knowing about PDHJ from TV had increased by 2018 to 65% and by 2019 to 77%. This increase is likely due to the G2G films on human rights that were broadcast on TV, and used in socialization sessions and PDHJ being the subject of one of the Laloran Justisa episodes, as well as an increasing number of people relying on TV as a source of information more generally.9 Similarly knowing about PDHJ from radio had increased by 2018 to 41% and by 2019 to 48%. This increase is likely due to PDHJ radio broadcasts through community radio (PDHJ 2016c, 2018c, 2019c). Each year respondents who had already heard of the PDHJ were asked to identify the activities they understood to be conducted by PDHJ from a list of 10 activities. Those activities that are actually part of the mandate of PDHJ were identified correctly by the vast majority of those respondents each year (between 90 and 97% of the time). Respondents were also asked three questions about activities that used to be part of the PDHJ mandate up until 2009. This is when the anti-corruption mandate of PDHJ was removed and transferred to the Anti-Corruption Commission (CAC). Although there are still significant percentages of respondents who believe that anti-corruption forms part of PDHJ’s mandate, this misunderstanding has decreased from an average of 76% across the three questions in 2016 to an average of 60% across the three questions in 2019 (PDHJ 2016c, 2018c, 2019c). During the project a decision was made to move the focus of outreach activities away from the community level to a focus on secondary schools. While respondents who had experienced PDHJ outreach activities were positive, they generally felt that such activities should be conducted more frequently. Clearly this would be dependent on adequate human and financial resources within PDHJ but the assessment team was concerned that neither the coverage (reaching enough locations) nor dosage (visiting frequently enough) of outreach activities was optimal for achieving increased knowledge, and attitudinal and behaviour change. One of the ways that both coverage and dosage could be increased is by forming more effective partnerships with civil society organisations as discussed below in Section 7.2.5. The other way that coverage and dosage could be increased, given the increasing importance of radio and television, would be to shift the emphasis of outreach activities more to mass media and less to in-person outreach visits, while ensuring that non￾urban communities are still exposed to the work of PDHJ. As was noted in the 2019 Perception Survey: For future efforts, PDHJ recommends using existing and audio and video material and developing new audio and video material in order to increase outreach through the media. In particular, short, succinct radio and TV spots broadcasted at strategic times could significantly boost recognizability of PDHJ and its mandate (PDHJ, 2019c). 9 This is also indicated in The Asia Foundation’s 2018 Community Police Perception Survey [in press] where it was found that between 2015 and 2018 all respondent groups at the national level relied to an increased extent on television as their primary source of security information, although there were significant differences at the municipality level, with much higher reliance on television in urban areas and much higher reliance on radio in rural areas. Similarly, The Asia Foundation’s 2018 Tatoli! Survey found that 64% of people nominated TV as the source of news and information that they trust the most, followed by radio at 14%. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 23 Finally, reviewing, updating and maintaining the PDHJ website would provide another opportunity to increase understanding of human rights, good governance and the mandate of PDHJ. 7.2.3 Capacity to monitor good governance and human rights issues Monitoring of government policies and programs, and their implementation for adherence to good governance and human rights is central to the mandate of PDHJ. Over the life of the G2G project PDHJ conducted a total of 528 monitoring visits to prisons, detention facilities (police cells) and schools–to monitor implementation of the school feeding program. Every year of the project a report was produced on each of the three focus areas, totalling nine reports over the life of the G2G project. Nine public audiences were conducted for the presentation of the results of monitoring, and monitoring reports were distributed electronically and in printed form. The results of monitoring were also covered in more generic socialization sessions. Unfortunately, the monitoring reports are not currently available on the PDHJ website for a broader audience. It has been reported that they have on occasion been available in the past. In addition to monitoring specific to the G2G program, PDHJ also reported that it carried out a range of other monitoring including in relation to Presidential, Parliamentary and suku elections; the distribution of hand tractors; the distribution of medicines; evictions; reunification of people separated from their families during the Indonesian occupation; construction projects; and submitted reports to UN bodies (PDHJ 2019a, 2018a, 2017a). PDHJ monitoring of PNTL detention facilities During the G2G project PDHJ has monitored PNTL detention facilities in 12 municipalities and RAEOA. The 2016 PDHJ monitoring report identified a number of issues including lack of facilities, inhuman treatment of detainees, absence of a food budget for detainees at the Postu Administrativu level, lack of documentation of detainees, lack of SOPs relating to arrest and detention, lack of sanitation and lack of clean water. General recommendations regarding more rigorous administrative processes, maintenance of cleanliness, and access to clean water were made. Specific recommendations about further training for PNTL on SOPs on arrest and detention, provision of funds for food, rehabilitation of facilities, provision of transport, increased access to medical facilities and access to legal assistance were also made (PDHJ, 2016f). The 2017 report makes the same recommendations as in 2016 (PDHJ, 2017e). The 2018 report makes very similar recommendations to 2016 but directs the recommendations to a broader range of state actors. The reports on monitoring of detention facilities would all benefit from more clearly expressed and actionable recommendations, and provision of information on uptake of recommendations from the previous year. PDHJ monitoring of prisons The 2016 PDHJ monitoring report on prisons makes 13 recommendations related to provision of information to prisoners; separation of male, female and juvenile detainees; overcrowding; cell conditions; adequate provision of food; initiation rituals; adequate time for family visits; telephone access; access to legal assistance; confidentiality; access to education; and opportunities for work. Similar, but fewer, recommendations were made in 2017 with the addition of recommendations about the career regime for prison guards and the need for mattresses in the prison. The 2019 report makes similar recommendation but also additional recommendations on the need for new prison facilities, timing of food, medical assistance for mothers and babies, and physical aggression by guards. The reports on monitoring of prisons would benefit from information on uptake of recommendations from the previous year. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 24 PDHJ monitoring of the School Feeding Program (Programa Merenda Eskolar-PME) The 2016 PDHJ report identifies very extensive problems with implementation of the PME, which it classifies variously as maladministration, abuse of power and illegal actions. It assesses that these problems occurred due to the absence of detailed policies and clear objectives, absence of training and information on the program, poor hygiene, poor facilities, ineffective supervision and documentation, and discrimination between rural and urban schools. Very extensive recommendations are made to a wide range of state institutions including National Parliament Commission C and F, Ministry of Education, Ministry of Health, Ministry of Finance and RAEOA in relation to the slow start of the program, budget, training, supervision, facilities and principles of good governance. Similar recommendations were made in 2017 and 2018. The reports on monitoring of the PME would benefit from information on uptake of recommendations from the previous year. While PDHJ reports meeting or exceeding targets on implementation of recommendations from monitoring, no information is provided to substantiate this. Monitoring and/or annual reports would benefit from providing details of uptake of recommendations from monitoring. A number of respondents from state institutions including PNTL, the Ministry of Justice and the Ministry of Education provided examples of monitoring visits carried out by PDHJ staff, and discussed their attempts to respond to recommendations, but details were often lacking. Respondents also often discussed how lack of budget had prevented them from responding to recommendations. While this is obviously a constraint, there is a role for PDHJ in encouraging state institutions to respond to recommendations that require no or little money to implement, rather than accepting budget as a blanket excuse. 7.2.4 Capacity to provide an effective and efficient complaints service Complaints mechanisms are not only important for addressing individual’s issues but also for allowing NHRIs to “build community support based on their public profile as accessible and accountable institutions, and thus resist budget cuts and other challenges” (Linos and Pegram 2017). Receiving complaints from the public that allege violations of human rights and good governance standards, investigating those complaints against public authorities, and making recommendations is central to the mandate of PDHJ. The G2G project placed considerable emphasis on developing the capacity of regional offices to receive complaints and carry out investigations, with the intention that this would provide improved access to PDHJ for communities. PDHJ reports meeting G2G project targets for the number of preliminary assessments and full investigations carried out by regional offices. However, the number of complaints received falls short of the target, and the receiving of complaints during socialization sessions has not been achieved. During 2018 PDHJ received 203 complaints, which compares with 186 in 2017 and 195 in 2016. In 2018 a total of 32% of complaints came from the regional offices and 68% from Dili. The means of receiving cases was 66% in person, 38% by letter, 1% by phone, 1% through the internet portal and less than 1% through complaints boxes. Women submitted 27% of complaints in 2018, which is an increase on 20% for 2016 and 22% for 2017. Complaints in 2018 were divided between human rights complaints (40%) and good governance complaints (60%). The legal requirement to carry out notifications to complainants at 10 and 45 days was reported as being carried out 70% of the time during 2018. Of the 203 complaints received in 2018, 15 proceeded to investigation stage with a total of 81 recommendations. The largest proportion of recommendations (59%) relate to the PNTL (PDHJ, 2019a). Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 25 A significant number of concerns were raised by a wide variety of the respondents regarding the complaints system during interviews. These related to method of collecting complaints, lack of follow up, perceived lack of effectiveness of the PDHJ complaints system compared with other options such as NGO advocacy, concerns that there were complaints that were considered too sensitive by PDHJ to take on, lengthy delays in processing, a lack of sense of urgency when an immediate intervention is required, recommendations that are not clearly expressed, legally informed, or actionable; and the anecdotally low number of cases undertaken in Timor-Leste compared with other human rights institutions in the region. It became apparent that the extensive system of complaints boxes is not effective for a variety of reasons, evidenced by less than 1% of complaints arriving in this way, and the boxes should probably be removed. Reasons provided in interviews include the boxes public location, the belief that the boxes are not secure, the lack of checking of the boxes, the lack of transport and funds to check the boxes, and the condition of the boxes and locks. Reasons provided for the lack of receipt of complaints during outreach sessions, as anticipated in the G2G design, include the public nature of the forum; and that there has been a move to carrying out socialization with secondary school students who may have less inclination to complain. Of greater concern is the not infrequent claim from respondents that complainants never hear anything from PDHJ after lodging complaints or do not receive updates, despite statutory obligations to do so. These claims include from a Municipal Administrator who lodged a complaint with PDHJ and the HAK Association after alleged violence towards his son by PNTL. He said that while he never heard anything back from PDHJ he is frequently contacted by HAK with updates on the case. A number of people explained that PDHJ has been constrained in responding due to budgetary issues (including citing the period of the duodecimal system), and that people either do not provide contact details or change their phone numbers so it is impossible to do follow up. While not discounting these explanations, it is clear that there are outstanding issues with the complaints system. When some of these issues were raised with the Provedor, she informed the assessment team that she is currently carrying out a review of the complaint boxes. It is the view of the assessment team that there needs to be a review of the effectiveness of the complaints system that is more extensive than a focus on the complaints boxes. 7.2.5 Collaboration with civil society Collaboration with civil society can help NHRIs to protect their independence and pluralism, enhance their effectiveness by deepening their public legitimacy, ensuring they reflect public concerns and priorities, give them access to expertise and valuable social networks, and provide a link with individuals or groups who are politically, socially or economically marginalised (International Council on Human Rights Policy, 2005). The Organic Law and Strategic Plan of PDHJ both anticipate partnerships between civil society and the PDHJ. The Organic Law establishes the advisory council comprising a range of civil society representatives; and there is no doubt that civil society could greatly extend the reach and effectiveness of the mandate of PDHJ to promote human rights and good governance, monitor, and receive complaints; in particular in a resource constrained environment. Civil society can also provide expertise and an entry point into communities. Collaboration with civil society could be achieved through joint programs, activities, and advocacy and through regular information sharing. Respondents internal and external to PDHJ presented a mixed picture of collaboration between civil society and PDHJ, and it appears to the assessment team that this collaboration Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 26 is still on an ad hoc basis and is yet to realise its full potential. Positive accounts of civil society collaboration were provided by a range of individuals and organisational representatives including in the area of land rights, reunification of people taken as children during the Indonesian occupation, and initiatives aimed at combatting gender-based violence. Other civil society representatives were more agnostic, or even hostile, to the PDHJ. In the view of the assessment team this attitude is based partly on a desire for improved collaboration with PDHJ, and partly on unrealistic expectations about what the PDHJ can or should achieve. The assessment team concurs with the recommendation in the 2015 CA that the PDHJ should develop and implement a strategy for relations with civil society involving more substantive collaboration in all of PDHJs functional areas 7.3 Advocacy for policy change (KAQ3) To what extent has this assistance influenced PDHJ’s ability to advocate for policy changes within the GoTL? FINDING 19: While PDHJ has had some important wins in advocating on specific cases, it has to date had less success in advocacy for more systemic or policy level change. PDHJ has made a number of policy recommendations as a result of its monitoring programme on detention facilities, prisons and the PME, and through its thematic reports but it is not clear from its reporting which of these recommendations has been actioned; and there is little focus on policy change in any PDHJ reporting. PDHJ has also collaborated with other institutions to produce guidelines on implementation of policy, and conducted training for state institutions on implementation of policy. Discussions on policy influence of PDHJ with Government respondents tended to highlight operational changes rather than policy changes. More recently the Provedor has concluded a number of MoUs with other state institutions which include a policy element. An example of this is the recent MoU with PNTL which prioritises training for special units, in response to the greater number of human rights violations committed by those units. As discussed below, there is potential for more collaborative relationships between PDHJ and state institutions in order to achieve increased levels of policy influence. Provided there is adequate legal capacity present, PDHJ is well placed to more actively and publicly review and recommend amendments to existing and draft laws and other regulations, comment on government policy relevant to human rights, and consider holding inquiries into specific economic, cultural and social rights.10 7.4 GoTL commitment (KAQ4) To what extent has this assistance influenced the GoTL’s expected long-term commitment to PDHJ demonstrated by the current and future budgets allocation to PDHJ? How and to what extent can PDHJ sustain activities without this assistance in the future? FINDING 20: While state institutions are surprisingly positive about PDHJ’s role this has not translated into sufficient state budgetary allocations for PDHJ to fulfil its mandate. There is currently no donor support for PDHJ. 10 In its General Comment No. 10 (14 December 1998), the UN Committee on Economic, Social and Cultural (ESC) Rights provided an indicative list of activities that National Human Rights Institutions can undertake to protect ESC rights. https://www.un.org/documents/ecosoc/docs/1999/e1999-22.htm Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 27 The Paris Principles state that NHRIs need adequate funding “to be independent of the government and not be subject to financial control which might affect this independence”.11 As noted by the International Council on Human Rights Policy (2005) “financial autonomy [of NHRIs] guarantees the overall freedom to determine their priorities and activities”. The lack of financial support for PDHJ in the state budget is widely acknowledged, and is the basis for a number of UN recommendations to adequately resource the PDHJ. It is the view of the Provedor and other PDHJ staff that continued donor assistance will be necessary for the PDHJ to fulfil its mandate. While the PDHJ successfully advocated for an increase in the 2019 budget, this appears to be a one-off that will enable PDHJ to fulfil a commitment to host a meeting of the South East Asian National Human Rights Institutions Forum (SEANF), and there is no indication it will be repeated. The state budget allocated to the PDHJ between 2015 and 2019, as shown in the Budget Books for each year, is detailed in the table below. Years Total Budget ($”000) 2019 1,968 2018 1,129 2017 1,616 2016 1,479 2015 1,411 There appears to be no current donor funding available to the PDHJ since the finalisation of the G2G assistance. 7.5 Influence and relationship with other government institutions (KAQ 5) To what extent has this assistance impacted the PDHJ’s influence on and relationship with other government institutions? How have the other government institutions received and acted upon PDHJ findings and recommendations? FINDING 21: The G2G assistance has increased the contact between PDHJ and other government institutions, particularly as a result of monitoring activities. This is particularly notable in relation to the regional offices. FINDING 22: The current Provedor is very active in renewing or concluding MoUs with other GoTL institutions but it is too early to say if this will translate into tangible outcomes. FINDING 23: The nature of PDHJ reporting makes it difficult to ascertain what recommendations have and have not been acted upon by state institutions Many of the issues relevant to this question have already been covered while addressing other KAQ. Many state institution respondents cited increased contact between PDHJ and their institution and were keen to discuss their operational responses to PDHJ recommendations and suggestions, felt that PDHJ presence in the regions fulfilled a preventative function, and implied that attitudinal and behavioural change had taken place as a result of PDHJ presence. However, institutions interviewed felt that the PDHJ visits had insufficient coverage or frequency to be optimal. Visits need to be comprehensive and repeated regularly so that progress over time can be more readily assessed. 11 The Paris Principles are the principle source of normative standards for NHRIs. First adopted by NHRIs at an international workshop in Paris in 1991, they were subsequently endorsed by the UN Commission on Human Rights and the General Assembly. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 28 The current Provedor is to be commended for working hard on building relationships and MoUs with other state institutions, although it is too early to say how effective this will be in improving state institution responsiveness to PDHJ findings and recommendations. These MoUs should be made public, and it would be helpful in promoting the effectiveness of PDHJ if more attention was paid to reporting outcomes of findings and recommendations. The 2015 CA highlighted the importance of PDHJ developing partnership relationships with other state institutions and recommended prioritising institutions and concluding MoUs. This is something that is currently underway. The CA also recommended the need to develop and implement a strategy for relations with state institutions, in order to enable more substantive collaboration, including on investigations, monitoring, implementation of PDHJ recommendations, and monitoring and reporting on uptake of recommendations. The assessment team concurs with this CA recommendation. There is an opportunity for Parliament to more actively engage with PDHJ, in particular through the annual reporting process and the parliamentary committees. Annual Reports are an opportunity to make government more accountable, and parliamentarians are a useful way of ensuring that PDHJs recommendations are properly considered by government authorities. 8. Conclusion The G2G assistance, implemented in challenging circumstances of political and budgetary uncertainty, has contributed to the development of the human and institutional capacity of the PDHJ; as well as the ability of the regional offices to increase public knowledge of human rights, good governance, and the PDHJ mandate, and to monitor and investigate good governance and human rights issues. This is however constrained by an unsatisfactory IT environment, limited telephone and transport resources, and insufficient legal resources to fulfil the PDHJ mandate. While PDHJ has some successes in resolving individual cases, it is difficult to discern a significant improvement in the ability to advocate for policy changes within the GoTL. While GoTL agencies appreciate the work of PDHJ this has not significantly translated into the ability to advocate for an increase in budget. This means that the PDHJ remains underfunded in an environment of an absence of any confirmed donor funding. GoTL agencies seem willing to respond to operational recommendations where budget allows but otherwise do not pay sufficient heed to the recommendations of the PDHJ. It is difficult to see how PDHJ can continue the important work commenced during the G2G project without either an increased allocation from the state budget and/or future donor funding. 9. Recommendations For donors 1. Donors may wish to consider funding specific aspects of support to the PDHJ which would capitalise on, and consolidate, the achievements already realised during the G2G assistance. This could include support to comprehensively reviewing the effectiveness of the complaint mechanism, ensuring that IT functionality is established and maintained including through the development of local staff capacity, and/or supporting the establishment of a more effective and comprehensive legal support unit. For Government and the National Parliament 2. GoTL and the national parliament should ensure that PDHJ receives adequate funds to perform all the functions set out in its mandate. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 29 3. Parliament should more actively engage with PDHJ, in particular through the annual reporting process and the parliamentary committees in order to make government more accountable and ensure that PDHJs recommendations are properly considered by government authorities. For relevant state authorities 4. Relevant state authorities should be proactive in seeking engagement with PDHJ, ensuring that their own staff are aware of their rights and obligations, and responding in a timely manner to PDHJ recommendations. For relevant CSOs 5. Relevant CSOs should consider in what ways they can support a better human rights culture by partnering with or supporting PDHJ for improved advocacy, outreach, monitoring, research, and better understanding of the complaints' mechanism. For PDHJ 6. The PDHJ should consider the benefits of ensuring that the IT environment within PDHJ allows for all staff to access the internet portal and associated KMS, e-CMS, internet and email. 7. PDHJ should pursue the updating and maintenance of a professional website. 8. PDHJ should consider the benefits of improved telephone and vehicle access for staff within the available budget. 9. PDHJ should ensure increased effectiveness of regional offices through promotion of good communications, teamwork and collaboration across functions. 10. PDHJ should pursue its plans to develop and implement a gender policy and strategy that will encompass both the internal functioning of the organization, and its external relations. 11. PDHJ should pursue the establishment of a fully staffed legal advice unit to enable it to fulfil its mandate. 12. PDHJ may wish to consider options for increasing its coverage and dosage for outreach and monitoring activities, within its available budget. This may include an increased emphasis on mass media for outreach and an increased emphasis on collaboration with civil society for monitoring. 13. PDHJ should review the effectiveness and functioning of the current complaints’ mechanism, with the intention of making it simple, accessible, speedy and transparent. 14. PDHJ should more actively and publicly review and recommend amendments to existing and draft legislation and other regulations according to international human rights standards, comment on government policy relevant to human rights, and consider holding inquiries into specific economic, cultural and social rights. 15. PDHJ should explore how it can be more accessible to the public, particularly in cases of emergency. 16. PDHJ should consider ways to make recommendations in reports better focused and targeted, setup a regular timetable for following up, and report regularly and publicly on implementation of recommendations. 17. PDHJ should consider development of a strategy for collaborating and partnering more fully with civil society, in order to improve advocacy, outreach, monitoring, research, and better understanding of the complaints' mechanism. 18. PDHJ should consider development of a strategy for collaborating and partnering more fully with prioritised state institutions. 19. PDHJ should make its MoUs with state institutions and civil society public. 20. PDHJ should continue to promote its achievements and advocate with GoTL for improved and adequate funding. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 30 10. Annexes Annex 1 Statement of Work (SOW) Annex 2 List of consultations Annex 3 References Annex 4 Signed Disclosure of Interest Annex 5 Data collection and analysis tools Annex 6 Assessment Team Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. I Annex 1 Statement of Work (SOW) STATEMENT OF WORK (SOW) FINAL ASSESMENT STRENGTHENING DECENTRALIZATION IN THE OFFICE OF THE OMBUDSMAN FOR HUMAN RIGHTS AND JUSTICE - PHASE II PROJECT 1. TITLE OF ACTIVITY Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. 2. PURPOSE The purpose of this assessment is to provide USAID/Timor-Leste, New Zealand Ministry of Foreign Affairs and Trade (NZMFAT), and the Government of Timor-Leste (GoTL) an external assessment of the effectiveness of the Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project (Fixed Amount Reimbursement Agreement (FARA) award, No. 472-G2G￾15-001). Of the $523,443 funding for the FARA with PDHJ $200,000 (38 percent) is from USAID and $323,443 is from NZMFAT (62 percent). The FARA began in September 2015 and ended in January 2019. The assessment will be used by USAID, the NZMFAT, and the Timorese’s Provedoria (Ombudsman) for Human Rights and Justice (Portuguese acronym – PDHJ) to identify and document the short- and long￾term effectiveness of this assistance including but not limited to the impact of this assistance for PDHJ to advocate for policy changes in Timor-Leste; GoTL’s expected long-term commitment to PDHJ; the PDHJ’s influence to and relationship with other government institutions; and lessons learned for any future assistance by USAID, NZMFAT, or other donors. The NZMFAT recently completed a comprehensive evaluation of their programs in Timor-Leste including assistance to the PDHJ available at https://www.mfat.govt.nz/assets/Aid-Prog-docs/Evaluations/2018/Timor-Leste-CPE-Report-2- Strategic-Assessment-Oct-2017.pdf. This assessment will complement not duplicate the NZMFAT evaluation, and the findings will be presented to in-country stakeholders, and the assessment report will be posted to USAID’s Development Experience Clearing House (DEC12). 3. BACKGROUND The FARA was a follow-on activity to previous USAID assistance. The goal of the FARA is to improve PDHJ’s ability to advocate for human rights and good governance by increasing community uptake and benefit from PDHJ services and improving the performance of the PDHJ. Specifically: (1) build the PDHJ’s ability to increase public knowledge of human rights, good governance, and the PDHJ mandate; (2) enhance the capacity of PDHJ staff, particularly in its four regional offices, to monitor and investigate good governance and human rights issues across the country, and; (3) improve PDHJ’s overall capacity and business processes. The PDHJ is the GoTL’s national human rights institution established by parliament in compliance with the 2002 Constitution which stipulates the establishment of “an independent organ in charge of examining and seeking to settle citizens’ complaints against public bodies, preventing and initiating the whole process to remedy injustice.” PDHJ had three mandates: to protect human rights, promote good governance, and combat corruption. The anti-corruption mandate was repealed in 2008 following the establishment of an independent Anti-Corruption Commission. The first Provedor was elected by parliament in March 2005 and the PDHJ started operations in March 2006 and since then three Provedors have led the institution. The PDHJ has a central office in Dili and four regional offices in Oecusse, Baucau, Maliana (Bobonaro Municipality) and Same (Manufahi Municipality). The PDHJ has a broad human rights and good governance mandate, principally to protect and prevent human rights violations. It deals with complaints and cases of human rights and good governance violations, implements education and promotion activities, and monitoring of human rights and good 12 USAID's Development Experience Clearinghouse (DEC) is the largest online resource for USAID￾funded technical and program documentation Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. II governance issues. PDHJ is mandated to integrate human rights and good governance principles into the legislation and policy of Timor-Leste. In addition to providing recommendations to promote accountability, the PDHJ also recommends how to strengthen systems and create an environment which will prevent future violations. PDHJ’s Strategic Plan 2011 – 2020 priorities include ensuring state compliance with human rights and good governance principles, ensuring that PDHJ has the capacity and independence to guarantee the implementation of its mandate, protecting vulnerable groups against human rights abuses, and ensuring equitable access to public services. 4. ASSESSMENT QUESTIONS13 The assessor is expected to answer the following assessment questions: 6. What were the key outcomes or substantive changes, positive or negative, to the human and institutional capacity of the PDHJ especially at the regional offices? 7. To what extent has the assistance achieved its goals to build the regional offices’ ability to increase public knowledge of human rights, good governance, and the PDHJ mandate, and to monitor and investigate good governance and human rights issues? What are the key factors that support and/or constrain the achievements? 8. To what extent has this assistance influenced PDHJ’s ability to advocate for policy changes within the GoTL? 9. To what extent has this assistance influenced the GoTL’s expected long-term commitment to PDHJ demonstrated by the current and future budgets allocation to PDHJ? How and to what extent can PDHJ sustain activities without this assistance in the future? 10. To what extent has this assistance impacted the PDHJ’s influence on and relationship with other government institutions? How have the other government institutions received and acted upon PDHJ findings and recommendations? 5. SUGGESTED METHODS OF ASSESSMENT The assessment will be conducted by using a qualitative method for collecting data for example but not limited to individual and group interviews and a desk-top review of relevant documents. The assessor must conduct key informant interviews with PDHJ staff, beneficiaries such as the community members and students who attended the outreach sessions, GoTL counterparts such as the Ministry of Education, Ministry of Interior, Ministry of Justice, Prison Directorate, the National Police of Timor-Leste, Ministry of State Administration, Ministry of Health and other relevant ministries, donors such as NZMFAT and the United Nations Development Program (UNDP), and other stakeholders. 6. PLACE OF PERFORMANCE The assessor must conduct its work in Dili, and travel to a minimum of two regional offices in Maliana, Same, Baucau, and/or Oecusse. The assessor will propose to USAID criteria for the selection of the regional offices to be visited. The criteria can include the numbers of activities implemented by the regional offices, the geographic coverage and/or population of the regional offices, and other criteria. Trips outside of Dili will require overnight stays. 7. STAFFING AND EXPERTISE An external, independent assessment team is sought to undertake this assessment. The team shall consist of an international and a Timorese consultant. The following are the responsibilities and requirements per team member: • International Consultant Responsibilities: 1. Serve as the team leader 13 The assessor should also examine questions through engendered lens. So in assessing all questions, the assessor should report any gender biases improved or worsened via assistance to PDHJ. For example did male or female staffer or recipients benefit more or less from USAID’s assistance. Where there any increases in GBV policy advocacy as a result of this assistance? Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. III 2. Develop assessment methodology, plan the work and carry out field visits 3. Conduct desk-top reviews and key informant interviews in conjunction with the Timorese consultant, and analyze the results 4. Organize in- and out-brief to USAID 5. Present assessment findings to NZMAFT, PDHJ and other stakeholders 6. Write the assessment report 7. Submit the draft and final assessment report to USAID/Timor-Leste Requirements: 1. A senior human rights or good governance or justice specialist/expert with at least 10 year experience in international development 2. At least eight years of experience evaluating or assessing donor-funded human rights, good governance, or justice programs 3. Thorough understanding of the country and its socio-political context 4. Proven record of excellent management and leadership of evaluation or assessment teams 5. Excellent English writing and communication skills 6. Prior experience in human rights, good governance, or justice sector in Timor￾Leste will be an asset 7. Ability to communicate in Tetun and/or Portuguese will be an asset • Timorese consultant Responsibilities: 1. Develop schedule for the regional office visits in coordination with USAID/Timor-Leste and PDHJ 2. Contact key informants, GoTL, and other stakeholders to set-up visits and interviews 3. Conduct key informant interviews in conjunction with the team leader, and assist in analyzing the results 4. Provide input regarding the human rights, good governance, and justice sectors specific to Timor-Leste Requirements: 1. University degree or equivalent in law, social sciences, or another relevant area 2. At least five years of experience working in the area of human rights, good governance, or justice in Timor-Leste. Cumulative five years’ experience in these sectors is acceptable. 3. Prior experience evaluating or assessing donor or government-funded programs 4. Experience interacting with GoTL, donors, and a broad range of stakeholders. 5. Familiarity with international context in post-conflict and developing countries 6. Good English writing and communication skills 7. Fluency in Tetun is required; the ability to communicate in Portuguese and/or another local Timorese dialects is an advantage 8. TASKS & DELIVERABLES • Desk-Top Review and Assessment Methodology Preparation. The assessment team will review FARA documents and other relevant documents to familiarize themselves with the USAID assistance and to gain a better understanding of the situation in Timor-Leste. Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. IV USAID/Timor-Leste will provide the relevant project documents to the assessment team prior to commencing the assessment. • In-Brief with USAID/Timor-Leste. The assessment team will hold an in-brief with the USAID’s Contracting Officer’s Representative (COR) and other USAID/Timor-Leste staff. The purpose of the in-brief is for introductions and to discuss the team’s understanding of the assignment, initial assumptions, assessment questions, methodology, work plan and timeline. • Field Work and Report Preparation. The assessment team will meet with the PDHJ including the regional offices, GoTL counterparts, beneficiaries, donors, NZMFAT, USAID, and other stakeholders. The assessment team will undertake data gathering and analysis, as appropriate, including interviews, discussion, gathering primary data, review of primary and secondary data, as well as other means of establishing a comprehensive overview of the assistance to answer the assessment questions. • De-Briefing with USAID/Timor-Leste. The assessment team will hold a de-brief meeting with the COR and USAID/Timor-Leste staff to present preliminary findings and recommendations. The de-brief will be conducted prior to the submission of the draft report to USAID/Timor-Leste. • De-brief with NZMAFT and PDHJ. The assessment team will hold a de-brief meeting with NZMAFT, PDHJ, USAID and other stakeholders after the de-briefing with USAID/Timor-Leste. The meeting aims to present the preliminary findings from the assessment after taking into consideration feedback and inputs from USAID/Timor￾Leste. • First Draft Assessment Report. A draft report (electronic version, using Microsoft Word or Adobe Document – PDF) will be submitted to USAID within seven days after the completion of field work and data analysis. USAID/Timor-Leste will provide comments on this draft through the COR within 10 business days. The report must meet the following criteria (please also see section 11. Final Report Format) : o Executive Summary (2-5 pages) in length that summarizes key points o State the purpose of, audience for, and anticipated use of assessment o describe the specific strategy, project, activity, or intervention to be assessed o provide background information o state assessment questions o describe limitations of analysis o including findings and conclusions, and recommendations in a separate section o Include in annex: description of methodology, all sources of information, any statement of differences, signed disclosures of conflicts of interests, summary of information about assessment team members, including qualifications, experiences, and role of team • Second Draft Assessment Report. The assessor must submit a second draft assessment report (electronic version, using Microsoft Word or Adobe Document – PDF) to the COR that responds to USAID’s comments regarding the first draft report. USAID/Timor-Leste will provide comments on this draft within 10 days after receipt. • Final Assessment Report. The assessor must submit a final assessment report in both Microsoft Word and Adobe Document – PDF formats to the COR that responds to USAID’s prior comments 10 days after receiving the second comments from USAID. The Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. V report must be uploaded to the DEC14. At that time, the COR will advise of permission rights for viewing (i.e. USAID only, other) of the report on the DEC. • Assessment Summary. The assessor will develop an assessment report summary in English and Tetun. The summary will be shared with GoTL and other relevant counterparts. The summary should include: – Title of the assessment – Date the report was submitted – Assessment Questions – Methodology used – Key findings – Recommendations – The summary should discuss broad details rather than specifics. Confidential, sensitive and/or classified issues should be omitted. 9. SUGGESTED SCHEDULE The period of performance of this assessment is approximately from April 29 to July 5, 2019 (30 business days). A six-day work/week is authorized. The table below is an illustrative schedule. The assessor shall propose a detailed schedule, assessment design, and implementation plan in its proposal. Task Month (year) Desk-top review and assessment methodology preparation Day 1-3 (business days) In-brief Day 3 Field work Day 4-13 Analysis and Report Writing Day 14-18 De-brief including presentation of the assessment findings to NZMAFT, PDHJ and other stakeholders Day 19 Submission of First Draft Report Day 20 – USAID will have 10 business days after receiving the first draft to review and provide comments Submission of Second Draft Report (to include a summary in English and Tetun) Day 25 – USAID will have 10 business days after receiving the second draft to review and provide comments Submission of Final Report Day 30 (July 5, 2019) 10. LOGISTICS 10. 1 RESPONSIBILITIES OF THE ASSESSMENT TEAM The assessor is responsible for all logistical needs necessary to performance of the assessment including, but not limited to computer access with internet, quantitative and qualitative software if applicable, document printing and copying, lodging, office space, transport, interpretation during data collection, and translation of source documents. The assessor must communicate directly with the COR, proactively identifying constraints that may impact the performance of the assessment. 10.2 RESPONSIBILITIES OF THE COR • Serve as USAID/Timor-Leste’s point of contact with the assessment team • Provide all FARA and other related documents to the assessor • Liaise with PDHJ as needed to facilitate the assessment process 14 Link to the USAID’s DEC document submission: https://dec.usaid.gov/dec/content/submit.aspx Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. VI • Meet with the assessor as necessary to provide technical direction • Schedule in-brief and de-briefs with USAID/Timor-Leste • Provide timely feedback to the assessor in compliance with the assessment schedule 11. Final Report Format The assessment final report must include title page, list of acronyms and abbreviations, table of contents, including list of annexes, executive summary; background of the local context and the strategies/projects/activities being assessed; the assessment purpose and main assessment questions; the methodology or methodologies; the limitations to the assessment; findings, conclusions, and recommendations (if applicable). The executive summary should be 2–5 pages in length and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, and conclusions [as well as recommendations and lessons learned, if applicable]. The assessment methodology must be explained in the report in detail. Limitations to the assessment must be disclosed in the report, with particular attention to the limitations associated with the assessment methodology. The annexes to the report must include: • The assessment SOW; • All data collection and analysis tools used in conducting the assessment, such as questionnaires, checklists, and discussion guides; • All sources of information/data, properly identified and listed; • Signed disclosure of conflict of interest forms for all assessment team members, either attesting to a lack of conflicts of interest or describing existing conflicts of interest; • Any “statements of difference” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the assessment team; and • Summary information about assessment team members, including qualifications, experience, and role on the team. 12. Other Requirements All deliverables that are in written format must be in plain15 , grammatically correct English language; be submitted in appropriate electronic format (i.e. Microsoft Word, Excel, Power Point Presentation, and PDF); and meet all the requirements. All raw data, quantitative and qualitative, collected by the assessment team must be provided in machine-readable, non-proprietary formats at www.usaid.gov/data as required by USAID’s Open Data policy (see ADS 579, USAID Development Data). The data should be organized and fully documented for use by those not fully familiar with the assessment. USAID will retain ownership of all survey and datasets developed. All modifications to the required elements of the SOW of the Purchase Order (PO), whether in technical requirements, assessment questions, assessment team composition, methodology, or timeline, must be agreed upon in writing and approved by USAID through the COR. Any revisions must be included in 15 Plain language (also called Plain English) is communication that your audience can understand the first time they read or hear it. There are many writing techniques that can help you achieve this goal. Among the most common are: logical organization with the reader in mind; active voice; short sentences; no jargons; easy-to-read design features. No one technique defines plain language. Rather, plain language is defined by results— it is easy to read, understand, and use. See more details at http://www.plainlanguage.gov/ Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. VII a revised SOW and be an annex to the assessment report. A representative(s) of USAID may participate in the meetings with government officials and/or field data collections. 13. Payment Schedule Payment will be provided in accordance with the below schedule: First Payment: 30% after the completion of field work Second Payment: 30% after the submission of the first draft report Final Payment: 40% after the submission and acceptance by USAID of the final report Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. VIII Annex 2 List of consultations Name Position PDHJ-current Jesuina Maria Ferreira Gomes Provedor Aureo Savio Director-General/Executive Secretary, PDHJ Benícia Eriana Magno Deputy Provedor for Human Rights Miguel Pereira de Carvalho Deputy Provedor for Good Governance Rosalina de Jesus Pires Director, Investigations Elsa do Rosario Viegas da Costa Director of Public Assistance Joanita Silvira da Costa Director, Baucau Territorial Delegation Agapito Rodrigues de Fátima Investigator, DT Baucau Ana Elvira Manuela Belo Promotions, DT Baucau Angelino Madeira Finance and Administration Assistant, DT Baucau Martinho Pereira Regional Director, DT Same Leila da Costa Investigator and Finance/Administration, DT Same PDHJ-former Silverio Pinto Baptista Former Provedor Horacio de Almeida Former Deputy Provedor of Human Rights Ruth Hugo Former Adviser Barbara Nazareth Oliveira Former Adviser State Institutions Helder Cosme Marcal Belo National Director, Social Reinsertion and Prisons, Director Dili Prison Hermes da Rosa Correia Barros National Director for Local Administration, Ministry of State Administration. Belchior Maria Almeida Barros National Director, School Social Action, Ministry of Education Euclides Vidal Madeira Chief of Information Unit, Anti-Corruption Commission Maria Celestina Viegas Executive Secretary, Baucau Municipality Vicente Soares Chief of Department of School Administration and Management, Ministry of Education, Baucau Municipality Rosalia do Espiritu Santo da Cruz Chief Department Human Resources, Manatuto Municipality Andre de Fatima Viegas Chief Department Management & Administration, MoE, Manatuto Municipality Henrique dos Santos Chief Administration, Postu Manatuto Villa Arantes Isaac Sarmento Municipal Administrator, Same Trinito Sarmento Director Education, Same Luis Casimiro Lopes Posto Administrator, Maubisse, Ainaro Jose Reali Legal and Policy Adviser, Civil Service Commission PNTL Chief Superintendent Carlos Jeronimo Inspector-General Inspector Virgilio Correia Legal Adviser, Commander-General’s Office Superintendent Orlando Gomes Chief, Department of Justice Assistant Superintendent Euclides Belo 2 nd Municipal Commander, Dili Assistant Superintendent Jony Viana das Regras 2 nd Municipal Commander, Baucau Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. IX Superintendent Agustinho Gomes Municipal Commander, Same Chief Inspector Armando Sila Operations Commander, Manatuto Inspector Afonso da Silva Squadron Commander, Sub-District Maubisse, Ainaro DONORS Carl Seagrave Program Office Director, USAID Ana L. da C. Gusmão Guterres Project Management Specialist, Democracy and Governance Program, USAID Nurbertus Widiastara Deputy Executive Officer, Executive Office, USAID Teodulo Clemente de J. Ximenes Project Management Specialist, Health Program Milca da Conceição Baptista Monitoring, Evaluation & Learning Specialist, Program Office Gabrielle Isaak Deputy Head of Mission/First Secretary- Development, New Zealand Embassy Augusto Soares Development Program Coordinator, New Zealand Embassy OHCHR Anjet Lanting Human Rights Adviser, OHCHR CIVIL SOCIETY ORGANISATIONS Maria Jose Guterres Director, Fokupers Jose Pereira Coordinator Parliament Watch Program, Judicial System Monitoring Program Jose Luis de Oliveira Executive Director, Asia Justice and Rights, Timor-Leste Manuel Monteiro Executive Director, HAK Association Elvira Maria da Costa Lawyer, Asistencia Legal ba Feto no Labarik (ALFELA), Baucau Joana da Costa Coordinator, PRADET, Baucau Saturnina Freitas Belo District Liaison Officer, Nabilan Program, Same Quintão da Costa Massa Director, Uma Dame, Same OTHER Aderito de Jesus Soares Former Anti-Corruption Commissioner, Timor-Leste Meabh Cryan Director, Tahan Consulting Unipessoal LDA Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. X Annex 3 References (* given the wide variety of citations in multiple languages for the name of the PDHJ, this assessment has opted for the use of the abbreviation of PDHJ throughout.) Adam Smith International, 2017. New Zealand Ministry of Foreign Affairs and Trade: Evaluation of New Zealand’s Country Programme in Timor-Leste, Evaluation Compendium – Part Two: Country engagement and strategic outcomes, 19 October 2017. Aryal, Rishi, 2016. Final Report: Knowledge Management System PDHJ Submitted 18/4/2016. Asia Justice and Rights (AJAR), 2018. Annual Report 2018 Asia Pacific Forum of National Human Rights Institutions (APF), 2019. Putting women at the heart of NHRI planning. https://www.asiapacificforum.net/news/putting-women-heart-nhri-planning/. Accessed 09 July 2019. Asia Pacific Forum of National Human Rights Institutions (APF), 2018. PDHJ trains defence personnel in human rights. https://www.asiapacificforum.net/news/pdhj-trains-defence￾personnel-human-rights/ . Accessed 09 July 2019 International Council on Human Rights Policy, 2005. Assessing the Effectiveness of National Human Rights Institutions. International Council on Human Rights Policy and OHCHR, Geneva. JSMP, 2019. Komunikadu Imprensa Parlamentu Nasionál 04 Jullu 2019 Kestaun aprensaun, prizaun preventiva liu prazu no fiskalizasaun abstrata sai preokupasaun deputadu sira-nian iha apresiasaun relatóriu anuál PDHJ 2018 JSMP, 2018. Timor-Leste: The PDHJ must make full use of all of its constitutional and legal competencies and powers in 2018 ANNI Report on the Performance and Establishment of National Human Rights Institutions in Asia, The Asian NGO Network of National Human Rights Institutions (ANNI) JSMP, 2013. Timor-Leste: Lacking Pro-Activeness and Transparency in 2013 ANNI Report on the Performance and Establishment of National Human Rights Institutions in Asia, The Asian NGO Network of National Human Rights Institutions (ANNI), Edited by Balasingham Skanthakumar Kluyskens, J., K. Roberts and J.L. Oliviera, 2013. Mid-term Evaluation of the UNDP/OHCHR Capacity Building of PDHJ Project 2010 - 2014 Final Report OECD, 2000, Glossary of Evaluation and Results Based Management (RBM) Terms OECD, 1991. The DAC Principles for the Evaluation of Development Assistance OHCHR, 2012. Human Rights Indicators: A Guide to Measurement and Implementation Government of Timor-Leste 2011 Timor-Leste Strategic Development Plan 2011-2030 Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XI Linos, K. and Pegram, T. 2017. What Works in Human Rights Institutions? The American Journal of International Law, Vol 112, no 3 July 2017, pp 1-61. PDHJ, 2019a. Relatório Annual 2018: Provedoria dos Direitus Humanus, Dili, Juñu 2019. PDHJ, 2019b. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Annual Report Year 3 - Date of Submission 28 January 2019 PDHJ, 2019c. Perception Survey 2019, PDHJ PDHJ, 2019d. Relatoriu kona-ba Monitorizasaun Implementasaun Programa Merenda Eskolar “Jestaun Merenda Eskolar” Etapa III-2019, Dili Januari 2019. PDHJ, 2018a. Relatório Annual 2017: Provedoria dos Direitus Humanus, Dili, Juñu 2018. PDHJ, 2018b. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Annual Report Year 2 PDHJ, 2018c. Perception Survey 2018, PDHJ PDHJ, 2018d. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Quarterly Report Milestone 11 Date of Submission 25-9-2018. PDHJ, 2018e. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Quarterly Report Milestone 10 Date of Submission 22-6-2018. PDHJ, 2018f. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Quarterly Report Milestone 9 Date of Submission 20-4-2018. PDHJ, 2018g. Relatoriu kona-ba Monitorizasaun Fatin Detensaun Polisia iha Timor-Leste, Dili. PDHJ, 2017a. Relatório Annual 2016: Provedoria dos Direitus Humanus, Dili, Juñu 2017. PDHJ, 2017b. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Quarterly Report Milestone 7 Date of Submission 28-6-2017. PDHJ, 2017c. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Quarterly Report Milestone 6 Date of Submission 21-4-2017. PDHJ, 2017d. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Quarterly Report Milestone 5 Date of Submission 10-1-2017. PDHJ, 2017e. Relatoriu kona-ba Monitorizasaun Fatin Detensaun Polisia iha Timor-Leste, Dili Novembru 2017. PDHJ, 2017f. Relatoriu kona-ba Monitorizasaun Prizaun iha Timor-Leste- Etapa II, Dili Novembru 2017 Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XII PDHJ, 2017g. Relatoriu kona-ba Monitorizasaun Implementasaun Programa Merenda Eskolar “Jestaun Merenda Eskolar”2017, Dili Novembru 2017. PDHJ, 2016a. Relatório Annual 2015: Provedoria dos Direitus Humanus, Dili, Juñu 2016. PDHJ, 2016b. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Annual Report- Date of Submission 28-9-2016 PDHJ, 2016c. Perception Survey 2016, PDHJ PDHJ, 2016d. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Quarterly Report Milestone 3 Date of Submission 19-7-2016. PDHJ, 2016e. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Quarterly Report Milestone 2 Date of Submission 26-4-2016. PDHJ, 2016f. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Quarterly Report Milestone 1 Date of Submission 20-4-2016. PDHJ, 2016g. Relatoriu kona-ba Monitorizasaun Fatin Detensaun Polisia iha Timor-Leste, Dili Setembru 2016. PDHJ, 2016h. Relatoriu kona-ba Monitorizasaun Prizaun iha Timor-Leste, Dili Setembru 2016. PDHJ, 2016i. Relatoriu kona-ba Monitorizasaun Implementasaun Programa Merenda Eskolar “Jestaun Merenda Eskolar”, Dili Setembru 2016. PDHJ, 2015. Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project Attachment 1 Project Description PDHJ, 2011. Strategic Plan of the PDHJ 2011-2020 PDHJ, Asia Pacific Forum of National Human Rights Institutions, United Nations Development Program Asia Pacific Regional Hub and United Nations Office of the High Commissioner for Human Rights, 2015. Capacity Assessment of the Office of the Provedor for Human Rights and Justice (PDHJ) of Timor-Leste, Final Report 30 June 2015. PNTL, 2019. PDHJ halo seminariu ba PNTL atu Asegura Direitu Komunidade LGBT. https://www.pntl.tl/pdhj-halo-seminariu-ba-pntl-atu-asegura-direitu-komunidade-lgbt/. Accessed 9 July 2019. República Democrática de Timor-Leste (RDTL), 2019. Orçamento Geral do Estado 2019, Planos de Acção Anual Livru 2 República Democrática de Timor-Leste (RDTL), 2018. Orçamento Geral do Estado 2018, Planos de Acção Anual Livru 2 Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XIII República Democrática de Timor-Leste (RDTL), 2017. Orçamento Geral do Estado 2017, Planos de Acção Anual Livru 2 República Democrática de Timor-Leste (RDTL), 2016. Orçamento Geral do Estado 2016, Planos de Acção Anual Livru 2 República Democrática de Timor-Leste (RDTL), 2015. Orçamento Geral do Estado 2015, Planos de Acção Anual Livru 2 The Asia Foundation, 2019 [in press]. Community Police Perception Survey 2018 The Asia Foundation, 2019 [in press]. Tatoli! 2018 Survey The Asia Foundation, 2016. Understanding Violence against Women and Children in Timor-Leste: Findings from the Nabilan Baseline Study – Main Report. The Asia Foundation: Dili. Soares, A. 2012. Combatting Corruption: Avoiding Institutional Ritualism in Leach, Michael and Kingsbury, Damien, The politics of Timor-Leste: democratic consolidation after intervention, Cornell University Press, Ithaca, N. Y United Nations, 2017. Convention Against Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment, Concluding Observations on the initial report of Timor-Leste, CAT/C/TLS/CO/1, 15 December 2017 United Nations, 2015. Economic and Social Council, Commission on Crime Prevention and Criminal Justice, Vienna 18-22 May 2015. E/CN.15/2015/L.6/Rev.1 United Nations Country Team in Timor- Leste, 2016. Report for the Universal Periodic Review (UPR) of Timor-Leste 26th Session of the UPR Working Group Oct-Nov 2016 Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XIV Annex 4 Signed Disclosures of Interest Name Bu Wilson Title Director/Principal Consultant Organization Hau Meni & Associates Pty Ltd Assessment Position? Team Leader/International Consultant Assessment Award Number (contract or other instrument) Order Number 72047219P00002 Requisition Reference Number REQ-472-19-000015 USAID Project(s) Assessed (Include project name(s), implementer name(s) and award number(s), if applicable) Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Implementing Partners USAID and New Zealand Aid I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature 10 July 2019 Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XV Name Abel Amaral Title Interim Deputy Director Organization Fundasaun Mahein Assessment Position? Team member/National Consultant Assessment Award Number (contract or other instrument) Order Number 72047219P00002 Requisition Reference Number REQ-472-19-000015 USAID Project(s) Assessed (Include project name(s), implementer name(s) and award number(s), if applicable) Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice Phase II Project, Implementing Partners USAID and New Zealand Aid I have real or potential conflicts of interest to disclose. Yes If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 7. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 8. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 9. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 10. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 11. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 12. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. Fundasaun Mahein has previously collaborated with PDHJ on the development of a training manual for F-FDTL. I don't believe this presents any conflict of interest for me in undertaking this assessment. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 10 July 2019 Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XVI Annex 5 Data Collection and analysis tools Key Evaluation Questions Who can help us answer this? Relevant Documents to answer this? Sub Questions 10 What were the key outcomes or substantive changes, positive or negative, to the human and institutional capacity of the PDHJ especially at the regional offices? • PDHJ national office • PDHJ regional offices • Relevant Civil Society Organisations, • GoTL counterparts • PDHJ report Y1, Y2, Y3 • FARA 1,2,3 • Perception survey 2016, 2018, 2019 • Capacity Assessment Report? • Are there any assessments of human and institutional capacity at the beginning of the Project Period as a bench mark? (Capacity Assessment) • What are the key outcomes of human and institutional capacity and changes for the national office and the regional offices compared to the benchmark? (List domains using capacity assessment) • In what ways did the national office support the development of the capacity of regional offices? • What has changed about the way that work is coordinated between national and regional offices? (Probe: Are regional offices doing investigations or only taking in complaints?) Any challenges? • Is there any difference between the outcomes for the different offices? What and Why? • Were there any unanticipated outcomes (positive or negative) • What are the remaining challenges? 11 To what extent has the assistance achieved its goals to build the regional offices’ ability to increase public knowledge of human rights, good governance, and the PDHJ mandate, and to monitor and investigate good governance and human rights issues? What are the key factors • PDHJ national office • PDHJ regional offices • OHCHR • PNTL commander • Municipal administrator • CSOs • Community • PDHJ report Y1, Y2, Y3 • FARA 1,2,3 • Perception survey 2016,2018, 2019 • Capacity assessment report? • Summary of goals of project • Summary of PDHJ workplan • Are there any assessments of outreach capacity at the beginning of the Project Period as a bench mark? (Capacity Assessment) • How do you interact with civil society on this outreach? Success? Challenges? • What does the public perception survey tell us about increased public knowledge of human rights, good governance, and PDHJ mandate? (M/F disaggregation) Can this be attributed to increased regional office capacity? How? Why? Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XVII that support and/or constrain the achievements? • Other Government institutions • What other evidence is there of this increased public knowledge? (Not number of trainings!) • What qualitative and quantitative evidence is there of improved monitoring/investigation related to good governance and human rights? • How are gender considerations incorporated into monitoring? • How are gender considerations incorporated into investigations? • How do the monitoring and investigation functions of PDHJ work together? Successes? Challenges? • Do you have a Gabinete Assesora Juridica to assist? • Key factors supporting achievements? • Key factors constraining achievements? 12 To what extent has this assistance influenced PDHJ’s ability to advocate for policy changes within the GoTL? • PDHJ national office • OHCHR • Government institutions • Committee A/B • PNTL commander • CSO • FARA 1,2,3 • Perception survey 2016, 2018, 2019 • Capacity assessment report? • Are there any assessments of capacity/ability on advocacy for policy change at the beginning of the Project Period as a bench mark? (Capacity Assessment) • What are key examples of policy influence? • What is different now about staff’s ability to advocate for policy change? What skills does PDHJ utilise in their advocacy work? How effective is this? • What are your mechanisms of cooperation with key institutions (that would enable this policy advocacy)? How effective is this? • What are the ongoing challenges internal or external? • How does PDHJ choose what to advocate on and what not to advocate on? How are gender considerations included? Is this done in collaboration with other organisations? • What are examples of PDHJ successful advocacy with GoTL? Why was it successful? • What are the key policy changes that you advocated on during this period? Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XVIII • Are your successes more on individual cases or on systemic issues? (Explore reasons) (probe on Use of Force) • What are examples of unsuccessful advocacy with GoTL? Why was it unsuccessful? • What else could PDHJ be working on? • What would be needed to be more successful? 13 To what extent has this assistance influenced the GoTL’s expected long￾term commitment to PDHJ demonstrated by the current and future budgets allocation to PDHJ? How and to what extent can PDHJ sustain activities without this assistance in the future? • International Agency/ embassy • government (MRLAP?), • OHCHR • Perception survey 2016, 2018, 2019 • Budget Books • Capacity assessment report? • What is the history of GoTL and donor allocation over the project period? And the projected budgets? • Are senior figures in GoTL committed to the future of PDHJ? • What was PDHJ’s budget justification in the most recent budget round? • What activities can PDHJ sustain within the current and projected GoTL allocations? • Are there any other potential sources of funding? 14 To what extent has this assistance impacted the PDHJ’s influence on and relationship with other government institutions? How have the other government institutions received and acted upon PDHJ findings and recommendations? • PDHJ national office • PDHJ regional office • HAK • AJAR • Government institutions • OHCHR • Committee A/B • Perception survey 2016, 2018, 2019 • Capacity assessment report? • Who are the key institutions that PDHJ seeks to influence? • What are the formal and informal processes to do this? • What key findings and recommendations has PDHJ made to other institutions? • How have those institutions responded? • How do you support key institutions to do better next time? • Are there any assessments of PDHJ’s influence on and relationship with other institutions at the beginning of the project period? • For each relevant institution what is different about PDHJ influence and relationship between the beginning of the project period and the end? • How did the assistance seek to strengthen PDHJ influence? How did the assistance seek to strengthen PDHJ relationships? Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XIX • How has this been assessed? • Does PDHJ publish the results of investigations? 15 *Cross-cutting question: Gender: In what ways did assistance to PDHJ improve or worsen gender biases? Did male or female staff or recipients benefit more or less from USAID’s assistance? Were there any increases in GBV policy advocacy as a result of this assistance? • PDHJ • OHCHR • PNTL VPU, • People with disability organization • women group • Rede Defensor Direitus Humanus • Perception survey 2016, 2018, 2019 • Capacity assessment report? • To what extent did women participate in PDHJ activities? • What is the M/F staffing profile of the PDHJ? • What percentage of training is attended by female/male staff? • How were the outcomes of the assistance different for women and men among the PDHJ staff in regional offices? • To what extent has PDHJ advocated on issues that are of particular relevance to women, and how successful has this advocacy been 16 *Cross-cutting question: Disability • PNTL VPU, • People with disability organization • women group • Rede Defensor Direitus Humanus • How did the assistance ensure participation of women and men with disabilities? • What approaches were used to make the project disability inclusive? What other approaches could have been done to make it more inclusive? • How have the project outcomes and outputs benefitted people with disabilities? If not, why not? Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XX Annex 5.1 Draft Interview Guide (to be reviewed as required) Introduction Questions for USAID and MFAT 1.1 Your priorities for this evaluation? Priorities for who we should meet with? Any cautions? 1.2 From your perspective the strengths and remaining challenges of the PDHJ as a result of assistance? (Probe domains: National/regional coordination, GoTL relations, civil society relations, monitoring, complaints handling, investigation, awareness raising, acting on constitutional mandate, case advocacy, systemic policy advocacy, financial sustainability, gender and disability) 1.3 What are the opportunities for PDHJ moving forward? 1.4 Opportunity to obtain the 2015 tripartite capacity assessment? Questions for PDHJ Head office 2.1 What are the key GoTL institutions you engage with? Over the project period what changes have there been in engaging GoTL institutions as partners? Can you describe the division of responsibilities for GoTL engagement at national and regional office? Do you have MoU’s with key GoTL institutions How are those implemented? Successes and challenges? 2.2 What have been your successes in being effective as an independent institution? What are the remaining challenges? Are there differences in effectiveness in the area of human rights and the area of good governance? 2.3 What are the key civil society institutions you engage with? Over the project period what changes have there been in engaging civil society institutions as partners? Can you describe the division of responsibilities for civil society engagement at national and regional office? Do you have MoU’s with civil society institutions How are those implemented? Successes and challenges? 2.4 How do the national office and regional offices plan the division of responsibilities? What changes have there been over the project period in delegations to the regional offices? What are the mechanisms for coordinating between offices? What are the successes and challenges? 2.5 What changes have there been over the project period to PDHJs monitoring capacity? What do you monitor? (Probe school feeding program, detention centres, prisons) Remaining challenges? 2.6 What changes have there been over the project period to PDHJs investigation capacity? Remaining challenges? 2.7 How do the monitoring and investigation parts of the organisation coordinate? How do the national and regional offices coordinate? Remaining challenges? 2.8 How do you ensure maximum coordination between the human rights and good governance functions? 2.9 Who are the key institutions that PDHJ seeks to influence? What are the formal and informal processes to do this? What key findings and recommendations has PDHJ made to other institutions? How have those institutions responded? How do you support key institutions to do better next time? For each relevant institution what is different about PDHJ influence and relationship between the beginning of the project period and the end? How did the assistance seek to strengthen PDHJ influence? How did the assistance seek to strengthen PDHJ relationships? Does PDHJ publish the results of investigations? 2.10 What are some key examples of policy influence? What is different now about staff’s ability to advocate for policy change? What skills does PDHJ utilise in this policy advocacy work? What are your mechanisms of cooperation with key institutions (that would enable this policy advocacy)? What are the ongoing challenges internal or external for policy advocacy? Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XXI 2.11 Are your successes more on individual cases or on systemic issues? (Explore reasons) (probe on Use of Force) 2.12 How does PDHJ ensure gender mainstreaming in its work both internally and externally? Does PDHJ have a gender strategy? What are the successes and challenges in addressing gender issues in PDHJ’s work? 2.13 How effective is PDHJ in including disability in its work? 2.14 How does PDHJ ensure a victim orientation in its work? 2.15 How does PDHJ obtain its legal expertise? Is there an internal legal unit (Gabinete Apoiou Juridica) as there is in most Government departments? 2.16 What institutional processes are there for professional development of staff? 2.17 What changes have there been in the development of the PDHJ to function as an independent institution, over the project period? What are the remaining challenges? (Do you anticipate the need for any changes in the Organic Law?) 2.18 What changes have there been in how PDHJ approaches raising awareness of its mandate? How successful has this been? Remaining challenges? 2.19 What changes have there been in how PDHJ approaches raising awareness of human rights and good governance issues? 2.20 Does the current GoTL budget allocation provide you with adequate budget to implement the strategic plan? What do you do to advocate for adequate funding (question re UPMA)? How are budget areas prioritised? Are senior figures in GoTL committed to the future of PDHJ? 2.21 With this USAID/MFAT funding finished what are the future possibilities for donor funding? 2.22 Does PDHJ have the right human resources to implement your Strategic Plan? Are there any challenges in this area? 2.23 Does the Organic Law provide you with the flexibility you require? If not, how is this being addressed? 2.24 Can we obtain a copy of the 2015 tripartite capacity assessment? 3 Questions for PDHJ Regional offices 3.2 What are the key GoTL institutions you engage with at the regional office? Over the project period what changes have there been in engaging GoTL institutions as partners? Can you describe the division of responsibilities for GoTL engagement at national and regional office? Do you have MoU’s with key GoTL institutions How are those implemented? Successes and challenges? 3.3 What are the key civil society institutions you engage with at the regional office? Over the project period what changes have there been in engaging civil society institutions as partners? Can you describe the division of responsibilities for civil society engagement at national and regional office? Do you have MoU’s with civil society institutions How are those implemented? Successes and challenges? 3.4 How do the national office and regional offices plan the division of responsibilities? What changes have there been over the project period in delegations to the regional offices? What are the mechanisms for coordinating between offices? What are the successes and challenges? 3.5 Does the current GoTL budget allocation at the regional office provide you with adequate budget to implement your work? What do you do to advocate for adequate funding for the regional office? 3.6 Do you have the adequate resources to run the regional offices (phone/internet/ access to e￾CMS)? 3.7 What changes have there been over the project period to the regional office PDHJs monitoring capacity? What do you monitor? (Probe school feeding program, detention centres, prisons) Remaining challenges? Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XXII 3.8 What changes have there been over the project period to the regional office PDHJs investigation capacity? Remaining challenges? 3.9 How do the monitoring and investigation parts of the organisation coordinate? How do the national and regional offices coordinate? Remaining challenges? 3.10 How do you ensure maximum coordination between the human rights and good governance functions? 3.11 Who are the key institutions that PDHJ seeks to influence at the regional level? What are the formal and informal processes to do this? What key findings and recommendations has PDHJ made to other institutions? How have those institutions responded? How do you support key institutions to do better next time? For each relevant institution what is different about PDHJ influence and relationship between the beginning of the project period and the end? 3.12 What are some key examples of policy influence? What is different now about staff’s ability to advocate for policy change? What skills does PDHJ utilise in this policy advocacy work? What are your mechanisms of cooperation with key institutions (that would enable this policy advocacy)? What are the ongoing challenges internal or external for policy advocacy? 3.13 Are your successes more on individual cases or on systemic issues? (Explore reasons) (probe on Use of Force) 3.14 How does PDHJ ensure gender mainstreaming in its work both internally and externally? Does PDHJ have a gender strategy? What are the successes and challenges in addressing gender issues in PDHJ’s work? 3.15 How effective is PDHJ in including disability in its work? 3.16 How does PDHJ ensure a victim orientation in its work? 3.17 How does PDHJ obtain its legal expertise? Is there an internal legal unit (Gabinete Apoiou Juridica) as there is in most Government departments? 3.18 What institutional processes are there for professional development of staff? Can you tell me more about how this works for the regional office? 3.19 What changes have there been in the development of the PDHJ to function as an independent institution, over the project period? What are the remaining challenges? (Do you anticipate the need for any changes in the Organic Law?) 3.20 What changes have there been in how PDHJ approaches raising awareness of its mandate? How successful has this been? Remaining challenges? 3.21 What changes have there been in how PDHJ approaches raising awareness of human rights and good governance issues? Questions for Civil society organisations 3.1 What has been your engagement with PDHJ over the last xx years. Has this been a formal or informal engagement? (MoU?) How often would you meet? 3.2 Over the project period what changes have there been in engaging CSOs as partners? 3.2 In what areas do you work together with PDHJ? (monitoring, investigation, awareness raising, training, focal points, policy advocacy, case advocacy). 3.3 Can you provide more detail on each of these areas? Specific examples? 3.4 How effective is PDHJ on policy leadership? 3.5 How effective is PDHJ in including gender issues in its work? 3.6 How effective is PDHJ in including disability in its work? 3.7 How does PDHJ ensure a victim orientation in its work? 3.8 How effective is this collaboration? What have been the successes and challenges of this collaboration? 3.9 How could this collaboration be more effective? Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XXIII 3.10 From your perspective what are the key successes and challenges for PDHJ? How could PDHJ be more effective? 4 Questions for GoTL institutions? 4.1 What has been your engagement with PDHJ over the last xx years. Has this been a formal or informal engagement? (MoU?) How often would you meet? 4.2 Over the project period what changes have there been in engaging GoTL institutions as partners? 4.3 Does your institution engage with PDHJ at both the national and regional level? Can you tell me more about that? 4.4 In what areas does your institution work together with PDHJ? (monitoring, investigation, awareness raising, training, focal points, policy advocacy, case advocacy). Can you provide more detail on each of these areas? Specific examples? 4.5 How effective is PDHJ in including gender issues in its work? 4.6 How effective is PDHJ in including disability in its work? 4.7 How does PDHJ ensure a victim orientation in its work? 4.8 How effective is this collaboration? What have been the successes and challenges of this collaboration? 4.9 How could this collaboration be more effective? 4.10 From your perspective what are the key successes and challenges for PDHJ? How could PDHJ be more effective? Hau Meni & Associates Pty Ltd and Fundasaun Mahein Final Assessment - Strengthening Decentralization in the Office of the Ombudsman for Human Rights and Justice - Phase II Project. XXIV Annex 6 Assessment Team The evaluation team was comprised of: • Dr Bu Wilson, Director/Principal Consultant with Hau Meni & Associates Pty Ltd, an Australian limited liability company specialising in design, evaluation and research. She has a PhD in law, extensive experience in Timor-Leste, and subject matter expertise in evaluation, governance, law and justice sector development and ending violence against women; and • Mr Abel Amaral, Interim Deputy Director with Fundasaun Mahein, a well -established Timor￾Leste NGO specialising in monitoring the development of the security sector. He has a law degree and extensive experience in human rights, governance and security sector monitoring and reform. The respective roles of the team members were as follows: The International Consultant, Dr Bu Wilson: • Served as the team leader; • Developed assessment methodology, planned the work and carried out field visits; • Conducted desk-top reviews and key informant interviews in conjunction with the National consultant, and analysed the results; • Organized in- and out-brief to USAID; • Presented assessment findings to NZMAFT, PDHJ and other stakeholders; • Wrote the assessment report; and • Had responsibility for submitting the draft and final assessment report to USAID/Timor-Leste. The National Consultant, Mr Abel Amaral: • Developed the schedule for the national and regional office visits in coordination with USAID/Timor-Leste and PDHJ; • Contacted key informants, GoTL, and other stakeholders to set-up visits and interviews; • Conducted key informant interviews on his own, and in conjunction with the team leader, and assisted in analyzing the results; • Provided input regarding the human rights, good governance, and justice sectors specific to Timor-Leste; and • Reviewed document drafts Additional logistical, research and review support was provided by Fundasaun Mahein (Joao Almeida, Sancho Xavier, Veriato Amaral Cardoso and Jordan Lee Borgman) and Hau Meni & Associates Pty Ltd (Dr Ingvar Anda).