CSJP Interim Performance Evaluation & Impacts Effectiveness Assessments Page Index Introduction Executive Summary 1 i Program Evaluation 6 Project Reports 16 Court Services • A1 - Re Aga Boswa (RAB) • A2 - Integrated Case Flow Management System’s Extended Pilot Project (ICFMSEPP) Sexual Offences and Community Affairs Unit • B1 - Public Awareness Campaigns (PACs) • B2 - Case Managers’ Extended Pilot Project (CMEPP) • B3 - Victim Assistance Officers’ Extended Pilot Project (VAOPP) • B4 - Sexual Offences Multi-Disciplinary Training Manual (SOMM) • B5 - Domestic Violence Multi-Disciplinary Training Manual (DVMM) • B7 - Uniform Protocol for Victim Management (UPVM) Justice College Capacity Building • C2 - Impact Assessment Project (IAP) • C3 - Visioning and Strategic Planning (VSP) • C4 - Magistrates’ Mentorship Program (MMP) Outsourcing the Management of Monies in Trust • D1 - Management of Monies in Trust (MMIT) Transforming the Judiciary • E1 - Judicial Officers Association of SA’s International Association of Judges (JOASA’s IAJ) • E2 - South African Judges Symposium (SAJS) • E3 - Leadership and Management Development Program for Female Judicial Officers (LMDPFJO) • E4, E5 - Towards a Unified South African Judiciary (USAJ) - Single Rule Making Authority (SRMA) • E6, E7, E8 - Interim Advisory Council/Board (IAC/B) - Restructuring and Transformation Committee (RTC) - Judicial Education and Training Committee (JETC) • E9 - Development and Implementation of the Prosecutor-Led Plea-Bargaining Training Program (DIPBTP) • E10 - Aspirant Judges Training and Development Program (AJTDP) • E11 - Training the (Trainer’s) Trainer (TTT) • E12 - Newly Appointed Judges’ Orientation Program (NAJOP) • E13 - Continuing Education for Judges Training Program (CEJTP) Capacity Building within the DOJCD • F1 - Establishing DOJCD Program Management Support Unit (PMSU) • F2 - Forensic Audit Unit External Anti-corruption Initiative • G1 - Specialised Commercial Court Centre 16 27 33 42 46 55 57 64 70 73 74 82 84 86 89 95 97 102 103 105 107 109 111 115 119 Appendices 126 Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page i Abbreviations and Acronyms Used AJTDP = Aspirant Judges Training and Development Program BAC-SA = Business Against Crime South Africa CBO = Community Based Organisation CEO = Chief Executive Officer CFO = Chief Financial Officer CJS = Criminal Justice System CJSP = Criminal Justice Strengthening Program CEJTP = Continuing Education for Judges Training Program CMEPP = Case Managers Extended Pilot Project CPSI = Centre for Public Service Innovation CSSC = Court Services Support Centre DCS = Department of Correctional Services DG = Director General DIPBTP = Development and Implementation of the Prosecutor-led Plea-bargaining Training Program DOJCD = Department of Justice and Constitutional Development DSD = Department of Social Development DVMM = Domestic Violence Multi-Disciplinary Training Manual “Ease”/EASE = Effective, Accessible, Swift & Efficient Justice FAU = Forensic Audit Unit FS = Free State HR = Human Resource IAP = Impact Assessment Project IAJ = International Association of Judges IAC/B = Interim Advisory Council/Board ICFMS = Integrated Case flow Management System ICFMSEPP = Integrated Case Flow Management System’s Extended Pilot Project JETC = Judicial Education and Training Committee LMDPFJO = Leadership and Management Development Program for Female Judicial Officers MD = Managing Director MMIT = Management of Monies in Trust MMP = Magistrates’ Mentorship Project NAJOP = Newly Appointed Judges’ Orientation Program NGO = Non-Governmental Organisation NPA = National Prosecuting Authority PAC’s = Public Awareness Campaigns PIC = Program Implementation Committee PMSO = Program Management Support Office PMSU = Program Management Support Unit POC = Program Operational Committee PPP = Public Private Partnership RAB = Re Aga Boswa RFP = Request For Proposal SAPS = South African Police Service SCCCI = Specialized Commercial Crimes Courts Initiative Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page ii SOCA = Sexual Offences and Community Affairs SOMM = Sexual Offences Multi-Disciplinary Training Manual SRMA/B = Single Rule Making Authority/Board TA = Transaction Advisor TTT = Training the (Trainer’s) Trainer USAID = United State Agency for International Development USAJ = Unified South African Judiciary UPVM = Uniform Protocol for Victim Management VAOPP = Victim Assistance Officers Pilot Project VAOEPP = Victim Assistance Officers’ Extended Pilot Project VSP = Visioning and Strategic Planning Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page iii THE INTERIM PERFORMANCE EVALUATION AND IMPACTS AND EFFECTIVENESS ASSESSMENT OF THE CJSP: EXECUTIVE SUMMARY The Criminal Justice Strengthening Program (hereafter referred to as the program) was established as a strategic partnership initiative between the DOJCD, USAID and BAC SA. The program aims to support and strengthen the capacities of the DOJCD and its strategic alliances, including the NPA, and thereby contribute towards the achievement of the DOJCD’s objectives (i.e. to make the SA Criminal Justice System effective, efficient, swift and accessible; justice with EASE). The objectives of this report are to present the findings of the evaluation carried out by the service providers (Decipher Consulting, Goals and Performance Analysts (GAP) and Linkages Development Agency (LDA)). The project objectives were stated in the RFP as “to develop a rational evaluation and assessment instrument / tool with a view to evaluate and assess the overarching impacts and effectiveness of the Criminal Justice Strengthening Program, its projects, approaches, targets and outcomes as well as its potential towards sustainability and institutionalization within the DOJCD”. 1. Methodology Employed In order to meet the requirements of the RFP, six key components were covered in the assessment: · Understanding whether the current design of the projects (approach, targets, intended outcomes) is indeed aligned with the DOJCD vision and mission, and to what extent the projects need to be tailored to be better aligned · Evaluating the performance of the projects against their current design and results framework by assessing progress (input indicators) and the achievement of set objectives, targets and other performance indicators (output indicators), i.e. what are the planned processes/activities, whether the projects are carrying out planned activities, and the extent to which they have achieved their stated objectives · Evaluating the current design for its effectiveness (goals and services generated by the project/s) · Evaluating the current design for its impact (impact indicators) · Evaluating the current design for its relevance, currency and sustainability of its offerings · Evaluating the current design for its ability to be institutionalised, understanding what the key success factors and potential challenges for institutionalisation are, and what lessons have been learned that can be applied among the projects to improve program implementation. The approach adopted to gather the required information is summarised in the diagram below. • Discuss requirements with key PMSO representatives • Analyse existing information and data to inform tools design (i.e. interview guide and survey questionnaire) • Design, review and refine stakeholder memorandum and evaluation tools • Identify and select people and entities from which data and information will be gathered • Schedule appointments for interviews with selected individuals Key activities • Send out stakeholder memorandum and survey questionnaires • Conduct individual interviews • Perform focus group and telephonic interviews • Collect objective performance data where available (statistics such as no. of cases, success rates, etc.) • Begin analysis of data as it becomes available • Develop preliminary feedback on findings Design of assessment/ evaluation tools Analysis and report writing • Perform analysis of data and information • Provide feedback to the PMSO • Synthesize findings into a report for each project and for the program as a whole • Distribute draft report to the PMSO and draft project reports to sub-program sponsors / project managers for content review • Finalise and submit first draft report Week 1 Weeks 2 & 3 Data gathering and evaluation Weeks 4 & 5 Review and refinement • Obtain review inputs from PMSO and sub-program sponsors/project managers • Refine report and submit second draft • Obtain review inputs • Refine and submit final report Week 6+ Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page iv The approach is discussed in more detail in the Introduction section. 2. Evaluation Limitations A number of limitations relating to the conduction of this evaluation exist which must be borne in mind by the reader when considering the findings. These include the following: · The Request for Proposal required this evaluation to be conducted within 22 days · A few of the CJSP projects (such as the SOCA and Court Services projects) have strong interdependencies with other Departmental (and other organizational) programs and initiatives but the evaluation of the non￾CJSP projects in these broader programs and the broader programs themselves was not within the scope of this evaluation · In some cases only a limited number of people were available for interviews within the given timeframe 3. Scope of evaluation of projects aimed at establishing committees or arranging conferences and workshops All projects as indicated in the RFP were originally evaluated on a similar basis, covering all aspects as stated in the Methodology section (1) above. However, on evaluation it became apparent that a number of the projects were of such a nature that they should not be strictly evaluated against all the aspects as per the stated methodology. These projects were typically of the nature of a committee to be established or a conference or workshop to be arranged, and most of them have been completed. Although these projects form an integral part of the CJSP program, and are critical components for achieving the DOJCD’s vision and mission, due to their nature they have been evaluated only on a sub-set of the aspects stated in the methodology. Some of the aspects which were typically not evaluated, or evaluated on a limited basis, are: · Project Management, Monitoring and Reporting · Progress against Key Performance Measures · Skills Development and Change Management · Current Project Challenges, Dependencies, Risks and Issues · Sustainability 4. Key findings and recommendations The Program is performing well against its aim of supporting and strengthening the capacities of the DOJCD and its strategic alliances. The CJSP-PMSO as a support structure is providing highly value-adding assistance, guidance and transfer of skills to DOJCD staff. The governing bodies of the Program (the PIC and POC) are well constituted with representation across all senior levels within the DOJCD and other participating departments right to the most senior level. Most of the members of these governing bodies interviewed have stated their commitment and remain enthusiastic and supportive of the objectives of the Program. These aspects combined indicate that the Program has been a highly effective transformation mechanism for improvement of effectiveness, efficiency, swiftness and accessibility in the DOJCD and its strategic alliances, and has the potential to continue to do so. Visible indicators are showing that improvements are already being achieved in a number of areas. All projects within the Program are relevant to the intended Program objectives and are spread across almost all the key capability areas of the DOJCD that need to be developed for achieving its objectives. Project capacity, which is mainly provided through DOJCD staff, is generally good, with most projects adequately staffed with appropriate skills. A general sense of the need to measure progress and a willingness to improve current actions in this regard was expressed by many project managers/coordinators. This is a positive step forward for the DOJCD coming from a history of limited measurement of performance. A large number of projects are well on track, and in two cases (Re Aga Boswa and MMIT) internal funding is being committed and DOJCD staff already involved on a full-time basis or in the process of being trained or mentored. Quite a few projects have now reached the stage where they are becoming an integral part of the DOJCD, especially those relating to training. A number of projects are however at risk in terms of sustainability, thereby running the risk of not achieving eventual or continued impact. The three key areas of risk are lack of funding/institutionalisation, legislation and commitment by senior management. To achieve sustainable success there are a number of key challenges that will have to be addressed going forward. These include the following: Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page v · To achieve adequate handover of the capabilities provided by the CJSP-PMSO to an appropriate structure within the DOJCD. This is currently in progress through the PMSU, and the full implementation of the PMSU is therefore a high priority, · To ensure consistent and continued skills transfer and training for DOJCD staff involved in the management and roll-out of these projects, · To ensure that the integrated view of the sub-programs and projects within the Program is fully understood by all stakeholders and participants in the management of the program and its sub-programs and projects, including those not on the Program governing bodies, so that its implications are identified, its implementation planned for and responsibilities assigned to specific people for management of interdependencies and synergies, · To ensure more hands-on participation of all governing body members and project managers to be continuously familiar with and actively managing integration requirements and interdependencies of all projects within the Program, · For the DOJCD and NPA to plan well ahead of time for continued internal funding (especially since the financial years of external funders and the DOJCD typically differ by six months) and · To develop leading indicators of performance and performance management mechanisms that are more specific to each sub-program and project (where these are not yet developed) so that the performance of each project in its own right can be more clearly tracked and so that early warning signs will be visible and can be pro-actively managed. Although there are some really good performers, project performance is varied and some are at high risk of not achieving their objectives. The following findings and recommendations are not necessarily applicable to all projects (i.e. they are not indicators of pervasive problems), but are aspects that have been found in one or more projects that could be improved to increase the likelihood of sustainable success. It also needs to be noted that these findings are made from a program viewpoint, not from a program management viewpoint. This means that many of the issues indicated are not necessarily within the mandate of the Program management and/or the project managers. There are many stakeholders involved, and quite a few projects impact on areas within the DOJCD and/or within its strategic partners where other initiatives or programs are also being implemented, and have therefore often become dependent on the planning, management and resource allocation done for those initiatives or programs. Although the Program management and project managers can provide guidance and support some of these issues can only be addressed through decisions and actions of senior management of the DOJCD and its strategic partners. a. Governing bodies PIC The PIC conceptually has a sound purpose: for senior members of stakeholders to direct and influence policy development that will support Program implementation. The record however shows that attendance has been poor. The PIC has a vital role to ensure that project integration issues and interdependencies are thoroughly understood so that policy development is indeed supportive of these issues. This can only be achieved if all members of the PIC consistently participate through attendance of PIC meetings, thereby actively fulfilling the roles conceived for them going forward and providing the necessary leadership for all other participants to be highly engaged and motivated. POC The POC is structured as a value team focused on area prioritization, alignments, resource allocation and expenditure monitoring. The POC provides operational leadership and support towards the effective implementation of projects and the management of interdependencies. Some sub-program sponsors however tend to focus only on projects which they have direct interest in rather than performing an overall integration and oversight role. The record shows that attendance of POC meetings is also poor, leading to inadequate management of project interdependencies and insufficient collaboration to achieve effective Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page vi transformation. The leadership should take more accountability and ensure the fulfillment of roles by individual members, else the credibility of and value added by this structure will be compromised. CJSP-PMSO There exists a high level of awareness of and appreciation for the CJSP-PMSO’s work across all projects of the Program. Competent and efficient project management and technical support is being provided, which in most cases is a key enabler of the success of projects. Sustained involvement of an external entity such as the PMSO in the implementation of change projects within an organisation (such as the DOJCD) is however not desirable, since most of these capabilities need to become institutionalized. This is a clear indication that a support structure such as the PMSU, which is an internal component of the DOJCD, needs to be established as a permanent feature during the transformation of the DOJCD and its strategic partners as a priority. In addition the CJSP-PMSO is stretched thinly at the senior, leadership level due to the number of sub-programs and projects and the extensive scope of the program. The CJSP-PMSO (or similar structure such as the PMSU) could be even more effective if more senior staff are added with the ability to engage at the most senior levels within the DOJCD and its strategic partners and provide leadership and mentorship to project managers. The PMSU (which is intended to be a DOJCD structure to continue the role of the CJSP-PMSO) should be resourced with experienced, professional, full￾time (as far as possible) project managers with the CJSP-PMSO providing only CJSP-specific information and knowledge transfer. b. Objectives The Program has been designed to support and strengthen the capacities of the DOJCD and its strategic alliances, including the NPA, and thereby contribute towards the achievement of the DOJCD’s objectives (i.e. to make the SA Criminal Justice System effective, efficient, swift and accessible; justice with EASE). Headline indicators and targets have been established for the DOJCD by the Program. These indicators and targets are reproduced here as given in the RFP for this evaluation: ƒ The realization of meeting the targeted Increases/Improvements in the following categories: ß Increase Conviction Rate by 10% ß Increase Public Confidence by 10% ß Increase Customer Focus by 10% ß Increase Public Awareness & Public Confidence by 25% ß Increase Self Confidence of the Public by 25% ß Increase Self Trust in the Criminal Justice System by at least 5% ß Increase Average Court Hours from the current 3 hours to 6 hours. ß Improve the Quality of Trails and Sentences by 10%. ß Improve the Morale and Motivation of Staff by 50%. ƒ The realization of meeting the targeted Reductions/Decreases: ß Decrease Case Cycle Time from 9-18 to 6-9 months ß Decrease Case Backlogs by 10% ß Decrease Secondary Victimization by 10% ß Decrease Sexual Offences and Domestic Violence by 10% ß Decrease Court Rolls by 10% ƒ Other Headline Targets include inter alia: ß Increasing Court Accessibility to the Public by 50% ß Developing Managerial Capacity within the DOJCD ß Building Confidence in Magistrates and Prosecutors in rural areas by 10% ƒ The realization of the following objectives: ß Repositioning of the Justice College, the training arm of the DOJCD ß The Development of an Assessment Tool of the Justice College ß Conducting an Impact Assessment of the Justice College Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page vii ß Appointment of a Transaction Advisor for the MMIT PPP ß Commissioning of a Feasibility Study & Options Analyses on the Monies In Trust ß Capacity Building for the DOJCD Staff (Training & Development ß Transforming the South African Judiciary and developing capacity to lead and sustain an effective justice system in South Africa ß The Development and Implementation of the Plea-Bargaining Training Program in order to appraise and immerse prosecutors in alternative forms of justice with a view to reduce case backlogs in the South African Courts and the DCS ß To identify, select, train and develop Aspirant Judges in South Africa with a view of increasing a Pool from which Judges can be appointed ß Developing and implementing the Training the Trainer Program for Judges ß Developing and implementing the Newly Appointed Judges Orientation Program ß Developing and implementing Continuing Education for Judges Training Program ß Establishing a Program Management Support Unit to help capacitate and sustain efforts within the DOJCD ß Establishing an internal Forensic Audit Unit within the DOJCD and develop capacity thereto ß Supporting BAC SA’s external anti-corruption efforts, i.e. the Specialised Commercial Crimes Court Centres. The indicators as reproduced above are comprehensive, and if the targets as stated above are achieved the DOJCD will have taken a significant step towards the realization of its vision and mission. It needs to be noted however that it would be unrealistic to expect that the Program on its own will result in the achievement of the stated targets, since most of these indicators cannot be changed through any one project, but require a combination of projects to start changing significantly, and in many cases also depend on a variety of other non-project influences, structures, processes, decisions and actions. It is therefore risky to use only these headline indicators as the measure of success of the Program or any of the specific sub-programs or projects within the Program. Although there are sub-programs and projects that have additional objectives and indicators specific to that sub-program or project (and quite a few that are very comprehensive) we have not found this on all projects. This makes it difficult to monitor and evaluate the effectiveness and success of a specific project, since the headline DOJCD targets will in most cases only change once a number of other related projects have also been implemented and rolled-out for a sufficient period of time. We therefore recommend that the Program management develop clear objectives and indicators specific to each sub-program and project where these have not yet been developed. These objectives and indicators need to be appropriate for the current phase of each project, and enable monitoring of the effectiveness and appropriateness of each sub-program and project on its own. This will significantly aid the Program management to steer projects and allocate resources and significantly aid the evaluation of the sub￾programs and projects of the Program. Some instances of inconsistency in reproduction of project specific objectives were found. This may be a result of different stakeholders producing their own versions of project plans, business plans and reports. The effect of this could be that stakeholders do not share the same understanding of what the project specifically needs to achieve and are not able to properly monitor the progress and effectiveness of the project. We therefore recommend that the CJSP-PMSO urge project managers and sponsors to channel all project documentation through the CJSP-PMSO to ensure consistency. Other stakeholders should be discouraged from producing their own versions of project related information. Where this is not possible, the Program management should be copied on all correspondence and documentation produced relating to the project so that the content can be quality controlled and consistency ensured. Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page viii c. Communication All the projects within the Program are highly relevant and various notable successes are apparent through our evaluation. It was however noted that these successes are not generally known throughout the DOJCD. Most staff members are only aware of the achievement of projects they are specifically involved with. It needs to be noted that we have not interviewed or surveyed staff not involved in any project of the Program, and therefore cannot comment on their awareness of the Program and its effects. Communication of successes can be a powerful tool to build pride and enthusiasm, gain support from all levels of staff, increase the pace of change and attract voluntary participation in change programs. We would therefore recommend that a comprehensive communication strategy is developed and a dedicated function added to the PMSU (or similar DOJCD unit) to regularly communicate progress and successes to the whole of the DOJCD and relevant strategic partners. Integration and sharing of learnings on different projects should be communicated regularly beyond the Program governing bodies, so that making the same mistakes on different projects can be avoided and best practices can be quickly duplicated throughout the organisation. d. Engagement of all stakeholders Projects have had the highest success where a strong sense of commitment and ownership amongst project owner, sponsor and manager/coordinator exist not only in hearts and minds, but also in action through regular and substantial engagement in the planning, coordination, monitoring, steering and resolving of issues of the project. This requires a substantial amount of time to be invested by all stakeholders on a project. All project participants therefore need to have sufficient time freed up to dedicate to project activities. Sponsors need to dedicate sufficient time to actively drive projects, provide support and motivation and liaise with other role players to manage interdependencies and risks, ensure effective collaboration and resolve issues. e. Project monitoring and measurement The headline indicators of the DOJCD as developed by the Program for the most part are lagging indicators, i.e. they measure the eventual outcome and effectiveness of the projects within the Program and typically would only start changing once a project or collection of projects have been fully implemented for a sufficient period of time. Many of these indicators are also not influenced solely by the Program sub-programs and projects, but dependent on a variety of other influences, structures, processes, decisions and actions. It is therefore risky to use only these headline indicators as the measure of success of the Program, or any of its sub-programs or projects, especially since they do not provide early indication of progress and effectiveness, and will only really provide an indication of success once projects have been fully implemented and institutionalized for a sufficient period of time. We therefore recommend that, although the Program projects have project plans and milestones which can be tracked to monitor progress, the Program management develops clear leading indicators (i.e. measures that will already start changing during implementation) and targets specific to each sub￾program and project where these are not yet developed (since there are quite a few projects with very comprehensive leading indicators already). These indicators need to be appropriate for the current phase of each project and enable early and continuous monitoring of the effectiveness and appropriateness of each sub-program and project. This will significantly aid the Program management to steer projects and allocate resources and significantly aid the evaluation of the sub-programs and projects of the Program during each of their lifecycle phases. f. Organisational structure and leadership Integrative views for the Program has been developed by the management of the Program over the last year, a plan and methodologies for better integration of the sub-programs and projects have been proposed and to some extent implemented. An example of this was to develop Re Aga Boswa as an integrative project. This has been successful in that many of Re Aga Boswa’s successor projects have been well integrated into one coordinated approach to design, develop, detail, streamline and operationalize a Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page ix Decentralized Court Support Services Delivery Model. Re Aga Boswa is expected to further integrate a number of projects in the Program. The Re Aga Boswa example included, there are a number of sub￾programs and projects within the Program that have a strong interdependence or can benefit from exploiting synergies. It is not necessary, and also not feasible to always integrate all related projects into one, but in such cases the interdependencies or synergy potential should be explored and well understood and managed at all levels in the project teams. This is especially important where projects span across, or affect, multiple departments (inside and outside the DOJCD) and/or different components of the judiciary. The Program management has recently laid out an integration plan to governing members of the Program. It seems however that this integrative view and project interdependencies are not yet sufficiently understood by project participants, and those affected by projects, outside the governing bodies. The separation of powers between the DOJCD, NPA and the Judiciary and the historical separation of the different components of the judiciary also complicates coordination and cooperation. Other than individual interactions between heads of the DOJCD, NPA and judiciary and interaction at ministerial level, the governing bodies of the Program and project manager impact teams are currently the only place where project interdependencies can be identified and managed. We therefore recommend that, where these do not yet exist, interdependencies and collaboration requirements should be made explicit in project and business plans and responsibilities should be assigned to specific project team members for identifying and resolving issues relating to these interdependencies, and sufficient time dedicated to these issues at Program governing body meetings. This should include cross-project planning to ensure that activities are correctly sequenced between interdependent projects and resources optimally allocated. The Program governing bodies should ideally be complemented by the PMSU (or other interim structure) to ensure central management of cross-cutting initiatives within the DOJCD and the Judiciary. We realize that this falls outside the Program’s mandate and responsibility, but a higher level concerted approach by the DOJCD, the NPA and the Judiciary to institutional transformation is required to ensure ongoing successful project institutionalisation and delivery in the future, particularly post the CJSP Program’s existence. A deliberate, pro-active change management approach needs to be developed and implemented to fast-track cooperation between independent and recently integrated bodies. The regular communication recommended in c. above should also be utilized for creating awareness of interdependencies and inducing cooperation throughout the DOJCD, NPA and Judiciary. g. Funding and sustainability Lack of planning well ahead of time and commitment for full project lifecycle would place successful outcomes at risk. This includes funding and the development and retention of skills and capabilities to ensure continued progress. At present, the DOJCD and NPA seems to rely heavily on donor funding for change programs and new core capabilities. USAID funding is spread across the whole CJSP program with little counter-party or other funding contributions being actively sourced to support ongoing needs. The difference in financial year-ends of the government and donors imply that continued funding needs to be budgeted for by the relevant departments at least six months before the financial year-end of the last year of funding by a donor. The MTEF planning and budgeting process provides a clear mechanism where this can be done. However, there seems to be reluctance by the DOJCD to use this mechanism, with only some large projects (RAB and MMIT) currently linked to the MTEF budget. A task force has been commissioned by the Minister of Justice and Constitutional Development to identify human resource requirements for sustainable service by the justice system. If a human resource budget increase resulting from this process does not sufficiently cover further funding of people involved in the CJSP projects and related new core capabilities, the sustainability of those projects not yet linked to the MTEF budget would be at risk if alternative sources of funding are not sourced in the next 6 months, or budgeted for in the next MTEF budget. Departmental budgeting falls outside the mandate of the CJSP. It is therefore important that the responsibilities of project sponsors relating to project institutionalization be clearly communicated, including their role in sourcing funding themselves and/or the process and requirements for ensuring inclusion in organizational budgets. Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page x h. Overview of notable successes, outcomes and indicators of effectiveness to date An overview of some of the notable successes, outcomes and indicators of effectiveness to date are reproduced here from the individual project reports: · RAB: Re Aga Boswa Court Managers have been appointed in all 12 sub-clusters of the KZN region. Delegation of authority for most duties has been successfully implemented. Court managers feel empowered to manage the administrative functions and decision making of the courts. Court managers perceive an improvement in turnaround on administrative tasks. · PACs: Outstanding results have been achieved with the school campaigns since the implementation of the Speak Out Industrial Theater. The quantitative output measures reflect an exceeding of expectations. These results, counted at project level, are the immediate effect of thorough planning and preparation of the school based activity and can be directly attributed to the approach used, that of targeting schools, providing the project with an automatic captive audience and also to the enthusiasm of the teachers who were always willing to accommodate the play in their schools, compelled by the knowledge that children are the silent victims of abuse within their families and communities. Schools have become aware of the prevalence of sexual and other forms of abuse (verbal and emotional abuse) and the need to address these. The schools are requesting that (pending the availability of funds) the play be brought to the schools at least twice each year. The demand is high. The school awareness programme has made significant differences to the lives of the children who were helped by the industrial theater to break free from their own silence and reported sexual abuse. These children’s lives have been changed for the better. To them the awareness program has made both an immediate and a long-term impact. The PACs together with the other project components of SOCA have led to the recognition of SOCA as the most innovative public programme by Standard Bank, and being awarded a National Centre for Public Service Innovation (CPSI) award. · CMEPP: The extended pilot project has been effective in increasing the average court hours and conviction rates as well as reducing case cycle times as a result of the multi-disciplinary process adopted, the commitment demonstrated by case managers and key stakeholder buy-in. · VAOEPP: At the level at which the project objective is pitched, i.e. appointing a number of VAOs, effectiveness of the project is significant as the objectives of appointing the officers were met within reasonable timeframes. At the level of “safeguarding” the interests and needs of the victims, effectiveness is significant from the analysis of the job performance grid. Unexpected consequences were a receipt of donations of Teddy Bears by the project from the United Transport Industries and Daimler Chrysler as well as National and International Publicity for the Mdantsane Centre. Effectiveness is evident in that the assistants are serving victims in ways that meets the informational, security and emotional needs of the victims and serves to meet the requirements that facilitate meeting the ends of justice: with perpetrators convicted and secondary victimization reduced, through safety plans. Conviction rates have increased, women are utilizing the centres, victims have been removed and placed in places of safety, and victims have been prepared for court – evidence of services being rendered and victims being serviced. · DVMM: Immediate impact and effectiveness is that the manual has been developed and exists, although the process still has to be completed with regards to accreditation of the manual. · IAP: As a result of the Impact Assessment the College has at its disposal concrete recommendations and suggestions on how to restructure, improve and reposition itself. It also now has a tool that will be used to assess the impact of its offerings. · MMP: The first workshop was successfully carried out. From the first workshop, the overall perspective is that the workshop has empowered the participants to a great extent, giving them practical skills to be able to mentor less experienced colleagues. The workshop content is relevant and comprehensive. The participant magistrates feel the workshops have made a significant difference in their capabilities, steering them from a position of total unpreparedness to being better prepared and better informed to tackle the job. Immediate impact will be better measured after the magistrates have started the mentorship. Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page xi · MMIT: The Feasibility study has been completed and the Project Team is awaiting approval of the Options Analyses Report before it can secure Treasury Authorization Level I and proceed to the next stage. · JOASAsIAJ: The two primary milestones were to plan and hold the Conference. These were completed and the Regional African Group of the IAJ Conference was successfully held on the 13th to 16th June 2003 at the Birchwood Executive Hotel, Johannesburg. Three publishable papers also resulted from the Conference. · SAJS: The symposium’s milestones were successfully achieved and increased general levels of awareness and understanding of the issues within the judiciary. The needs and outputs expressed at the symposium have been translated into twelve tangible projects that collectively contribute to the transformation of the Judiciary. · LMDPFJO: Delegates rated the Conference as very good to excellent in terms of administrative and logistical arrangements as well as the quality of presenters. A Working Group on Women in the Judiciary has been established (the first meeting was held on 20 August 2004 consisting of executive committee and provincial representatives). Standing committees are to be set up to deal with specific issues identified. 75 Additional people joined the SA Chapter of the IAWJ, the SA office was established at the Justice College and a website layout is now in place. · IAC: On balance it is our opinion that the IAC was successfully implemented as a small and cost effective project with senior management buy-in focused on establishing a new National Justice College of South Africa. · AJTDP: Currently, 24 Aspirant Judges have been trained and 16 of them are appointed as Acting Judges. Feedback received from Trainees interviewed indicates that the project is effective and immediate impact is evident. · TTT: Through this project the Judiciary now has two trainers who are qualified to train other trainers. · FAU: The Forensic Audit Unit is already impacting fraud and corruption management within the DOJCD, despite the delays in project progress due to organizational issues it is experiencing. · CCCI: Plans for the Durban, Cape Town and Port Elizabeth Specialised Commercial Crimes Court Centres are in place. Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page xii i. Overall evaluation findings The evaluation findings have been compiled as separate reports for each of the twenty eight CJSP projects, as well as an additional report covering the program itself, including the governance structures. These reports comprise the remaining sections of this document. By way of summary, the table below contains a high-level evaluation of each project in terms of project relevance, performance, measurement, capacity, sustainability and impact. In rating each project as good (dark grey colouring), average (light grey colouring) or poor (white colouring) the following sub-factors were considered: · Project Design – Clearly defined project objectives, commonly expressed and understood – Objectives aligned to project purpose – Objectives specific and measurable (i.e. can develop tangible measures from stated objectives) · Project Performance – Project initiated/begun – Project met deadlines and is delivering/has delivered – Project carefully monitored to ensure successful delivery · Performance Measurement – Performance measures clearly defined – Measures base-lined – Measurement practiced regularly to meet defined targets · Project Capacity – Human resources involved sufficient to meet needs – Relevant and appropriate skills for the project were/are available · Sustainability – Project is at risk (white shading); or – Project has some issues to address (light grey shading); or – Project is sustainable (dark grey shading) (Note sustainability includes view of extent to which project will be embedded and/or institutionalised within the DOJCD going forward i.e. covers more than just the scope of the CJSP’s role) · Impact – Definite, visible/measurable impact (dark grey shading); or – Limited impact (i.e. visible/measurable to an extent) (light grey shading); or – No impact or too early to measure impact (white shading) The overall findings are graphically depicted as follows (see next page): Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page xiii Project Re Aga Boswa Integrated Case Flow Mgmt System Public Awareness Campaigns Case Managers Extended Pilot Project Victim Assistance Officers Extended Pilot Sexual Offences Multi-Disciplinary Training Manual Domestic Violence Multi-Disciplinary Training Manual Uniform Protocol for Victim Management Impact Assessment Project Magistrate's Mentorship Project Visioning and Strategic Planning Management of Monies in Trust Judicial Officers Association of SA International Association of Judges SA Judges Symposium Leadership and Mgmt Development for Women in the Judiciary Unified SA Judiciary Committee Single Rule Making Authority Interim Advisory Council/Board (IAC/RTC/JETC) Development and Implementation of the Prosecutor￾led Plea Bargaining Training Programme Aspirant Judges Training the Trainer Newly Appointed Judges' Orientation Programme Continuing Education for Judges Training Programme Establishing DOJCD Programme Mgmt Support Unit Establishing DOJCD Forensic Audit Unit Supporting BAC's Commercial Crimes Courts Initiative Project Capacity Sustainability Demonstrated Impact Project Design Project Performance Performance Measures Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page xiv j. Summary of sub-program and project evaluation A high-level summary of the sub-program and project evaluations follow on the next pages. Sub-program A: Court services = Poor = Excellent Performance A2 A1 Code • Address shortcomings in infrastructure, change management and project management • Review and redefine project design and approach - Alignment with CFM sub-committee plan - Standardised approach to implementation - Hands-on project management capacity • More structured communication and integrated stakeholder approach to encourage collaboration and engagement • Develop and implement a robust change management model to engage the judiciary/prosecution and support roll-out • Court Services to review project management approach including role definitions, monitoring methodologies and project team handover based on experience and recent operational changes Recommendations • Highest priority project seen as biggest potential impact on EASE but inconsistent pilot implementation • Magistrates’ resistance is a symptom of poor change management and lack of structured approach to involving stakeholders in achieving objectives • Well conceived design and objectives • Review delayed progress but back on track • Lack of broad-based buy-in • Business process mindset and delegations empowerment create a good platform for modernising court services • Well positioned for sustained funding for institutionalisation Description Integrated Case Flow Management System’s extended pilot project (ICFMSEPP) Re Aga Boswa (RAB) Project Performance A2 A1 Code • Address shortcomings in infrastructure, change management and project management • Review and redefine project design and approach - Alignment with CFM sub-committee plan - Standardised approach to implementation - Hands-on project management capacity • More structured communication and integrated stakeholder approach to encourage collaboration and engagement • Develop and implement a robust change management model to engage the judiciary/prosecution and support roll-out • Court Services to review project management approach including role definitions, monitoring methodologies and project team handover based on experience and recent operational changes Recommendations • Highest priority project seen as biggest potential impact on EASE but inconsistent pilot implementation • Magistrates’ resistance is a symptom of poor change management and lack of structured approach to involving stakeholders in achieving objectives • Well conceived design and objectives • Review delayed progress but back on track • Lack of broad-based buy-in • Business process mindset and delegations empowerment create a good platform for modernising court services • Well positioned for sustained funding for institutionalisation Description Integrated Case Flow Management System’s extended pilot project (ICFMSEPP) Re Aga Boswa (RAB) Project Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page xv Sub-program B: SOCA = Poor = Excellent • It is anticipated that the Act will be promulgated some time in 2005 • Project not off the ground as manual development depends on promulgation of the Bill Sexual Offences Multi￾Disciplinary Training Manual (SOMM) B4 • Accreditation to be done ASAP • Stakeholder communication remains critical • Manual development successfully completed with high acclaim (National ICSP award from Standard Bank) • No accreditation done as yet Domestic Violence Multi￾disciplinary Training Manual (DVMM) B5 Performance B7 B3 B2 B1 Code • Task team requires buy-in from stakeholders other than NPA for continuity and sustainability; requires extensive planning for meaningful participation • Funding outside of CJSP needs to be secured • Data sources need to be agreed upon, aligned, formalised and documented • SOCA needs to institutionalize the project to minimize chances of people moving on to other jobs due to insecurity • High level stakeholder support needed to address future funding, project management, infrastructure requirements and skill levels • Case Managers on contracts lack job security– institutionalisation by the NPA needed ASAP • Future funding and interdepartmental coordination need to be in place for the commitment and ownership of the project by NPA to reach reasonable levels of sustainability Recommendations • Protocol still in early development stage • Coordination of huge stakeholder group and continuation of task team are at risk due to delegation of responsibilities to lower level staff • Notably enthusiastic NGO involvement • Five assistants are meeting informational, security and emotional needs of victims • Conviction rates have increased, women are utilizing centers, victims placed in places of safety and prepared for court • Case cycle time reduction from 9-24 months to an average of 6 months due to multi-disciplinary approach, case manager commitment buy-in from key stakeholders • Other countries are interested in the Model • School campaigns exceeded expectations • Adult workshops not meeting expectations • National ICSP award from Standard Bank for most innovative public programme Description Uniform Protocol for Victim Management (UPVM) Victim Assistance Officers’ Extended Pilot Project (VAOPP) Case Managers Extended Pilot Project (CMEPP) Public Awareness Campaigns (PACs) Project • It is anticipated that the Act will be promulgated some time in 2005 • Project not off the ground as manual development depends on promulgation of the Bill Sexual Offences Multi￾Disciplinary Training Manual (SOMM) B4 • Accreditation to be done ASAP • Stakeholder communication remains critical • Manual development successfully completed with high acclaim (National ICSP award from Standard Bank) • No accreditation done as yet Domestic Violence Multi￾disciplinary Training Manual (DVMM) B5 Performance B7 B3 B2 B1 Code • Task team requires buy-in from stakeholders other than NPA for continuity and sustainability; requires extensive planning for meaningful participation • Funding outside of CJSP needs to be secured • Data sources need to be agreed upon, aligned, formalised and documented • SOCA needs to institutionalize the project to minimize chances of people moving on to other jobs due to insecurity • High level stakeholder support needed to address future funding, project management, infrastructure requirements and skill levels • Case Managers on contracts lack job security– institutionalisation by the NPA needed ASAP • Future funding and interdepartmental coordination need to be in place for the commitment and ownership of the project by NPA to reach reasonable levels of sustainability Recommendations • Protocol still in early development stage • Coordination of huge stakeholder group and continuation of task team are at risk due to delegation of responsibilities to lower level staff • Notably enthusiastic NGO involvement • Five assistants are meeting informational, security and emotional needs of victims • Conviction rates have increased, women are utilizing centers, victims placed in places of safety and prepared for court • Case cycle time reduction from 9-24 months to an average of 6 months due to multi-disciplinary approach, case manager commitment buy-in from key stakeholders • Other countries are interested in the Model • School campaigns exceeded expectations • Adult workshops not meeting expectations • National ICSP award from Standard Bank for most innovative public programme Description Uniform Protocol for Victim Management (UPVM) Victim Assistance Officers’ Extended Pilot Project (VAOPP) Case Managers Extended Pilot Project (CMEPP) Public Awareness Campaigns (PACs) Project Not started Sub-program C: Justice College Capacity Building = Poor = Excellent Performance C4 C3 C2 Code • Important to track and capture experiences of mentored magistrates in the course of doing their jobs as a measure of impact • Institutionalisation is a critical factor for sustainability • The pending policy direction is needed before strategic planning can begin • Prepare a work breakdown structure or plan to assist the College in implementing the accepted recommendations • A Parliament decision on the type of College is needed before recommendations can be implemented Recommendations • First workshop successfully held • The project is creating a pool of mentors that can be utilised by the DOJCD in future • The project is pending policy direction on the new Budgeted National Justice College of South Africa • As a result of IAP the College has concrete recommendations and suggestions on how to improve and reposition itself • A training measurement tool was developed and presented • A policy decision by Parliament is pending Description Magistrates’ Mentorship Project (MMP) Visioning and Strategic Planning (VSP) Impact Assessment Project (IAP) Project Performance C4 C3 C2 Code • Important to track and capture experiences of mentored magistrates in the course of doing their jobs as a measure of impact • Institutionalisation is a critical factor for sustainability • The pending policy direction is needed before strategic planning can begin • Prepare a work breakdown structure or plan to assist the College in implementing the accepted recommendations • A Parliament decision on the type of College is needed before recommendations can be implemented Recommendations • First workshop successfully held • The project is creating a pool of mentors that can be utilised by the DOJCD in future • The project is pending policy direction on the new Budgeted National Justice College of South Africa • As a result of IAP the College has concrete recommendations and suggestions on how to improve and reposition itself • A training measurement tool was developed and presented • A policy decision by Parliament is pending Description Magistrates’ Mentorship Project (MMP) Visioning and Strategic Planning (VSP) Impact Assessment Project (IAP) Project Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page xvi Sub-program D: Outsourcing the MMIT = Poor = Excellent Performance D1 Code • The project depends heavily on buy-in processes to ensure all stakeholders agree on the way forward • Continued efforts needed to ensure interdepartmental buy-in and commitment in future project stages • Comprehensive communication strategy key enabler to success Recommendations • Feasibility study completed and awaiting approval of the Options Analyses Report before Treasury Authorization Level I can be secured • The project is however behind schedule • Quality project management skills and high levels of commitment by project team Description Management of Monies in Trust (MMIT) Project Performance D1 Code • The project depends heavily on buy-in processes to ensure all stakeholders agree on the way forward • Continued efforts needed to ensure interdepartmental buy-in and commitment in future project stages • Comprehensive communication strategy key enabler to success Recommendations • Feasibility study completed and awaiting approval of the Options Analyses Report before Treasury Authorization Level I can be secured • The project is however behind schedule • Quality project management skills and high levels of commitment by project team Description Management of Monies in Trust (MMIT) Project Sub-program E: Transforming the judiciary (1 of 2) = Poor = Excellent • Change in mind-set on behalf of judges and magistrates is needed to work towards a common goal, as constitutionally required • Provide training in civil matter adjudication to magistrates • Committee established and process to analyse and transform rules initiated • Projects however behind schedule Unified SA Judiciary Committee (USAJ) Single Rule Making Authority (SRMA) E4 E5 • Required legislation to be addressed as a matter of urgency as it places sustainability at risk and impacts College’s ability to budget for future funding needs (e.g. High Court Judges’ training) • IAC established • On balance IAC was a small and cost effective project with senior management buy-in focused on lobbying and addressing an important constitutional imperative Interim Advisory Council (IAC) Restructuring and Transformation Committee (RTC) Judicial Education and Training Committee (JETC) E6 E7 E8 Performance E3 E2 E1 Code • Key measures to be adopted by Working Group to effect performance management • Integrate views on change management, stakeholder communication and coordinated collaboration with other parties • Institute less formal mechanisms for judicial officers to more continuously raise/discuss pertinent issues • Provide integrated view on project interdependencies and how they collectively contribute to achieving intended outcomes Recommendations • Project operationally highly successful in achieving its stated objectives • Conference report is still outstanding • Sustainability via the SA office of the IAWJ • Successful achievement of objectives and high degree of stakeholder support/buy-in • Institutionalisation via the twelve other projects within the sub-program • Successful achievement of objectives • Institutionalisation via ICFMSEPP - Link to ongoing training programs not clear - No explicit review of conference outcomes Description Leadership & Management Development Program for Women in the Judiciary (LMDPFJO) South African Judges Symposium (SAJS) Judicial Officers Association of SA’s International Association of Judges Conference (JOASA’s IAJ) Project • Change in mind-set on behalf of judges and magistrates is needed to work towards a common goal, as constitutionally required • Provide training in civil matter adjudication to magistrates • Committee established and process to analyse and transform rules initiated • Projects however behind schedule Unified SA Judiciary Committee (USAJ) Single Rule Making Authority (SRMA) E4 E5 • Required legislation to be addressed as a matter of urgency as it places sustainability at risk and impacts College’s ability to budget for future funding needs (e.g. High Court Judges’ training) • IAC established • On balance IAC was a small and cost effective project with senior management buy-in focused on lobbying and addressing an important constitutional imperative Interim Advisory Council (IAC) Restructuring and Transformation Committee (RTC) Judicial Education and Training Committee (JETC) E6 E7 E8 Performance E3 E2 E1 Code • Key measures to be adopted by Working Group to effect performance management • Integrate views on change management, stakeholder communication and coordinated collaboration with other parties • Institute less formal mechanisms for judicial officers to more continuously raise/discuss pertinent issues • Provide integrated view on project interdependencies and how they collectively contribute to achieving intended outcomes Recommendations • Project operationally highly successful in achieving its stated objectives • Conference report is still outstanding • Sustainability via the SA office of the IAWJ • Successful achievement of objectives and high degree of stakeholder support/buy-in • Institutionalisation via the twelve other projects within the sub-program • Successful achievement of objectives • Institutionalisation via ICFMSEPP - Link to ongoing training programs not clear - No explicit review of conference outcomes Description Leadership & Management Development Program for Women in the Judiciary (LMDPFJO) South African Judges Symposium (SAJS) Judicial Officers Association of SA’s International Association of Judges Conference (JOASA’s IAJ) Project Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page xvii Sub-program E: Transforming the judiciary (2 of 2) = Poor = Excellent • It is too early to measure effectiveness/ impact because no new judges have been trained as yet; recommendations can thus not be made in this regard • Project schedule has been changed due to decision to postpone training of 5 judges planned for October (14 judges in one sitting is more sensible) Newly Appointed Judges Orientation Program (NAJOP) E12 • It is too early to measure effectiveness/ impact because no judges have been trained as yet; recommendations can thus not be made in this regard • Project schedule has been changed due to decision to postpone training of 5 judges planned for October (14 judges in one sitting is more sensible) Continuing Education for Judges Training Program (CEJTP) E13 Performance E11 E10 E9 Code • It is too early to measure effectiveness/ impact because no additional trainers have been trained as yet; recommendations can thus not be made in this regard • Capture and address issues raised by trainees to affirm their participation, increase process ownership and sustain enthusiasm • Strengthen the mentorship program by increasing participation of other Judges • A decision needs to be made ASAP on how to proceed Recommendations • Two Judges attended the CJEFI Fellow Conference in December 2003 • Two Judges have been trained as Trainers • To date twenty-four aspirant Judges have been trained, sixteen of whom are now acting Judges • Trainee feedback indicates the project is effective and immediate impact is evident • Project not yet implemented - depends on coordination between Judiciary and NPA • Judiciary has appointed a coordinator while NPA has recently substituted the former project manager with a new person Description Training the (Trainer’s) Trainer (TTT) Aspirant Judges Training and Development Program (AJTDP) Development and Implementation of the Plea￾bargaining Training Program (DIPBTP) Project • It is too early to measure effectiveness/ impact because no new judges have been trained as yet; recommendations can thus not be made in this regard • Project schedule has been changed due to decision to postpone training of 5 judges planned for October (14 judges in one sitting is more sensible) Newly Appointed Judges Orientation Program (NAJOP) E12 • It is too early to measure effectiveness/ impact because no judges have been trained as yet; recommendations can thus not be made in this regard • Project schedule has been changed due to decision to postpone training of 5 judges planned for October (14 judges in one sitting is more sensible) Continuing Education for Judges Training Program (CEJTP) E13 Performance E11 E10 E9 Code • It is too early to measure effectiveness/ impact because no additional trainers have been trained as yet; recommendations can thus not be made in this regard • Capture and address issues raised by trainees to affirm their participation, increase process ownership and sustain enthusiasm • Strengthen the mentorship program by increasing participation of other Judges • A decision needs to be made ASAP on how to proceed Recommendations • Two Judges attended the CJEFI Fellow Conference in December 2003 • Two Judges have been trained as Trainers • To date twenty-four aspirant Judges have been trained, sixteen of whom are now acting Judges • Trainee feedback indicates the project is effective and immediate impact is evident • Project not yet implemented - depends on coordination between Judiciary and NPA • Judiciary has appointed a coordinator while NPA has recently substituted the former project manager with a new person Description Training the (Trainer’s) Trainer (TTT) Aspirant Judges Training and Development Program (AJTDP) Development and Implementation of the Plea￾bargaining Training Program (DIPBTP) Project Not started Sub-program F: Capacity building within the DOJCD = Poor = Excellent Performance F2 F1 Code • Task team / sub-committee mandated to resolve organisational structure issues • Project manager to prepare a clear business plan and blue print and submit to the CJSP￾PMSO • Define measures at unit level rather than individual investigation level and use as tool for staff motivation and morale • The project must be considered a high priority initiative by the DOJCD • Participation of senior stakeholders to: - Prioritiseway forward together with the DG - Fast-track appointment of the PMSU COO - Attract and recruit talent into the PMSU Recommendations • FAU is already impacting fraud and corruption management within the DOJCD • FAU is experiencing delays in project progress due to change in direction • It is not operating at its potential and requires a rethink by the DG • Highest potential impact on overcoming challenges experienced by DoJCD • Steering Committee has been established • Delays and future uncertainty however put implementation at significant risk Description Establishing DOJCD – Forensic Audit Unit (FAU) Establishing DOJCD – Program Management Support Unit (PMSU) Project Performance F2 F1 Code • Task team / sub-committee mandated to resolve organisational structure issues • Project manager to prepare a clear business plan and blue print and submit to the CJSP￾PMSO • Define measures at unit level rather than individual investigation level and use as tool for staff motivation and morale • The project must be considered a high priority initiative by the DOJCD • Participation of senior stakeholders to: - Prioritiseway forward together with the DG - Fast-track appointment of the PMSU COO - Attract and recruit talent into the PMSU Recommendations • FAU is already impacting fraud and corruption management within the DOJCD • FAU is experiencing delays in project progress due to change in direction • It is not operating at its potential and requires a rethink by the DG • Highest potential impact on overcoming challenges experienced by DoJCD • Steering Committee has been established • Delays and future uncertainty however put implementation at significant risk Description Establishing DOJCD – Forensic Audit Unit (FAU) Establishing DOJCD – Program Management Support Unit (PMSU) Project Evaluation, Impact & Effectiveness Assessment of the CJSP Executive Summary Page xviii = Poor = Excellent Performance G1 Code • Obtain clear support from senior NPA management to: 1. Guarantee MTEF funding for Specialised Commercial Crime Court Centres 2. Develop and implement a talent management strategy to attract and retain scarce, specialist skills required Recommendations • Project is well designed, with a proven project management approach • Stakeholders are engaged and play active role in project implementation • Change management has led to altered mindsets and operational behaviours • Pretoria Centre is institutionalised and performance has exceeded expectations • Johannesburg, Durban and Port Elizabeth courts are on track • Lack of NPA funding has put Cape Town on hold Description Supporting BAC-SA’s Specialised Commercial Crimes Court Initiatives (CCCI) Project Performance G1 Code • Obtain clear support from senior NPA management to: 1. Guarantee MTEF funding for Specialised Commercial Crime Court Centres 2. Develop and implement a talent management strategy to attract and retain scarce, specialist skills required Recommendations • Project is well designed, with a proven project management approach • Stakeholders are engaged and play active role in project implementation • Change management has led to altered mindsets and operational behaviours • Pretoria Centre is institutionalised and performance has exceeded expectations • Johannesburg, Durban and Port Elizabeth courts are on track • Lack of NPA funding has put Cape Town on hold Description Supporting BAC-SA’s Specialised Commercial Crimes Court Initiatives (CCCI) Project Sub-program G: Supporting BAC SA’s External Anti-corruption Initiative Evaluation, Impact & Effectiveness Assessment of the CJSP Introduction INTRODUCTION As stated before the project objectives as per the RFP were “to develop a rational evaluation and assessment instrument / tool with a view to evaluate and assess the overarching impacts and effectiveness of the Criminal Justice Strengthening Program, its projects, approaches, targets and outcomes as well as its potential towards sustainability and institutionalization within the DOJCD”. The following approach was followed to achieve these objectives: Phase 1: Design of Evaluation/Assessment Instruments/Tools: At the outset of the project a meeting was held with Matshabe Nthabu, as the key representative of the CJSP￾PMSO, to obtain clear direction for the evaluation. This discussion was primarily held to obtain clarity on the purpose and proposed outline of the report but also covered the following issues: • Request for key stakeholder contact details for interview scheduling purposes • Sign-off on medium of questionnaire (i.e. electronic versus paper) • Pressure testing of proposed approach and work plan – Split of projects across available resources – Sample size for interviewing and surveying – Focus group and field visit requirements • Obtain guidance on potential issues, blockages and risks (e.g. availability of stakeholders for interviews) • Discuss administration and logistical issues (e.g. telephone and fax facilities, travel expenses) • Agree on mechanism and regularity of CJSP-PMSO interaction • Confirm project status • Determine program level evaluation expectations Prior to performing the primary research, existing information and data available in reports, business proposals and reference material were analysed to extract project goals and targets, project plans, performance indicators, baseline data, other input and output indicators, services generated and issues identified. This information was then used as supporting secondary data to provide context to the design of the questionnaires and interviews. The questionnaire was developed in an Excel template with the option of printing a hard copy to fill in by hand or filling it in electronically by the reader, easing the data capturing of responses. These tools employed a strategy, implementation, capacity and sustainability framework aimed at understanding project progress and key successes, challenges and impacts. The key aspects of this framework are summarised in the following diagram: Page 1 Evaluation, Impact & Effectiveness Assessment of the CJSP Introduction • Design of the project (theory and results framework) • Goals • Outputs / objectives • Activities • Alignment with DOJCD vision and mission • Profile of beneficiaries Sustainability • Organisational management • Personnel requirements • Infrastructure requirements • Government and community participation • Related opportunities • Future funding • Resources at hand and resources needed • Challenges • Risk mitigation plans Strategy • Monitoring systems of the project • Progress • Achievement of milestones and deliverables • Achievement of goals / objectives • Effectiveness • Impact Capacity • Human development • Skills development / capacity building Implementation • Design of the project (theory and results framework) • Goals • Outputs / objectives • Activities • Alignment with DOJCD vision and mission • Profile of beneficiaries Sustainability • Organisational management • Personnel requirements • Infrastructure requirements • Government and community participation • Related opportunities • Future funding • Resources at hand and resources needed • Challenges • Risk mitigation plans Strategy • Monitoring systems of the project • Progress • Achievement of milestones and deliverables • Achievement of goals / objectives • Effectiveness • Impact Capacity • Human development • Skills development / capacity building Implementation The questionnaires and interview guides as well as a stakeholder memorandum outlining the reasons for the evaluation were tested with a few CJSP role players to determine the appropriateness, length, logic flow and other design aspects before sending. A copy of the memorandum as well as the program and project level survey questionnaires and interview guides used as the primary assessment tools are attached in Appendices A, B and C. Also included in this phase of the methodology were the identification and selection of people and entities from which data and information were to be gathered and the scheduling of interviews and logistics where relevant. Phase 2: Data Gathering: Project level information was derived primarily from interviews with project managers/coordinators and program level qualitative insights were gathered from discussions with PIC, POC and CJSP-PMSO members. Focus group discussions and telephonic interviews were held with project beneficiaries and survey questionnaires were sent to those project team members not interviewed when appropriate and relevant to project objectives. A summary of the individuals interviewed and surveyed per project is depicted in the following table: Page 2 Evaluation, Impact & Effectiveness Assessment of the CJSP Introduction Sub￾program Code Sub-program /Governance Structure Project Code Project Person Role Method of Contact Mr Harold Motshwane Team Leader, Rule of Law: USAID Interview Mr Stephen Snook Democracy and Governance: USAID Interview Dr. B.S.V. Minyuku CJSP-PMSO Director Interview Mr Simon Jiyane POC Chairperson, MD Court Services Interview Ms Beryl Simelane Deputy CEO: NPA Interview Adv. Thoko Majokweni Sub-Program Sponsor: SOCA Interview Adv. C. Van Riet Sub-Program Sponsor: JC Interview Mr Alan Mackenzie Sub-Program Sponsor: CFO Interview Mr Jackie Ngeva MD: HR Business Unit, DOJCD Interview Mr George Chetty PMSO Financial Manager Interview Mr Matshabe Nthabu PMSO Technical Support Services Consultant Interview Mr JB Skosana Project Manager - Major Role Player Interview Mrs. L. Venter Durban Magistrates Senior Registrar Interview Mr Manuel Snr Court Manager: Pietermaritzburg Interview Ms.S. Zwane Durban Magistrates Court Manager Questionnaire Ms.Z.C. Pienaar Umlazi Magistrates Court Manager Interview Ms.R. Tladi Cape Town Senior Court Manager Survey Ms. Mokoena Pretoria Senior Court Manager Survey Mr.C.Z. Mncwabe Shared Court Services Centre Manager: Durban Interview Ms Shabalala RAB Project Stream Leader: Durban Interview Ms Botma Pietermaritzburg Magistrates Court Manager Interview Mr Naicker RAB Project Stream Leader: Change Interview Mr Rodney Isaacs RAB Project Stream Leader: Residual Interview Group of administrative staff CSSC Staff Focus group Mr Mabaso Chief Magistrate: Durban Interview Mr Ntshangase Snr Magistrate : Empangeni Interview Mr Radyn Snr Magistrate : Emlazi Interview Ms. Memme Sejosengwe Project Manager Interview Ms. Phylis van Rooyen Durban Magistrates Acting Registrar Interview Mr Chris Eksteen Johannesburg Regional Court President Interview Adv. Thoko Majokweni Project Sponsor Interview Adv. Nolwandle Qaba Project Manager Interview Ms. Jacqui De Velier Pillar to Post Manager Interview Siphethu School - Daveyton Beneficiary - School teacher Interview Katlego School - Daveyton Beneficiary - School teacher Interview Rynfield Primary - Benoni Beneficiary - School teacher Telephonic survey Pietersburg Comprehensive School Beneficiary - School teacher Interview Ms. Dorothy Langa - Seshego Beneficiary - School teacher Interview Boiketlo School - Seshego Beneficiary - School teacher Interview Adv. Pierre Smith Project Manager Interview Mr. Justice Mnisi Case Manager - Umlazi Telephone interview Ms. Voyokazi Ngcobozi Case Manager - Wynberg Telephone interview Adv. Thoko Majokweni Project Sponsor Interview Adv. Buyi Nkala Project Manager Interview Ms.Sbongile Cebekhulu Victim Assistance Officer (East London) Telephonic survey Ms. Ntombethemba Msutu Victim Assistance Officer (Lebode) Questionnaire Ms.Julia Moitheri Victim Assistance Officer (Kimberley) Questionnaire Ms. Grace Modiba Victim Assistance Officer (Wynberg) Questionnaire Mr. Sipho Patrick Mkonza Victim Assistance Officer (Soweto) Interview B4 Sexual Offences Multi-Disciplinary Training Manual (SOMM) Adv. Pierre Smith Project Manager Interview Adv. Thoko Majokweni Project Sponsor Interview Adv. Tshidi Kambula Project Manager Interview Ms.K.Lekubu-Wilderson Author/Committee Member Questionnaire Mr. B. Korff Author/Committee Member Questionnaire Mr.T. Nkosi Author/Committee Member Questionnaire Ms. M. Ramagoshi Author/Committee Member Questionnaire Mr. J.S. van der Merwe Author/Committee Member Questionnaire B7 Uniform Protocol for Victim Management (UPVM) Adv Buyi Nkala Project Manager Interview Court Services Sexual Offences and Community Affairs A B PMSO PIC POC A1 Re Aga Boswa (RAB) A2 Integrated Case Flow Mgmt System (ICFMSEPP) B1 Public Awareness Campaigns (PAC) B2 Case Managers Extended Pilot Project (CMEPP) B3 Victim Assistance Officers Extended Pilot (VAOEPP) B5 Domestic Violence Multi-Disciplinary Training Manual (DVMM) H Page 3 Evaluation, Impact & Effectiveness Assessment of the CJSP Introduction Sub￾program Code Sub-program /Governance Structure Project Code Project Person Role Method of Contact C2 Impact Assessment Project (IAP) Mr. Karel Kruger Justice College Project Manager Interview Ms. Belinda Molamu Project Manager Interview 8 Magistrates Mentors Focus group C4 Visioning and Strategic Planning (VSP) Adv. C. Van Riet Project Coordinator Interview Mr Alan Mackensie Project Sponsor Interview Mr. Graemme Dott Project Manager Interview Mr Matshabe Nthabu PMSO Technical Support Services Consultant Interview Ms. Connie Molwantwa Project Coordinator Interview Mr Matshabe Nthabu PMSO Technical Support Services Consultant Interview Mr Cagney Musi Chairman of Organising Committee Survey Mr Maumela Organising Committee Member Survey Dr. B.S.V. Minyuku CJSP-PMSO Director Interview Mr George Chetty PMSO Financial Controller Interview Judge Cecil Somyalo Delegate Telephone survey Judge Denis Davis Delegate Telephone survey Judge Thabani Jali Delegate Telephone survey Judge Mojalefa Rampai Delegate Telephone survey Judge Z. Peko Delegate Telephone survey Ms. Belinda Molamu Project Coordinator Interview Ms. Lucy Mailula IAWJ Chairman Interview Judge Mokgoro Committee Member Survey Ms. Valerie Qgiba Committee Member Survey Judge Theron Committee Member Survey Ms. S Naidoo Delegate Telephone survey Ms. S. Monaledi Delegate Telephone survey Judge M Maya Delegate Telephone survey Judge R Allie Delegate Telephone survey Ms J Phiri Delegate Telephone survey Judge B Nkabinde Delegate Telephone survey E4 Unified SA Judiciary Committee (USAJ) Dr. B.S.V. Minyuku CJSP-PMSO Director Interview Mr George Chetty PMSO Financial Controller Interview Dr. B.S.V. Minyuku CJSP-PMSO Director Interview Adv. C. Van Riet Sub-Program Sponsor: JC Interview Mr George Chetty PMSO Financial Controller Interview E9 Development and Implementation of the Prosecutor-led Plea Bargaining Training Programme (DIPBTP) Dr. B.S.V. Minyuku CJSP-PMSO Director Interview Dr. B.S.V. Minyuku CJSP-PMSO Director Interview 17 Aspirant Judges Trainees Survey Judge I. Farlam Co-ordinator Survey Judge Kriegler "Trainer" Judge Survey Dr. B.S.V. Minyuku CJSP-PMSO Director Interview Judge I. Farlam Co-ordinator Survey Judge Theron Key Contact Survey Dr. B.S.V. Minyuku CJSP-PMSO Director Interview Judge J. Kriegler Co-ordinator Survey Dr. B.S.V. Minyuku CJSP-PMSO Director Interview F1 Establishing DOJCD Programme Mgmt Support Unit (PMSU) Dr. B.S.V. Minyuku CJSP-PMSO Director Interview F2 Establishing DOJCD Forensic Audit Unit (FAU) Mr. Max Budeli Project Manager Interview Mr. T. Bouwer Project Leader Interview Mr Chris Jordaan Project Sponsor: Head SCCU NPA Interview Ms. Karen Borcher Project Manager Interview E Transforming the Judiciary E3 Leadership and Management Development for Women in the Judiciary (LMDPFJO) Training the Trainer (TTT) E5 Single Rule Making Authority (SRMA) E6 E7 Justice College Capacity Building C F Capacity Building GExternal Anti￾corruption Initiative SA Judges Symposium (SAJS) C3 Magistrate's Mentorship Project (MMP) D1 Management of Monies in Trust (MMIT) E1 Judicial Officers Association of SA International Association of Judges (JOASA’s IAJ) E12 Newly Appointed Judges' Orientation Programme (NAJOP) E13 Continuing Education for Judges Training Programme (CEJTP) E10 Aspirant Judges Training and Development Program (AJTDP) Outsourcing of MMIT D E2 E11 Interim Advisory Council/Board (IAC/B) Restructuring and Transformation G1 Supporting BAC's Commercial Crimes Courts Initiative (CCCI) The main purpose for conducting both interviews and questionnaires was so that not only current progress, performance and achievement of objectives are understood, but also to ensure participatory evaluation. In this way issues were identified and explored together with the people deeply involved with the project (CJSP staff, DOJCD staff, donor staff, other role players and beneficiaries (in selected cases only)) so that the root causes of problems faced could be discovered, and the evaluators and role players could together start generating meaningful solutions. The main purpose for conducting focus-group interviews in addition to the questionnaires and individual interviews was so that groups of people that fulfill different roles in the projects could participate in constructive debate and brainstorming to develop a systemic view of issues and potential solutions and facilitate a comparison of CJSP implementation against benchmarks in selected program areas. Page 4 Evaluation, Impact & Effectiveness Assessment of the CJSP Introduction In addition to data and information obtained from CJSP reports, questionnaires and interviews efforts were made to obtain data from reports, systems and logs kept by projects, other sources of statistics and from direct observations during field visits to ensure that objective performance indicators were included in this evaluation. The analysis of data and information obtained commenced as soon as it became available (i.e. in parallel with data gathering) to ensure that the strict evaluation timelines were met. Phase 3: Analysis and Report Writing: Data and information not yet analysed during the previous phase was analysed during this phase of the project to derive trends and assess the achievement of objectives, targets and impact. All of the findings, issues and solutions from this and previous phases were then integrated into a report that provides a thorough view of each project’s current status, alignment with the DOJCD’s vision and mission, progress, performance against output indicators, effectiveness, impact and potential for successful institutionalization. Where appropriate potential solutions as derived together with project role players were also provided. Evaluation Findings The evaluation findings have been compiled as separate reports for each of the twenty eight CJSP projects, as well as an additional report covering the program itself, including the governance structures. These reports comprise the remaining sections of this document. Page 5 Evaluation, Impact & Effectiveness Assessment of the CJSP Program Evaluation The Criminal Justice Strengthening Program 1. Background During the course of this evaluation it was found that a general level of confusion / misunderstanding exists within the DOJCD as to what the difference is between the CJSP as a program and the CJSP-PMSO. For the purposes of this report, we have defined these terms as follows: The Criminal Justice Strengthening Program (or the CJSP or the program) refers collectively to the strategic partnership initiative between the DOJCD, USAID and BAC-SA, the three-tiered governance and management structure (PIC/POC/PMSO), the twenty-eight underlying projects and all the people involved within the PIC/POC/PMSO and projects themselves. The CJSP-PMSO (or PMSO) refers only to the people within the Project Management Support Office as a support function to the program. The Criminal Justice Strengthening Program (CJSP) is depicted graphically, by way of summary, as follows: Program Implementation Committee (PIC) Program Operational Committee (POC) Program Management Support Office (PMSO) Court Services (A1, A2) SOCA (B1, B2, B3, B4, B5, B7) Justice College Capacity Building (C2, C3, C4) Outsourcing the MMIT (D1) Transforming the Judiciary (E1, E2, E3, E4, E5, E6, E7, E8, E9, E10, E11, E12, E13) Capacity Building within the DOJCD (F1, F2) External Anti-Corruption Initiative (G1) 7 Sub￾Programs 28 Projects Governance Structures The Criminal Justice Strengthening Program (CJSP) Strategic partners DOJCD USAID BAC SA 2. Program Rationale and Alignment with DOJCD Strategy The Criminal Justice Strengthening Program (hereafter referred to as the program) was established as a strategic partnership initiative between the DOJCD, USAID and BAC-SA as a result of the challenges experienced by the DOJCD in its application for, implementation of and reporting on USAID funds. The program aims to support and strengthen the capacities of the DOJCD and its strategic alliances (e.g. SAPS, DCS, NPA) and thereby contribute towards the achievement of DOJCD’s objectives (i.e. to make the SA Criminal Justice System effective, efficient, swift and accessible). In practice the program is set up to achieve its aims through the management and execution of twenty eight projects, which collectively constitute seven focus areas, or sub-programs: A. Court Services B. Sexual Offences Community Affairs (SOCA) C. Justice College Capacity Building D. Outsourcing: Management of Trust Monies E. Transforming the Judiciary F. Capacity Building G. External Anti-Corruption Initiative Page 6 Evaluation, Impact & Effectiveness Assessment of the CJSP Program Evaluation Page 7 In some of these sub-programs (e.g. Court Services, SOCA), projects represent a portion of the transformation initiatives being implemented in the justice system. In these instances the program provides a central point only for the management of the USAID funded portions of broader initiatives in the justice system. The CJSP has been designed to support and strengthen the capacities of the DOJCD and its strategic alliances, including the NPA, and thereby contribute towards the achievement of the DOJCD’s objectives (i.e. to make the SA Criminal Justice System effective, efficient, swift and accessible; justice with EASE). Headline indicators and targets have been established for the DOJCD through the CJSP. These were reproduced in the Executive Summary. These headline indicators are comprehensive, and if the targets as stated are achieved the DOJCD will have taken a significant step up in the realization of its objectives. Whilst alignment of program objectives to those of the DOJCD are relevant and important it cannot realistically be expected for these indicators to be changed through any one project of CJSP, or even a collection of CJSP projects. Although the projects and sub-programs of the CJSP are critical components and drivers of change, and indeed have demonstrated impact on the headline targets, achieving the stated targets for conviction rates, case cycle times, costs or customer perceptions (as expected with the EASE targets) nationally would require a combination of projects and in most cases also a variety of other non-project influences, structures, processes, decisions and actions to be achieved. Some of these measures are important for individual projects, but only at the level of the pilot area (e.g. individual courts within which a pilot is implemented) rather than for the Department as a whole. The results achieved would then, all else being equal, be expected to result in a similar level of success once rolled out nationally and measured at the department level in the future. Although there are sub-programs and projects that have additional objectives and indicators specific to that sub-program or project we have not found this on all projects. This makes it difficult to monitor and evaluate the effectiveness and success of a specific project, since the headline DOJCD targets will in most cases only change once a number of other related projects have also been implemented. Instances were also found where specific objectives have been developed for projects, but in some of the documentation these specific objectives were not reproduced, but replaced by some of the headline indicators developed by the CJSP for the DOJCD. This may be a result of different stakeholders producing their own versions of project plans, business plans and reports. It is however notable that the CJSP program continues to drive the institutionalization of a measurable, output￾based approach to business, and that the CJSP has made a considerable contribution to the development and use of performance measures in the DOJCD. It is arguable that the mere practice of focusing on outputs and measuring results is of itself a great achievement, given the base from where the DOJCD has started. The headline indicators of the DOJCD as developed through the CJSP for the most part are lagging indicators, i.e. they measure the eventual outcome and effectiveness of the projects within the CJSP and typically only start changing once a project has been fully implemented for a period of time. Headline indicators should therefore not be used as the only measure of success of the CJSP, especially since they do not necessarily provide early indication of progress and effectiveness, and will only really provide an indication of success once projects have been fully implemented and institutionalized for a sufficient period of time. It would therefore be useful for the CJSP to develop clear leading indicators (i.e. indicators that will already start changing during implementation) and targets specific to each sub-program and project where these are not yet developed (in addition to the current milestones and objectives) which can be tracked to monitor progress. These indicators need to be appropriate for the current phase of each project and enable early and continuous monitoring of the effectiveness and appropriateness of each sub-program and project. This will significantly aid the CJSP management to steer projects and allocate resources and significantly aid the evaluation of the sub-programs and projects of the CJSP during each of their lifecycle phases. Since the CJSP is “designed to support and strengthen the capacities of the DOJCD’s core business functions” it would also be appropriate to measure the extent of skills transfer provided and the ability of the program’s management (primarily the PMSO) to support projects in achieving their individual objectives. These could be measured by means of satisfaction surveys from project members and beneficiaries themselves. 3. Program Governance and Management Structures The Criminal Justice Strengthening Program has a three-tier governance and management structure comprising the Program Implementation Committee (PIC), the Program Operational Committee (POC) and the Program Management and Support Office (PMSO). Evaluation, Impact & Effectiveness Assessment of the CJSP Program Evaluation Page 8 a. PIC The PIC is structured as a virtual team focused on nurturing sustainable relations with the strategic partners and providing program oversight, evaluation and budget approach. The PIC provides strategic guidance, approval of program budgets, ratification of modifications to project work-plans and business cases with the aim to transfer sufficient skills to the DOJCD to move forward on its own and make the temporary USAID/BAC-SA/DOJCD relationship obsolete. The PIC conceptually has a sound purpose: for senior members of stakeholders to direct and influence policy development that will support CJSP implementation. However, in practice the record shows that attendance has been poor. The PIC has a vital role to ensure that project integration issues and interdependencies are thoroughly understood so that policy development is indeed supportive of these issues. This can only be achieved if all members of the PIC consistently participate through attendance of PIC meetings, thereby actively fulfilling the roles conceived for them going forward. b. POC The POC is structured as a value team focused on area prioritization, alignments, project resourcing, budget and expenditure monitoring. The POC provides operational leadership and support towards the effective implementation of projects and the management of interdependencies. The POC is also responsible for the approval of modifications to project work-plans and business cases, reviewing PMSO reports, allocating resources and monitoring budget and expenditure. The key role of the POC is to integrate the synergies which exist between the projects, which lend credence to the decision for sub-program sponsors to be members of the structure. Some sub-program sponsors however tend to focus on projects which they have direct interest in rather than performing an overall integration and oversight role. The evaluation highlighted that the POC meetings continue to be poorly attended, leading to the inadequate management of project interdependencies. This also hampers the development of collaboration between the three justice institutions to achieve transformation. It is recommended that the leadership take more accountability and enforce the fulfillment of roles by individual members, else the credibility of and value added by this structure will be compromised. To date efforts to improve attendance at PIC/POC meetings have been made through joint meetings scheduled every 2 months. This has resulted in notable success as people felt their absence would be noted, which provided the incentive to increase attendance. However, it was also found that when the Director General was not present attendance reduced again. c. PMSO The Program Management and Support Office is structured as an impact team focused on program management, alignment of cross-cutting interdisciplinary issues and effective delivery of products/services. Key leadership functions of the PMSO include alignment of its actions with the program’s strategic objective, development of a strategy that will measure the program in terms of its contribution to strategic targets and leading the implementation of the strategy. In summary the PMSO’s core role is to: • Provide intellectual and technical leadership and guidance to drive the CJSP-PMSO work-plan • Gather, interpret, collate and present program and project information • Perform financial management and project management support • Perform procurement management • Perform contract management • Help establish DOJCD’s PMSU and FAU • Help to build capacity and transfer the required skills to DOJCD staff • Monitor and report on the program’s progress and communicate results/successes of the program It emerged from the evaluation that there exists a high level of awareness of and appreciation for the work conducted by the PMSO across all the projects. In particular, Dr Minyuku is widely regarded as an Evaluation, Impact & Effectiveness Assessment of the CJSP Program Evaluation Page 9 important and effective role player in the program. He is credited with tackling issues on behalf of senior stakeholders and assisting project managers with the development of acceptable solutions and driving meetings towards tangible decisions or outcomes. In terms of technical support (business plans, reports, minutes, project management support, financial reporting), the PMSO team is considered both competent and efficient. The CJSP however has a large number of sub-programs and projects (and quite a few of these are of significant size and scope), whereas the PMSO is a relatively small team. They are therefore spread very thin, both at the administrative/technical support level and at the leadership level. DOJCD project management themselves often seem to lack sufficient capacity and/or capability, and although it could be argued that this is not the conceived role of the PMSO, they expect administrative and logistical support from the PMSO. With the PMSO already stretched thinly, this has posed capacity challenges at times. It could therefore be argued that administrative and logistical support capacity within the program as a whole is generally inadequate, presenting an urgent need for the PMSU or similar interim structure to be fully staffed. It is also suggested that the PMSO could be more effective if the team comprised more senior staff with the ability to influence role players and provide leadership to project managers. Dr Minyuku alone cannot be expected to solely perform this function. It is our understanding that recruitment of senior members for the PMSU is in progress. This should remain a priority. The staffing of the rest of the structure with competent and experienced professional project managers should also be a priority, with the PMSO providing only program-specific information and knowledge transfer. If not, the transfer of skills to the PMSU, as and when it comes on board, will place significant additional pressure on Dr Minyuku’s time and that of the rest of the PMSO. This would constitute a realistic risk of compromising the current level of support offered by the PMSO to the projects. Evaluation, Impact & Effectiveness Assessment of the CJSP Program Evaluation Page 10 4. Observations The key observations made during the course of this evaluation are dealt with in separate sections below. a. Project Relevance and Impact Each of the twenty eight projects is, without exception, relevant to the intended objective of the program, either directly through skills transfer or via the building and piloting of capabilities required to improve the future functioning of the Criminal Justice System. Some of the more notable successes to date include: • Development of an impact assessment tool to measure training impact at the Justice College; • Building professional court management capacity through the Magistrates’ Mentorship Programme; • Setting up the South African Chapter of the International Association of Women Judges; • Creating a larger pool of judges with a more diverse representation as a direct result of the Aspirant Judges Training Project; • Elevating the issue of sexual offenses, particularly against women and children, as a priority and addressing the public needs through specialized courts; • An increasing general impression that the chaos within courts is beginning to improve/disappear and service delivery at courts is improving dramatically, particularly with the introduction of professional court managers and the Re Aga Boswa court management model in KZN, which allows prosecutors and adjudicators to focus on their core functions. The pilot project has demonstrated that efficiency and attitude towards service can be impacted by a structured intervention that identifies wastage in the system and brings all parties together to address process gaps & inefficiencies. It is evident that project successes have been most felt where a strong sense of commitment and ownership exists amongst the project team members from the project owner, sponsor and manager/coordinator to the team members as implementers. Less successful projects tend to have members in the value delivery chain who are not entirely committed to the project objectives or do not back their commitment up with demonstrated active involvement. The most noteworthy example in this regard is the ICFMP, which requires an urgent level of rethink/reinvigoration for it to begin to deliver any tangible results. b. Business and Project Management Approach Given the historical past of the Criminal Justice System and the significant changes that have taken place since 1994, it is most noteworthy that a definite measurable change in the business approach is beginning to surface as a direct result of the program’s initiatives. Where once the department was input and internally focused a multidisciplinary approach is now being employed. The need to interact with other stakeholders (e.g. departments, communities and business) is more widely recognized. Examples of this are evident in the interaction of prosecutors with their communities, increased concern for the empowerment of victims by court officials and a general concern about service within courts beginning to replace the previous task-oriented culture. The department, as a direct result of the introduction of business principles by BAC-SA within the program, is beginning to employ more modern principles and practices including project management, change management and financial management. Whilst a lot still needs to be done to institutionalize these practices as part of the Department’s habitual culture, the changes that have already taken place are significant. In terms of the program’s financial management, strong financial controls are in place to ensure that funds are effectively applied, managed and accounted for. This is evident in the quality of the program’s independent audit reports and continuous measurement and reporting on current expenditure to budget. To date, only 30,6% (R8,407,330) of the 3rd Year Cycle’s budget (R27,511,00) has been expended but this is largely due to a decision by the PIC and POC to stagger the implementation of CJSP projects to better meet the required obligations (and USAID’s plan to stagger its obligation in two tranches). The most recent CJSP-PMSO financial expenditure statement (YTD September 2004) is summarised in the table below: Evaluation, Impact & Effectiveness Assessment of the CJSP Program Evaluation Page 11 CJSP-PMSO’s 3rd Year Cycle Financial Summary Sub-Program Budget Expenditure YTD Sep ‘04 % Spent Court Services R 3,555,000 R 253,422 7.1% SOCA R 3,743,000 R 1,678,673 44.8% Justice College Capacity Building R 1,000,000 R 355,830 35.6% Outsourcing the MMIT R 3,800,000 R 1,070,637 28.2% Transforming the Judiciary R 4,330,000 R 956,567 22.1% Capacity Building for DOJCD R 4,190,000 R 1,308,634 31.2% BAC-SA’s External Anti-Corruption Initiative R 1,500,000 R 620,024 41.3% PMSO R 5,393,000 R 2,163,544 40.1% Total R 27,511,000 R 8,407,331 30.6% The primary enabler to success within the program is seen as competent and committed Project Managers. The most successful projects were found to have skilled external project managers. However, whilst project managers in the program are widely respected for their competence, most are stretched beyond capacity. The nature of their projects, being transformation, often leads to them accepting additional related responsibilities. In addition, most of them have line responsibilities that require a substantial portion of their attention. This ultimately impacts the quality of the project and leads to gaps often being unattended. These gaps include the timeliness and relevance of reporting and attendance at project managers’ meetings. The full time focus of project managers is critical to ensure progress and effective management of the projects and their interdependencies. The CJSP-PMSO has created standard reporting formats and is responsible for reporting to all relevant stakeholders on CJSP sub-programs and projects. Where other reporting is required of project managers and sponsors this creates unnecessary duplication, potential for inconsistency and wastage of scarce time. Project managers and sponsors should therefore be urged to not create additional reports, but use reporting that is already generated for and by the CJSP-PMSO. Project managers and sponsors should be encouraged to raise issues regarding additional reporting requirements for stakeholders outside the PMSO so that the PMSO can help avoid duplication. c. Project and Program Measures The overall trend in performance in areas impacted by the CJSP is definitely positive, even if not dramatic. In a number of cases the feeling was raised that it is very difficult to quantify project measures, particularly impact measures, largely because of the interdependencies that exist between specific projects and other related initiatives (e.g. the Impact Assessment Project’s impact is influenced by variables other than just training such as prosecutors, magistrates, etc). A number of stakeholders also felt that the measurement of case backlogs needs reviewing as it should take into account the quality of thinking applied to cases rather than just the number of cases processed. Also, the measure of case finalization does not distinguish between successful and withdrawn cases. It could be value-adding to track a separate measure for each. The Batho Pele principles required improved overall service delivery and customer satisfaction. It is therefore important that a performance mindset is being instilled within the Department and that attempts are actively being made to gather and track information to support whether or not performance is actually improving. Measures however often suffer from factors such as the lack of incorporation of all the relevant factors and the unavailability of credible baseline information. In this respect, there is a need to convince or compel project leaders to clearly define their project measures and to begin to refine these measures over time as and when new project insights are gained. This will effectively serve as a performance management system to drive project success going forward. In some areas there is still a reluctance by operatives to establish such measures and baseline information and or to be measured within the DOJCD. The short-term focus should therefore be on instilling a positive mindset towards measurement, after which refinement of the measures themselves should be done to ensure that the most meaningful indicators of performance are used. Rather than projects struggling to gather/measure baseline statistics which are either inaccurate or difficult to obtain, it is suggested that projects use currently available statistics as the baseline measures to work from in the short term (e.g. concrete numbers of aspirant judges trained and appointed as judges; current number of women members of the SA Chapter of the IAWJ and current numbers of women judges, etc.). Evaluation, Impact & Effectiveness Assessment of the CJSP Program Evaluation Page 12 d. Organisational Structure and Leadership A number of key factors relating to the structure and leadership of the program and its related stakeholders were identified as having a significant impact on the performance and sustainability of projects. These include: • The new Minister of Justice and Constitutional Development has undertaken to review all department plans, which is slowing down the implementation of many projects, creating uncertainty amongst stakeholders and placing strain on already stretched project managers; • Delays in finalizing the necessary policy directions and legislation is putting a number of projects at risk (e.g. the Sexual Offences Act and policy regarding the establishment of the New National Justice College of South Africa); • POC members and sub-program sponsors are not practicing quality leadership in terms of effectively aligning, integrating or managing the synergies between projects or sufficiently enforcing Project Managers’ attendance at meetings for them to do likewise; • The separation of the three pillars of justice (the DOJCD, the NPA and the Judiciary) has led to an unintended separation in operational delivery of transformation. The historical separation of the different components of the judiciary also complicates coordination and cooperation. Other than the individual interactions between heads of the DOJCD, NPA and judiciary and interaction at ministerial level the governing bodies of the CJSP are currently the only place where project interdependencies can be identified and managed. We realize that this falls outside the scope of the CJSP, but an effective formal government body at department or ministerial level to manage broader issues of coordination and cooperation would have a substantial positive impact on streamlining delivery between the different parts of the Criminal Justice System. The PIC’s and POC’s mandate as structures to create the effective delivery of transformation is limited to the twenty eight underlying projects. The transformation issues are, however, more extensive across the DOJCD, NPA and Judiciary, with the result that the PIC/POC cannot be expected to single-handedly transform the three structures to the extent desired. It is suggested that a higher level concerted approach by the DOJCD to institutional transformation is required to ensure ongoing successful project institutionalisation and delivery in the future, particularly post the PIC/POC’s existence. Many of the organizational structure and leadership issues faced by the program are understandable considering the maturity of the government and the enormity of its task in the first 10 years of democracy. However, if the transformation of justice is to succeed, the structure and governance of the justice system will need to be actively transformed too. As a result the program’s governance structure needs to not only implement and manage relevant projects but also contribute towards the demonstration of workable structural solutions and governance practices. This can only be achieved by strong leadership intervention from all parties. As a start PIC/POC meetings need to be actively attended by all stakeholders and the Chairpersons need to play a stronger leading role in central alignment and coordination of the initiatives. The ultimate impact of a strengthened justice system will require the co-operation of additional stakeholders including the SAPS, the Department of Social Development and the Department of Correctional Services. Should the PIC/POC/PMSO not be effective in developing effective working relations within the current program structure, the potential for effectiveness within the broader Criminal Justice Cluster is at high risk. Furthermore, the lack of formal structures for collaboration (e.g. steering committees or task forces on common issues) provides the following challenges for project sustainability: • The inability to effectively plan resources (human, financial, infrastructure) to meet justice objectives (leading to an over-reliance on donor funding); • The inability to optimize resources, in terms of cost and utilization (e.g. IT infrastructure, change management competence, communication, etc.); • Inconsistent levels of transformation which frustrate/confuse those involved in the system at an operational level; • The inefficient management of issues and risks which as a result often remain unaddressed for lengthy periods of time. Evaluation, Impact & Effectiveness Assessment of the CJSP Program Evaluation Page 13 e. Communication and Change Management Indications are that silo mentalities within the DOJCD and NPA are still prevalent in some areas, threatening the realization and effective management of project interdependencies. There appears to be a subtle unwillingness amongst parties to show empathy for other institutions’ view points and needs. As discussed above, there does not appear to be an overarching program to address these current mindsets throughout the justice system. At the project level, project managers are not equipped with a solid change management approach, customised to the needs of the justice system. • Re Aga Boswa developed its own change management model for impacted staff which has proven very effective in changing the mind-sets of court service staff in KZN. This included regular communication with staff (meetings, newsletters) by the change management stream until August 2003. Thereafter, it has been the responsibility of each Court Manager to provide this information. Court managers meet regularly to receive and share information. Most court managers however felt that insufficient information for communication has been available since the change management stream exited in August 2003. • DOJCD’s HR business unit has developed a change management model that (a) Prepares and equips staff for change (b) Provides support (“journey management”) in terms of communication, skills assessment and development during a change process (c) Evaluates impact post implementation Unfortunately, the change management unit is understaffed and unable to provide the level of support required. In addition, the model is designed for DOJCD staff only and does not address the needs of all stakeholders. • In a number of cases it was felt that Project Managers and their more senior managers are not sufficiently aware of their responsibility to play a leading role in determining change impact and change management requirements. Successful implementation could be hampered by an unwillingness to cooperate if there is a lack of, or a perceived lack of, consultation up front (such as the current perception within the LCMC). Broad consultation and communication with all impacted stakeholders are key enablers for success – including discussing what’s not working (lack of implementation success) and providing support for issues/risks. In addition, buy-in from DOJCD middle management is needed through continued program efforts of offering and providing assistance/support to overcome current challenges. This will require a deliberate change management and reeducation approach by Project Managers. The PMSU is also a risk mitigation strategy to involve DOJCD stakeholders more and ensure skills transfer as well as a mechanism for improved communication of the program. This structure needs to be implemented as a matter of urgency to overcome the inherent resistance that exists whenever an external entity (such as the PMSO) is involved with the implementation of change projects within an organisation (such as the DOJCD). f. Funding and Sustainability The most critical elements to sustainability are seen to be the success / effectiveness of the projects themselves (which in itself will enable departmental funding in future), commitment from department managers and access to resources / funding well ahead of the date when donor funding is expected to end. At present, the DOJCD demonstrates an over reliance on donor funding to fund core activities, which is of concern considering that the funding is expected to cease in November 2006. USAID funding is spread across the whole program with little to no counter-party or other funding contributions being actively sourced to support the ongoing needs of the projects. Given that donor funds are dollar-based, the strengthening of the Rand also places additional pressure on projects that have no other means of funding the shortfall. The adoption of a contingency to mitigate this risk is a sound business practice but one which would be better balanced with access to alternative funding sources. Furthermore, a clear and evident process for ensuring that projects become part of the ongoing divisional budget is lacking. Large projects (e.g. RAB and MMIT) are already linked to the MTEF but smaller projects (e.g. the SOCA projects) have a Evaluation, Impact & Effectiveness Assessment of the CJSP Program Evaluation Page 14 more tenuous link. This places considerable stress on contract staff whose salaries are covered by donor funds (e.g. Case Managers and Victim Assistance Officers) as their positions are considered insecure. The downstream impact is that such staff are lost to more secure forms of employment, with a resultant loss of continuity for the project. The lack of budget also places significant challenges on projects needing to appoint staff and with a shortage of training capacity to develop much needed skills. These challenges, however, will hopefully be addressed at a senior level, where currently the Minister of Justice and Constitutional Development has commissioned a task force, chaired by Jackie Ngeva, to identify the human resource requirements to ensure sustainable service by the justice system. The task force will: • Analyse the impact of increased service required as a result of new legislation & policy (all DOJCD and NPA business units); • Determine staffing requirements across the system; • Determine training resource requirements; • Submit a budget recommendation to the Minister by 30 September who will apply to Treasury for an increase in budget allocation (current budget of R1bn for 2005 is seen as very restrictive and likely to prevent any capacity building in the DOJCD & NPA). g. Capacity Building Capacity building is seen as a focus area within the program – both in terms of it being the focus of some of the projects themselves (e.g. Training the Trainer, Aspirant Judges Training, Magistrates’ Mentorship Program, etc) as well as the transfer of skills from the PMSO to the PMSU. However, skills transfer in this regard is not being achieved as quickly as would have been hoped. To date, capacity building initiatives of the PMSO have been mainly technical and administrative skills transfer within the PMSO as well as providing training on project management principles and tools for Project Managers and assisting Project Managers to arrange and facilitate a number of workshops and conferences. The DOJCD completed a skills audit last year and has developed a skills development plan. Currently, only the Court Managers’ Learnership Program and a Management and Leadership Development Program have been implemented. Ongoing delivery on the plan depends on budget for training programs and training resources. In terms of the other strategic partners, the NPA does not appear to have a clear skills development plan as there are no HR skills / competence in the NPA to develop and deliver such a plan. Skills development for the Judiciary is highly dependent on the creation of a single united judiciary. It is expected that the component of the program dedicated to transforming the judiciary should successfully address capacity building in this regard. Evaluation, Impact & Effectiveness Assessment of the CJSP Program Evaluation Page 15 5. Key Recommendations • The integration of projects is an important element required to ensure the overall success of the program and ultimately the achievement of the DOJCD’s strategic objectives. Integration is not coincidental but requires a well conceived and carefully executed plan. The program’s governance and management members have constructed such a plan which includes a description of the interdependencies of current and future projects, other departmental / strategic partner initiatives and how these initiatives collectively contribute to the intended outcomes. The POC/PMSO should ensure that this plan sufficiently describes all integration and cooperation requirements and the relevant stakeholders responsibilities in the management of interdependencies and synergies and then communicate this roadmap to the entire program’s stakeholders, allowing all parties to understand how individual projects relate to other projects and to the collective journey. A few individuals within the program have a clear sense of this journey but it is not yet broadly implemented, communicated or understood. • An explicit, unambiguous communication and change management strategy is needed to ensure information sharing and fostering of buy-in amongst stakeholders across the criminal justice value chain. • A mechanism for independent bodies across the justice system to work together is needed as a matter of urgency or sustainability is compromised going forward. This mechanism must be supported by strong leadership and a willingness to actively manage its deployment. • The DOJCD needs to actively manage the transition from USAID funded pilots to DOJCD institutionalised national roll-out, including sourcing alternative funding or providing adequate funds in the next MTEF budget (at least six months before donor funding ends). The PMSU, once established, must clearly understand and measure the level of institutional maturity for each of the projects. A systems approach is recommended to ensure overall integration and sustainability going forward, where people put the holistic big objective ahead of their own personal aims/agendas. • The program’s governance (or the PMSU once established) should consider setting up a central resource pool of dedicated skilled project managers who would be available to manage ongoing DOJCD initiatives as and when required. For this to succeed, project management needs to be recognised within the DOJCD as a valued, professional role with a definite career path. In this way the Department would not only be more successful in attracting suitable candidates but also in retaining quality managers once specific projects reach completion. • Project managers and sponsors should be urged to use reporting that is already generated for and by the CJSP-PMSO for all their communication and preferably channel all project documentation through the CJSP-PMSO to ensure consistency and avoid duplication. Project managers and sponsors should be encouraged to raise issues regarding additional reporting requirements for stakeholders outside the PMSO so that the PMSO can help avoid duplication. • Project funding from all stakeholder sources should be coordinated, managed and reported on by one central structure. It appears that this will in fact be the case with the PMSU but could be played by the POC/PMSO in the interim until the PMSU is firmly established. The difference in financial year-ends of the government and donors imply that continued funding needs to be budget for by the relevant departments at least six months before the financial year-end of the last year of funding by a donor. Departmental budgeting falls outside the mandate of the CJSP. It is therefore important that the responsibilities of project sponsors relating to project institutionalization be clearly communicated, including their role in sourcing funding themselves and/or the process and requirements for ensuring inclusion in organizational budgets. Decisions also need to be made and communicated on the reallocation approach to be applied to USAID funds. • Sub-program sponsors need to actively ensure that project managers not only establish measures for their projects but also actually apply the measures in tracking project performance. This is fundamental to good governance of the program and a key enabler for achievement of the Batho Pele principles of improved overall service delivery and customer satisfaction and should therefore not be considered as an afterthought. • The training strategy of the DOJCD and the NPA should be devised and implemented together in order to: – Optimise resources (e.g. Justice College) – Optimise subject matter/content expertise on legislation and process – Prevent fragmented / unaligned skills development Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 Project Code: A1 Sub-Programme: Court Services Project: Re Aga Boswa (RAB) 1. Background a. Project Problem Statement Principle 4 of the 34 Constitutional Principles says: “There shall be separation of powers between the Legislature, Executive and Judiciary, with appropriate checks and balances to ensure accountability, responsiveness and openness” To achieve the intended separation of powers, the judiciary has de-linked from the public service and now acts as an independent branch of government, accountable directly to Parliament through the Chief Justice. Prior to the Constitution coming into effect, magistrates were responsible for administrative functions and financial resources of their courts and were accountable to the Executive (DOJCD) for these responsibilities. To comply with the Constitution, it was required that the magistrates no longer perform administrative functions. Accountability for administrative functions and financial resources of the courts, however, remained with the Department of Justice and Constitutional Development. Thus, it is required that judicial and administrative functions be separated. Administrative functions play an important role in the efficiency of service to the public. Regional offices provide most administrative support to courts. This centralized model is viewed as inefficient and the source of delays in providing administrative services to courts. Magistrates have held responsibility for managing administration and have had to divert substantial time to driving and escalating administrative service, impacting their ability to focus on their judicial functions. It was thus also important that service delivery be addressed in the process of separating judicial and administrative functions. b. Project Purpose To design, develop, detail, streamline and operationalize a decentralized Court Service Delivery Model that: • Allows for implementation of separation of powers • Delivers capacity and ability to manage service at court level • Ensures access to specialist knowledge and expertise provided by the respective support business units (e.g. HR, ISM) of the Department of Justice and Constitutional Development • Supports the overall transformation objectives of the court system by ensuring access to justice for all, embracing the language and cultural diversity of the public it supports and conforming to the principles of Batho Pele 2. Project Definition a. Objectives By implementing the Court Management Model nationally, the project seeks to: • Build management capacity at court level for swift (decision making) justice • Contribute to the reduction of case cycle time from 9-24 months to 6-9 months • Contribute to the increase of the conviction rate by 10% • Make courts accessible to the public • Reduce the average cost per case • Establish dedicated support to case flow management processes Page 16 Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 b. Key Deliverables Components of Re Aga Boswa: • Business process re-engineering to design and develop a rational operating model and functional structure for decentralized court management • Prove concept through pilot of the Court Service Delivery Model in KZN: - Build capacity at the courts by appointing & devolving delegations to Court managers - Create Court Support Services Centre to support transactional requirements that cannot be implemented at court level - Create Centre of Excellence to provide specialist support to court management from the Finance, Procurement, HR, ISM and Court Services business units - Capacitate the courts to deliver service to the public - Redeploy resources from regional office to courts - Implement training and change management programs to ensure smooth transition to new model. - Refine model and structure during and after implementation - Obtain approval to institutionalize • Roll-out of the Court Service Delivery Model • Change management to inform all stakeholders and obtain their buy-in to the Court Management Model • Create a policy and regulatory framework to support the Court Management Model (process maps; manual and legislation) • Design and implement a uniform performance management system for courts, aligned with the strategic objectives of the DOJCD as well as the judiciary, NPA and the legal aid board • Strengthen the strategic management support group to provide strategic management and leadership support to the courts and the Court Services business unit Re Aga Boswa is also expected to act as an integrator project and support and align the implementation of: • Court Process Project • Integrated Case Flow Management System • Integrated Justice Court Centres • Management of Moneys In Trust • Specialised Courts c. Key Stakeholders • Judiciary • National Prosecuting Authority • Department of Justice and Constitutional Development - Court Services National Office Sections (Court Performance; Policy & Strategy; Support of Vulnerable Groups; Facilities Management) - HR, ISM, Finance, Procurement • External stakeholders - Departments forming the Justice, Crime Prevention and Security (JCPS) cluster - Role players in the Integrated Justice System (IJS) Page 17 d. Key Project Team Members Sponsor: Simon Jiyane Project manager: JB Skosana Project Streams: HR IT Change Management Training Service Level Agreement Regulatory Framework Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 Page 18 : 3. Project Performance Notes - The evaluation of progress below covers all deliverables and is not restricted to deliverables within the scope of the CJSP. The “bigger picture” view has been evaluated for the purpose of assessing sustainability of the Re Aga Boswa model. - The evaluation covers core deliverables of the project and does not include an assessment of the integrator role of the project. The initial contact list provided to the evaluator did not include representatives from other projects requiring integration support from Re Aga Boswa. Additionally, delays in obtaining a meeting with the Project Manager resulted in limited time to meet with a broadened contact list. a. Progress against Project Milestones Key Milestones Status Planned Close Date Actual Close Date % Complete Progress Assessment Business process re￾engineering and design of the Court Management Model Completed Dec-02 Dec-02 100% Pilot implementation of Court Management Model in KZN Implementation Dec-03 Sep-04 75% A Certain components of the model have not yet been fully implemented Approval of Re Aga Boswa Blue Print from Director-General and Minister In progress Mar-04 10-Sep￾04 50% A Change in ministry led to a modification and review of the Re Aga Boswa blue print and suspension of roll-out. Approval is expected KZN trial run, impact assessment & approval of roll￾out1 In progress Sep-04 20% G New task as a result of ministerial review Begin roll-out to Gauteng & Western Cape Planning Apr-04 Oct-04 A Ministerial review has delayed implementation Complete roll-out to Gauteng, Western Cape and Free State Planning Mar-05 Some recruitment has already taken place Complete roll-out to remaining provinces Planning Mar-06 Implementing a uniform performance management system Planning Mar-05 Policy & regulatory framework In progress Mar-05 50% Process maps & manuals completed. Audit in progress to determine f legislative changes required Strengthening of the strategic management support group Planning N/A Notes: 1 A trial run of the model will take place in KZN during September and October, where refinements will be implemented in the model. Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 b. Project Management, Monitoring and Reporting Project Management Approach: In the KZN pilot, the project had 8 streams: • Training & development • Change management • Implementation • Site & infrastructure • Human resources • Quality management • Project management • Integration Staff members within the Department were seconded from line functions to the project as stream leaders and formed the core project team. In addition, court managers participated significantly in the delivery of each stream. Consultants from the IBM / Nokusa consortium were appointed to support and guide all the streams, especially in the design phase of the project. The project manager, JB Skosana, provided strategic leadership to the project team and managed issues that required national office intervention or strategic insight. The project stream leaders together led and managed the day-to-day delivery of the activities of the project. The project stream leaders were based in Durban until August 2003, at which point the team was disbanded and the members returned to their line responsibilities. At this point, there remained deliverables that still required implementation. The court managers, however, were considered capable of running the model as an operationalised model and hence project management intervention was no longer required. Court Services have recognized that the KZN court managers needed more support after August 2003 than they originally intended providing. They plan to provide additional project resources in KZN to support the institutionalization of the Court Management Model in KZN. In addition, an activities manager was appointed in August to support institutionalization of the model, including roll-out. Project Team Monitoring & Reporting The project stream leads provided weekly progress reports to the project manager until August 2003. Project Governance Structures: The project has a combined impact team & steering committee (a list of members can be found in Appendix A) The purpose of the steering committee is • To monitor progress of Re Aga Boswa • To ensure integration with other initiatives being implemented through Re Aga Boswa • To discuss and resolve areas of conflict / misalignment around integration. c. Progress against Key Performance Measures Analysis of court performance in KZN was underway at the time of the evaluation and hence results were not available for this report. Page 19 Measure Baseline Target Actual (2004) Case back logs Reduce by 10% Case cycle time 9-24 months 6-9 months Average cost per case None available Average court hours Swift decision making 6 months 1 month 3 months Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 Most court managers interviewed felt that these measures are not directly influenced by the implementation of Re Aga Boswa: • It will enhance service delivery to the courts, but is only one of many inputs influencing the above measures Page 20 • Court managers identified proper case flow management as the initiative most likely to influence the above measures. d. Skills Development and Change Management Project Team Skills Development The most notable skills development to the greater project team has been through on-the-job skills transfer from the IBM consultants. The key areas where skills transfer took place: • Process mapping • Project management • Change management The court managers, in particular, have been equipped with these skills which have enhanced their ability to deliver change and develop their own staff and area of responsibility. Court Services Staff Skills Development Court Services staff members have received once-off training interventions including computer literacy, customer service, records and information management, and registry management & archiving. On the job training has also been provided for selected areas of competence. Court managers have received training and development to enable them to fulfill the managerial responsibilities they have acquired as a result of Re Aga Boswa. Skills developed include strategic / business planning, performance management, job-related training including delegations and HR process and leadership. The KZN trial run in August and September includes tasks aimed at transferring skills from National Office to court managers. Various Court Services units are currently developing training manuals that focus on key aspects of the Court Management Model. The senior court services managers will be delivering training on these aspects to the Court Managers reporting to them. Change Management Change management activities have focused on the needs of Court Services staff. These activities included: • The Habits Training program was developed by the Training & Development stream, with the support of the IBM consultants assigned to this stream. The program was designed to change mindsets and get the involvement of staff in turning around performance of the courts. It is supplemented by a board game that is played on a weekly basis, reinforcing the concepts of the program and developing team work amongst Court Services staff at a court. Impact of the program is monitored through the “Voice of the Customer” survey which measures satisfaction of all court stakeholders including the public. The program has been well received by Court Services staff and Court Managers believe it is the main driver for the improvement in morale and attitude to service in courts. • Regular communication with staff (meetings, newsletters) by the change management stream provided updates and a newsletter to staff until August 2003. Thereafter, it has been the responsibility of each Court Manager to provide this information. Court managers meet regularly to receive and share information. Most court managers however felt that insufficient information for communication has been available since the change management stream exited in August 2003. Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 e. Notable Outputs and Successes to Date Output / Success Description Implementation of Court Managers in KZN Court Managers have been appointed in all 12 sub-clusters of the KZN region. Delegation of authority for most duties has been successfully implemented. Court managers feel empowered to manage the administrative functions and decision making of the courts. Court managers perceive an improvement in turnaround on administrative task since responsibility was decentralized Process culture operationalized at court level Through the process re-engineering exercise with the IBM consultants, court managers have adopted a process orientation to managing their courts. They have developed process mapping skills which they can apply in the execution of their duties. Court managers interviewed rated process mapping as the most significant success of the project in that: - it is an effective tool for training staff - it provides staff with process maps which prevents uncertainty in how tasks should be executed - court managers are better equipped to identify root causes and address them High morale of court services staff at court level The Habits program, service training and increased empowerment of court services have created teamwork and a customer service mindset in court services staff. The delegations of power have also contributed to improving accountability for court performance f. Current Project Challenges, Dependencies, Risks and Issues Implementation of the Court Management Model not yet Complete in KZN Key components of the Court Management Model have not yet been implemented in KZN. The outstanding components are: • The Centres of Excellence have not been implemented or fully capacitated (some capacity from the regional office remained). This impacts Court Managers as they are not yet fully supported in executing responsibilities where they have limited experience or skills development. • Full implementation of the Court Support Services Centre (CSSC) – creation of an open plan workspace is currently underway. • Registrars have not yet been appointed (to enable integrated case flow management). The KZN model requires refinement based on experience and recent operational changes. • Implementation of the Justice Yellow Pages (JYP) and the Basic Accounting System (BAS) have enabled courts to process most transactions, reducing their dependence on the CSSC. This has lead to much reduced demand on the CSSC (operating at 10% of envisaged workload), with a corresponding increase in workload at court level. • There has been limited redeployment of staff from the Regional Office to courts, particularly to courts outside major cities or towns. Many courts are perceived to be operating without sufficient capacity to deliver the services they have been delegated. • The business process engineering exercise clearly defined the roles of Senior Court Managers, Court Managers and the various functional operatives of the courts and the CSSC. The definition of the roles of Court Managers is a perceived gap to be addressed. Magistrates Buy-In Responses from court managers and magistrates indicate that communication and consultation with the magistrates did not lead to magistrates’ broad-based buy-in to the implementation of Re Aga Boswa. Magistrates interviewed supported the concept of Re Aga Boswa but were dissatisfied with the level of consultation conducted with them. Court Services however perceive that some magistrates, in practice, appear to be reluctant to abandon administrative roles despite the separation of powers. This requires vigorous change management to change both attitudes and mind-sets. Page 21 Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 The Project Manager and the CJSP-PMSO Program Director have held meetings with the judiciary to communicate and consult with them on key components of the Re Aga Boswa project. In addition, the project steering committee included representatives of the judiciary who attended steering committee meetings where progress, issues and decisions were discussed. Despite this, the magistrates have concerns about the model and its implementation, which the KZN Provincial Magistrates Courts Judiciary have documented and presented to the Director-General of the DoJCD. In summary, these concerns are: • The judiciary does not perceive a significant output from the business process engineering exercise. • The cluster structure of Court Services is not aligned with the provincial judicial cluster system. • The judiciary perceive the Senior Court Manager role, CSSC and Centre of Excellence as another version of the regional office that Re Aga Boswa sought to decentralize. • There has been no “prior meaningful and constructive involvement, consultation and deliberation with the Magistrates’ Courts Judiciary” during the conceptualization and implementation of the project. • The judiciary should have direct control over administrative functions that support the judicial process. Page 22 In many areas, proactive court managers have taken the initiative to gain the buy-in with the magistrates they support and the parties have implemented management processes and meetings to ensure collaboration in the management of the courts. During implementation, issues that arose between magistrates and court managers indicated that the parties needed clarification of their respective roles. To clarify roles, Court Services issued circular 69 which explained the roles and governance structures. Follow-up by the Project Manager revealed that circular 69, while a blunt instrument for the issues, has to some extent achieved its objective. Court managers and magistrates have not had further communication or consultations on the subject. However, the Court Services Business Unit and the Magistrates Commission are working together to clarify roles. Ministerial Review of DoJCD Projects The Ministerial review resulted in a number of important project modifications: • Grading of courts per population dynamics to refine volumetrics • Three-tier Court Management Structure – court manager, area court manager and senior court manager • Services Delivery Cluster Alignment with Other Court Services Initiatives Court managers interviewed were concerned with the number of independent changes (eg. JYP, BAS, ICFMS) that were being implemented in the courts. These changes were not viewed as being aligned with Re Aga Boswa or being implemented with a common, consistent change management approach. g. Current Action Plan to Finalize KZN Pilot for Roll-Out Approval During the trial run in September and October, the activities below are planned towards obtaining ministerial approval of roll-out: • Obtain sign-off from judiciary, prosecution and court managers of: - Delegations - Administrative clusters - Re-demarcation of magisterial boundaries • Finalise and implement revised delegations • Preliminary workstudy and job evaluation report, assessing capacity at courts to deliver the Court Management Model • Integration of registrars, interpreters and inspectors in the Court Management Model • Establish and capacitate the Centre of Excellence for KZN • Develop training material by Court Services Program manager for skills transfer • Mentorship and quality enhancement program Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 4. Effectiveness Indicators The project has not set effectiveness measures and hence indirect impact cannot be commented on. 5. Sustainability Institutionalization The Court Management Model is in the process of being institutionalized. A blue print for institutionalization and roll-out has been drafted for approval from the Ministry. The Court Services Business Unit is confident that it will be approved at the end of the trial run in KZN. The key factors for sustainability of the Court Management Model are: Factor Status Sustainability Levels of skill and capacity There is a perceived lack of capacity in the KZN courts, driven by insufficient staff and limited training to develop skills. Training thus far has been in the form of once-off interventions. A mentorship program has been developed for court managers. There is also a Court Managers Learnership. No skills development program is currently available for the supervisory level reporting into Court Managers. The training intervention for court services staff is not supported by on-the-job training Sufficient The DoJCD has appointed a service provider to conduct a nationwide work study and job evaluation of Court Services. This will inform the headcount, levels and skills development requirements of Court Services. Their recommendations are due before the end of the year. These recommendations will form the basis for determining capacitation of both courts and national office support functions (eg. HR) Engagement of stakeholders (Court services) Through the business process engineering exercise, court managers in KZN were able to participate actively in the design of the Court Management model. They also gained important skills development through interacting with IBM consulting. This lead to their buy-in and commitment to the model. This approach will not be appropriate for roll￾out as the model is now designed. Some court managers in other provinces were given the opportunity to participate in the KZN exercise, but the majority have had little exposure to the model Limited Court Services recognize the roll-out challenge and intend addressing it through enhancing training & change management and creating a forum to share experiences. There is not as yet a clear plan to make these enhancements and it is thus not possible to assess the approach as sustainable Engagement of stakeholders (judiciary) Refer to the issue raised above regarding the buy-in of magistrates. The judiciary is an important and influential stakeholder and the main internal beneficiary of the project. The project has not yet delivered a successful approach to obtaining their buy-in. A further factor influencing buy-in is that the judiciary is undergoing a transformation process that will take some time to achieve its objectives. Separation of powers and silo management in the justice system create an extremely challenging environment for buy-in Limited Court Services has initiated the development of an operational protocol to regulate the relationships between the stakeholders of the justice system. The protocol will be discussed for the first time at a meeting between the Minister, Director-General and Chief Justice, scheduled for October. This is an important dialogue but cannot be constrained to fixed deadlines. It is likely that roll-out of Re Aga Boswa will be achieved before the protocol is designed and implemented Page 23 Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 Page 24 Factor Status Sustainability Communication and change management Change management and communication approach applied until August 2003 was valued by Court Services. It exited, however, before all key deliverables were completed and did not accommodate other stakeholders. An impact assessment has recently been conducted, but results were not available at the time of this report Limited / poor Both the DoJCD HR business unit and Court Services have a clearly articulated approach to communication and change management. Both areas, however, lack the financial and human resources to deliver this approach. A change management function has been defined for HR but it is not currently staffed Infrastructure requirements There is significant disparity in the physical and IT infrastructure of courts. Many courts do not have connectivity or computers. This limits Court Services ability to address inefficiency in court processes and to fully decentralize court management Limited There is a 5-10 year plan to address inequalities in infrastructure Project management and co￾ordination Court Services currently have a project manager, activities manager and project co￾ordinator to drive national roll-out The success of delivery in KZN lay in committed, dedicated and driven project stream leads that drove the day-to-day implementation process The project would have had stronger progress had there been better project tenure, stakeholder communication and project monitoring. At times, limited capacity from the Project Manager at national office lead the stakeholders to believe issues were not being addressed openly and transparently Cannot evaluate It will be important to apply the learnings of KZN in establishing project teams for roll-out. Insufficient information was gathered to determine the readiness of project management for roll-out Future funding A lack of available budget currently limits the DoJCD’s ability to address capacity issues in Court Services and HR and infrastructure availability in courts. Change management and training competence and delivery is also impacted by lack of funds Sufficient The above mentioned workstudy will provide analysis to support an increased allocation of budget. In parallel, the Court Services Business Unit is developing a revised budget request that will be fine tuned based on the results of the workstudy. Court Services are confident that funding will be made available for their core capacity needs. It is likely that donor funds will be sought to enable the training and change management requirements for roll-out 6. Key Findings a. Project Performance Re Aga Boswa is well conceived and its design is relevant to the identified problems created by the separation of powers between the executive, legislation and the judiciary. While this evaluation does not include an assessment of the extent to which RAB acts as an integrator, feedback from Court Managers indicates the need to review the implementation and change management approach to integration. The project, while behind schedule, is on track to implement its key deliverables. The deliverables not yet implemented are important inputs to the full functioning of the Court Management Model in KZN. The ministerial review appears to have led to a decrease in momentum and staff morale. The trial run and finalization of the blue print will, however, drive the pilot towards completion and approval for institutionalization. Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 The appointment of court managers and the delegation of authority have been significant milestones in enabling the turnaround of court performance. Through the KZN pilot, Re Aga Boswa has demonstrated the potential of the Court Management Model to transform mindsets and service of court staff and management. Magistrates’ broad-based buy-in in KZN remains the most significant risk to success in KZN as they could escalate their concerns to the minister and hence place approval at risk. Many of the concerns raised by the magistrates indicate that they have a limited understanding of how the Court Management Model is expected to operate. While the judiciary have been involved in consultations and steering committee meetings, their buy-in has not been achieved. Finding an effective method for engaging the magistrates in KZN and future roll-outs remains a significant challenge for the project. The trial run in KZN currently only plans to obtain magistrates sign-off on delegations and court clusters. An impact assessment is planned for the end of the trial run and will inform the roll-out decision. The project team should consider a more structured communication and stakeholder approach to encourage collaboration and engagement of all stakeholders, including magistrates: • Establish a monitoring task team with representatives from the judiciary, prosecution and Court Services (to be selected by each body). The task team would be responsible for assessing trial run progress, collating learnings and championing support for the Court Management Model within their organizations. • Provide frequent, transparent and detailed progress reports on the trial run to Magistrates and court managers. b. Sustainability and Institutionalisation The project is well positioned for approval of institutionalization, provided it can attain the buy-in of the magistrates in KZN. The workstudy and budget exercise should address the capacity, skills development and funding requirements for a sustainable Court Management Model. In the roll-out phase, the speed of project delivery is likely to be faster than in KZN. It will thus be important for the project team to review the design and approach of the project to address areas of concern: Change Management A robust change management model must be developed and implemented to: • Provide a practical interim solution to engaging the judiciary and prosecution while the operational protocol is debated and implemented. • Create a mechanism for engaging court managers in a manner that delivers the same mindset change and enthusiasm achieved in KZN. • Support roll-out from conception to implementation to review. Based on the experiences in KZN, the following actions are recommended to create a resilient change management model: • In each province, identify dynamic representatives of the judiciary, prosecution and court management to form a provincial change management committee. The committee would be responsible for developing and monitoring change management and communication plans, championing the Court Management Model in their organizations and ensuring feedback to the project. • Incorporate change management experience from KZN and other development projects or programs (eg. Commercial Crime Courts, Transforming the Judiciary) into a structured change management plan. • Empower court managers by seconding them to project manage components of the roll-out in their province. Project Management Based on experiences in KZN, Court Services should review the project management approach to ensure readiness to support roll-out. Factors to be reviewed should include: • Role definitions for project team members, especially stream leads and the project manager. • Progress monitoring methodologies (meetings, reports, etc) • Criteria for the exit of the project team / handover. Page 25 Evaluation, Impact & Effectiveness Assessment of the CJSP Report A1 Annexure A: RAB Steering Committee Members Adv. Patric Mzolisi Mtshaulana Chairperson Mr Simon Jiyane MD Court Services Dr Biki Minyuku Program Director CJSP-PMSO Mr Alan MacKenzie CFO, DoJCD Mr Jackie Ngeva MD HR, DoJCD Mr H Motshwane USAID Mr T Bouwer BAC Mr Brian King BAC Mr H Fourie IJS Program Director Mr George Chetty CJSP-PMSO Mr K M Nqadala Regional Court President Ms M Sejosengwe DoJCD Adv JSM Henning NPA Ms T Matzke NPA Mr Mbuyiseli Deliwe DoJCD Mr Marthinus Langenhoven DoJCD Mr DSV Ntshangase DoJCD Mr Matshabe Nthabu CJSP-PMSO Mr Steve Killick Service provider Mr Moses Hadebe Service provider Mr Mike Makhura Service provider Mr Roelof J Groesbeek Service provider Ms Nonkosi Cetwayo DoJCD Mr Hishaam Mahomed DoJCD Mr Zanele Qaba Ernst & Young RAB Streams Stream Stream Lead Human Resources Dr. M. Lebaka Change Management Mr. Tsietsi Molema Training Ms. Jackie Botma Service Level Agreement Mr. Jonathan Ratshibvumo Regulatory Framework Mr. Rodney Isaacs IT Mr. Matthews Mothabeng Stream members Ms. Sonti Winnie Ms. Cetywayo Nonkosi Mr. Mhlanga Mlandela Mr. Kabang Makola Mr. Mohamed Hishaam Mr. Ngoma Mzwandile Mr. Moodly Pat Ms. Seabi Petunia Mr. Joordan Gabriel Mr. Manuel Joseph Mr. Mokoena Joyce Ms. Glanz Lorraine Ms. Bridget Shabalala Ms. Stone Lorraine Ms. Mahlangu Venile Mr. Harmelen Nico Mr. Mguqulwa Vuyane Ms. Tladi Raesibe Page 26 Evaluation, Impact & Effectiveness Assessment of the CJSP A2 Report Project Code: A2 Sub-Programme: Court Services Project: Integrated Case Flow Management System’s Extended Pilot Project (ICFMSEPP) 1. Background a. Project Problem Statement The completion of a criminal court case with EASE (Effective, Accessible, Swift & Efficient) relies on the collaboration between all parties involved in the Justice, Crime Prevention and Security (JCPS) value chain. Each party in the value chain has a separate reporting line viz. Department of Correctional Services, South African Police Service, the National Prosecuting Authority, Court Services, the judiciary and legal counsel (private and Legal Aid). There are also witnesses, defendants and victims involved. The lack of collaboration and integration across the value chain is the most significant cause of extended case cycle times and growing case backlogs. The judiciary is expected to manage case flow by leading and managing all role players. The judiciary, however, are not equipped with a guide or set of standards for case flow management in South Africa. In addition, they are not equipped with information and reporting to measure and manage the role players in a case. Some courts/judicial clusters have independently developed and implemented their own case flow management approach. In the past, there has been no uniform standardized system for case flow management to ensure consistent quality. The use of case flow management is also not actively encouraged and the judiciary is not compelled to adopt the practice in the courts under their leadership. b. Project Purpose To mobilise the Judiciary to play an active and leading role in case flow management. To support the Judiciary in determining, testing and implementing a best practice approach to case flow management. 2. Project Definition a. Objectives • Reduce case backlogs • Reduce case cycle times b. Key Deliverables • Development of a homegrown Integrated Case Flow Management System – Understand current flows – Codify best practice – Synthesise into an integrated set of processes, mechanisms, standards, structures, roles & responsibilities • Pilot implementation of the Integrated Case Flow Management System at 6 sites • Monitor pilot and refine Integrated Case Flow Management System based on learnings. • Develop a national roll out plan for Integrated Case Flow Management. c. Key Stakeholders • DoJCD Court Services • The Judiciary • NPA Page 27 Evaluation, Impact & Effectiveness Assessment of the CJSP A2 Report • Department of Correctional Services • South African Police Service • Legal Aid • Private Legal Counsel d. Key Project Team Members Project manager: Memme Sejosengwe 3. Project Performance NOTE: The evaluator had difficulty obtaining a meeting with the project manager and only met with her on 26 August. This was followed by further delays in obtaining a complete contact list for surveys and interviews. This limited the number of respondents consulted as well as depth of consultation during the evaluation of ICFMS. The evaluation that follows reflects information gathered from these respondents, one interview with the project manager, a meeting and feedback from the LCMC CFM sub-committee and available reports and documents. The evaluation would have benefited, in terms of detail, had there been time available to survey more respondents and hold a follow-up meeting with the project manager. a. Progress against Project Milestones Key Milestones Status Planned End Date Actual End Date Risk of Delivery Feasibility study Completed G Workshop with international expert to customize ICFMS and develop a pilot plan Completed Oct-02 Oct-02 G Pilot in 6 sites1 In progress Jun-03 A Pilot has not been implemented at all 6 sites Review and evaluation ICFMS til t it Jun-03 R Dependent on closure of pilot Define registrar role and business processes R Dependent on establishing a workable model for interaction with stakeholders Finalise a roll-out plan for ICFMS R Dependent on establishing a workable model for interaction with stakeholders Notes: 1 Pilot is running in Wynberg Magistrates, Durban Magistrates and Soshunguve Magistrates. For various reasons, pilots are not operating in Johannesburg and Pretoria Magistrates or High Courts. b. Project Management, Monitoring and Reporting A steering committee has not been established for ICFMS. A project plan covering high level activities and budget was developed for the project manager. Documents for each pilot site were provided – these, however, only gave insight into the design of case flow management in each site. The milestones documented above were identified through a review of CJSP performance reports. Based on feedback from pilot sites, there is no formal project level forum or monitoring process for reporting progress in pilots. In Wynberg and Durban, the participants proactively prepare reports for their stakeholders and these are used to refine case flow management and address challenges at those sites. An activities manager was appointed on a contract basis to provide project management capacity. Unfortunately, the appointed resource found alternative permanent employment within the DoJCD. Page 28 Evaluation, Impact & Effectiveness Assessment of the CJSP A2 Report c. Progress against Key Performance Measures The following measures and targets have been identified for the project: • Increase average court hours from 3hrs to 6hrs • Reduce case backlogs by 10% Performance against the above measures is not tracked at a project level. The reasons given for this are: • Lack of a uniform system, • Lack of definitions of measures Page 29 • Use of manual data collection makes data integrity questionable and comparison meaningless. d. Skills Development and Change Management A workshop was held in October 2002 where George Gish, an international expert in case flow management presented best practice and guided the delegates towards developing a South African case flow management system. The workshop was attended by 56 representatives from the Justice, Crime Prevention and Security (JCPS) cluster. e. Notable Outputs and Successes to Date An important output of the seminar in October was the development of design plans for case flow management at the pilot sites. The international expert also assisted in the development of tracking/reporting forms (case flow management notebook) to support implementation of case flow management. Pilots were implemented during 2003 at Durban Magistrates, Wynberg Magistrates and Soshanguve: Durban Magistrates Office A meeting was held at inception of the pilot where all role players agreed on approach to implementation. An acting registrar was appointed in the Durban Magistrates’ Office. The acting registrar has, with the support of the case flow management notebook, initiated a number of administrative processes to support case flow management. These are manual processes supported by part-time secondment of 13 administrative staff at the Magistrates’ office and at the courts. The manual processes focus on the capturing of court rolls and related data (time in court, reason for delay), ensuring sufficient data is collated for meaningful report. The acting registrar has through these manual processes been able to provide the JCPS role players in Durban with detailed reports on the performance of the courts in the Durban cluster. The acting registrar has identified areas where improvements can be made to the processes. Wynberg Regional Court The Wynberg Regional Court has created a temporary position called the “Court Nag” to facilitate case flow management. The Court Nag’s role is to co-ordinate the role players of a court case to ensure all necessary tasks and dependencies are addressed in order for a case to proceed in court. During the course of 2003, the JCPS role players in Wynberg met on a monthly basis, where key issues affecting case flow management were discussed. Soshanguve Magistrates Courts The magistrate was unfortunately not reachable during the evaluation. A review of CJSP reports indicates that JCPS role players in the area have met and work together towards addressing issues impacting case flow management. f. Current Project Challenges, Dependencies, Risks and Issues Magistrates Implementing Independent Case Flow Management Initiative The magistrates have resisted a Court Services lead introduction of ICFMS. They have had reservations about a model based on US experience, despite being involved in the workshop that designed the model. Additional meetings and workshops have been held with the judiciary to overcome these reservations but have not achieved this objective. Evaluation, Impact & Effectiveness Assessment of the CJSP A2 Report Page 30 The Lower Courts Management Committee appointed a Case Flow Management (CFM) sub-committee in September 2003 to investigate the state of case flow management in the lower courts and prepare recommendations on how case flow management should be implemented in the lower courts. They have, in collaboration with the prosecution, conducted assessments of courts. The sub-committee met at the end of August to review their findings and prepare a draft guideline for case flow management. This guideline is expected to become the primary reference for case flow management and as such will be revised and published on a regular basis. Apart from the concepts of CFM, the guideline also seeks to define the roles of all stakeholders in CFM and the objectives of establishing CFM. The task team has set the following plan for finalizing and implementing their case flow management guidelines: Final draft Case Flow Management (CFM) guideline 31 October 2004 “Indaba” to present CFM early November 2004 Printing and distribution of guideline 31 December 2004 Begin roll-out of CFM February 2004 Sub-committee visits country-wide Feb – July 2004 This initiative is being run independently of the ICFMS project and of Court Services projects. A junior official has represented Court Services at sub-committee meetings. While the sub-committee has identified key areas of support required, their plan is limited to the introduction of the CFM guidelines and concept. The intention of the “Indaba” is to bring all role player in the JCPS cluster to review the CFM document and discuss its implementation. Certain Pilot Sites not Implemented Pilots were not implemented in Johannesburg Regional Court, as well as high courts in Pretoria and Johannesburg. The Johannesburg Regional Court has resisted pilot implementation on the basis that a case flow management process has been in place for many years at the regional court. The High Courts have not entered pilot phase due to changes to judicial rules required to enable the case flow management process. This is dependent on the implementation of the Single Rule Making Authority. Furthermore, high courts have traditionally had the support of registrars / clerks of the court, who in effect provide case flow management support. Interaction between DoJCD National Office and Pilot Sites Pilot sites interviewed indicated that there was little or no interaction between the National Office and the pilot sites. The implementation of case flow management at each site has been without centralized guidance or co-ordination. ICFMS has not been institutionalized in these areas and this is impacting the sites from addressing important resource and organizational constraints. The common issues facing pilot sites include: • Accountability and commitment from all role players in the JCPS cluster • Human resources and funding to support case flow management • IT infrastructure Roll-out of the Integrated Justice System (IJS, also known as Case Roll Management System) IJS is a system designed by a Middelburg magistrate. IJS is seen as the preferred interim system to support ICFMS. Automation is an important enabler of consistent reporting and measurement to drive case flow management. It will avoid capacitation of costly manual processes. The system has been successfully piloted in 46 sites. The DoJCD does not currently own the system and the intellectual property remains with the developers. The system has not been rolled out because of delays in the procurement process – driven by a dispute between the DoJCD and SITA on how it should be procured. 4. Effectiveness Indicators Effectiveness indicators have not been defined for ICFMS. Evaluation, Impact & Effectiveness Assessment of the CJSP A2 Report 5. Sustainability The key factors for sustainability of the ICFMS are: Factor Status Sustainability Infrastructure Apart from the IJS implementation documented above, the DoJCD is also implementing the Court Process System, a document management system and suitable network/connectivity at court level. These IT upgrades/implementations are expected to take place over the next 5-10 years Poor / Limited ICFMS can be introduced to all courts on manual infrastructure – at a cost. Without IJS, however, reporting will continue to lack integrity and the judiciary will be unable to assess and manage consistency in the application of the principles of case flow management. There is also a risk that the various system implementations are not made in an integrated manner, impacting consistency in case flow management Project Management and Co-ordination Roll-out of ICFMS will require the co-ordination of training, change management and implementation at over 700 courts. Each court will require customization. To ensure the objectives of the project are met, the project will also need to monitor the quality of the implementation. Currently, the project does not have the capacity to provide this level of project management. The Re Aga Boswa project is only expected to deliver the Court Services structure, staffing and processes for case flow management. This covers just one of the many role-players and aspects of implementation. The judiciary have a strong desire to lead implementation, but are not skilled or resourced to also project manage implementation Poor It appears that no workable project management approach exists or has been planned to ensure effective roll￾out of ICFMS Change Management Currently two of the 6 key role players (i.e. Court Services and the judiciary) are actively involved at a strategic level in the design and implementation of ICFMS. Those two role players are working on case flow management independent of each other Poor Successful implementation of ICFMS will require a change in mindset and in collaboration that requires the involvement of all stakeholders. The stakeholders do not yet have a forum to jointly manage and govern the project Future funding Delays in resolving issues with IJS have led to budget being lost. The project has spent a fraction of funds made available by the CJSP Cannot evaluate Insufficient information to evaluate funding sustainability Page 31 6. Key Findings ICFMS is the project with the most significant direct impact on the objectives of the CJSP (i.e. on EASE). It is enabled by other projects / programs (eg. Re Aga Boswa, Case Managers Pilots, Transforming the Judiciary, IJCC). Without ICFMS, however, it is unlikely that these projects will have the full desired impact on EASE. It should thus be considered the highest priority project in the CJSP. To date, the project has completed some tasks but has much progress to make before it delivers impact. Resistance from the Judiciary and inconsistent pilot implementation have prevented significant results from being achieved by the project. Evaluation, Impact & Effectiveness Assessment of the CJSP A2 Report Despite this, the project is considered relevant by stakeholders and the ICFMS project has at least catalyzed a change in approach to case flow management. The LCMC, in establishing the Case Flow Management sub￾committee, has shown their commitment to addressing weaknesses in case flow management. The pilot sites value the benefits already delivered by case flow management and have a keen understanding of the challenges facing them in institutionalizing ICFMS. The ICFMS project is not currently positioned towards delivering a sustainable, institutionalized case flow management model. The project needs to address shortcomings in infrastructure, change management and project management. To avoid failure of the project, a thorough in-depth assessment and review of the project design and approach should be conducted: Alignment of the ICFMS Project and the CFM Sub-committee Plan • Engagement and role definition of all stakeholder at a strategic level is a critical enabler for transforming case flow management in courts. • The Lower Court judiciary has mobilized itself and has shown their intention to take an active role in the case flow management concept. Their plans include drawing together the role players in the JCPS cluster. It will be a significant challenge for the judiciary to convert their intention to a practical leadership role that brings together stakeholders at a strategic level to determine the way forward for implementation. • The Court Performance unit of Court Services should actively support the judiciary, rather than work in parallel with them. • The two parties should collaborate and drive the establishment of an influential national steering committee for case flow management to provide strategic oversight and decision making on key issues. The steering committee should be composed of representatives from all impacted parts of the JCPS cluster. Engagement of the high courts will also need to be considered. • The committee should review findings of the CFM sub-committee and the ICFMS project to determine a reinvigorated strategy for driving ICFMS going forward. • Each role player must be given distinct and clear responsibilities in delivering the project. • The committee must make recommendations on the funding (DoJCD and donor) of the implementation of ICFMS. Standardised Approach to Implementation Page 32 • Other development projects responsible for delivery to multiple sites have benefited from standardized project design, project monitoring / measurement, change management approach, governance structure and training strategy. • The steering committee (or a task team appointed by it) should design a standard methodology for implementation that can be used by each court during the roll out process (or pilot sites, depending on the strategy for ICFMS). • Each court should be empowered to adapt the case flow management to meet their unique needs. This should however be within strict guidelines, as determined by the steering committee. Project Management Capacity • Creative solutions should be sought to provide hands-on project management, especially considering resource constraints. Project management must be well positioned and empowered to deliver in an environment of complex and transforming stakeholder relationships. • The steering committee should review what has worked well in other projects or departments. The CJSP should also review where there is capacity in or alignment with other projects that could be leveraged. • The role of Court Services in ICFMS should be reconsidered and should be focused on Court Services’ areas of responsibility in enabling ICFMS (IT, Court Management Model/Re Aga Boswa refinements & capacity, legislative framework, business process) Evaluation, Impact & Effectiveness Assessment of the CJSP B1 Report Project Code: B1 Sub-Programme: Sexual Offences and Community Affairs (SOCA) Project: Public Awareness Campaigns (PACs) 1. Background a. Project Problem Statement Guided by the broader strategic notion of improving access of the justice system to vulnerable groups, the project is a response to the significance of underreporting of sexual offences and domestic violence cases. The victims of the victims of sexual offences and domestic violence are mostly women and children. Many of the victims do not report the cases due to ignorance of the nature of the phenomenon, ignorance of their rights and of the availability of sexual offences and other courts as channels for successful resolution of cases. From the interviews held with different role-players problem analysis had been done through different surveys. It would enhance problem theory if these surveys could be adequately referenced in the project documents and reports, to avoid piecing together different pieces of information and in the process omitting valuable and rich institutional knowledge. The danger of eroded institutional knowledge is phenomenal as original project theory and hypothesis may be distorted. Knowledge management is essential for institutionalization purposes and for future evaluations and assessments. The PAC is an innovative model and should be properly documented for replication, reporting purposes and for effectively selling it as a model. Knowledge management would provide clarity, concreteness and specificity on the project and would enhance project benefits and impact. b. Project Purpose The purpose of the project is to raise awareness and support victims of sexual offences and domestic violence to report and testify with confidence in cases of child maintenance, sexual offences and domestic violence and to take advantage of the SA Criminal Justice System. The project purpose level statement states the direct benefit to be derived from the project by the target groups as raised awareness and confidence to testify and report cases. Social benefits of leading a human rights-based life for women and children are expected to follow. c. Objectives To reach out and conscientize women and school going children in South African communities on their human rights, child maintenance issues, the prevalence, incidences and impacts of sexual offences and domestic violence, the existence and availability of the sexual offences and other courts; Assistance Programs as well as the Support Networks (of Victims of sexual offences and domestic violence in SA). The target groups are clearly women and children. What the project does “reach out and conscientize women and school going children” is clear in the project objective statement. 2. Project Definition a. Key Deliverables The project is about Public Awareness Campaigns to raise awareness, delivered through community workshops for women and trough industrial theatres aimed at Primary school-going children. • Primary School Awareness Programmes • Community workshops for adults The deliverables are achievable and have clear timeframes. Page 33 Evaluation, Impact & Effectiveness Assessment of the CJSP B1 Report b. Key Stakeholders Children - those who have been/are victims and those who are vulnerable Women - those who have been/are victims and those who are vulnerable The project delineates the target group as vulnerable women and children and actual victims of sexual offences and domestic violence. The project targets communities identified by Senior Public Prosecutors (SPPs) as high-risk areas, which is a reasonable strategy, given the fact that the project can only be introduced incrementally. The fact that the project includes women and children, who are vulnerable, lends great impact to the project as it incorporates a strong preventive element in its approach. This factor adds value and high relevance to its existence and nature. This approach creates a spirit of “I am not alone in this” as victims learn and start to understand the phenomenon of social abuse and its manifestations. Women start to address issues as women and children stand for each other. Evidence of this can be found in the reports that school friends have reported abuse on behalf of friends in some of the schools after the “Speak Out” industrial theater (also illustrating immediacy in effectiveness). c. Key Project Team Members Sponsor: Adv. Thoko Majokweni (SOCA Sub-Program Sponsor) Project manager: Adv. Nolwandle Qaba 3. Project Performance Primary School Awareness Programme The school awareness campaign was first piloted in 51 schools in the KZN PROVINCE during the international children’s day and month of October, in 2002. The start date for the pilot was July 2002 and the official end date was September 2002. KZN Province was chosen because it was at the time the province presenting the highest HIV/AIDS infection rate. The campaign was thereafter implemented in the Free State and in Mpumalanga in 2003 through the Speak Out industrial theatre. In 2004 the play was performed in schools in Gauteng and Limpopo Provinces. The project was also extended to the North West and the Northern Cape Provinces IN June/July 2004. The theme of the 15 minutes play is to encourage the children of school-going age to speak out to a trusted person should they find themselves, or another child close to them, in the unfortunate position of being subjected to sexual abuse. Adults Community Workshops The adult workshops are organised through the assistance of Senior Public Prosecutors in select areas acting as Project leaders and working with the SOCA Project Manager. The community adult workshops are carried out through the coordination and involvement of a panel of a multi-disciplinary team from several service￾providing government Departments sharing and enlightening the audience on aspects of domestic violence, sexual abuse and on the different resolution services and processes available to them. A select member of the panel chosen by the area Senior Prosecutor facilitates the workshops. Initially the workshops were organized to reach community members directly reaching 5,286 people out of the targeted 7,200 in the 2002/2003 financial year. During the 2003/04 financial year the approach was changed and “training of trainer” workshops are organized. Leaders in select communities are identified and are organized by the SPPs, to be trained as trainers, to be in a position to address issues of sexual abuse and domestic violence within their communities. Page 34 Evaluation, Impact & Effectiveness Assessment of the CJSP B1 Report a. Progress against Project Milestones Since the school campaign is outsourced to an external implementer, project progress is viewed against the yearly plans. Key Milestones Status* Planned Dates Comments Pilot campaign in 51 schools in KZN Complete 1st yr 2002 100% complete according to plan 50 schools in Free State Complete 2nd yr 2003 100% complete according to plan 50 schools in Mpumalanga Complete 2nd yr 2003 100% complete according to plan 50 schools in Limpopo Complete 3rd yr 2004 100% complete according to plan 50 schools in Gauteng Complete 3rd yr 2004 100% complete according to plan (Schools in the North West and Northern Cape Provinces were covered during June and July 2004) Adult Community Workshops 16 Community workshops coordinated by Senior Public Prosecutors and conducted in select areas during 2002/03 Complete 2002-2003 100% 6 “Train the trainer” workshops for community leaders In progress 2004 – 2004 First workshop planned for George in August 2004, rescheduled due to insufficient attendance by community leaders The school public awareness campaigns have gone exceedingly well against the milestones, with the numbers of school children reached exceeding the original targets. The outsourcing of the Speak Out industrial theater to Pillar to Post contractor worked well in that the campaign gained external project management resources that contributed to the smooth running and organizing of the project by people who specialize in the field and who have the required expertise. Pillar to Post dedicated adequate resources in terms of organizing and scheduling the school visits. Adequate resources were afforded the project, for example there was transport and a designated driver. The director is a dedicated person, committed to changing the circumstances of children and women. The community workshops reached substantive numbers of women in the selected areas during 2002/3. For the current funding cycle the first training of trainers’ workshop was being planned at the time of the evaluation. Except for the quantitative measure, measuring the outputs (number of women attending the workshops, or number of leaders trained), there is no mechanism to capture the outcomes and impact of the workshops. No case studies were found to illustrate the pre and post effects of the project on individuals whose lives may have been changed by information derived from the workshops. It is recommended that, on a small scale, case studies of lives changed as a consequence of the workshop/training of trainer intervention, using pseudo names, be documented. This could serve as a powerful tool of illustrating effectiveness and impact. For the 2004 funding cycle the workshops are designed to focus on community leaders. The first workshop was being planned for August 2004, during the time of the evaluation. The broader plan is to hold 6 workshops in 6 sites in total. The six sites identified for the workshops are George, Upington, Nelspruit, Odi, Parrow and Bloemfontein. The criteria for the training of trainer workshops, is that 100 identified leaders should be in attendance. The Senior Prosecutors charged with organizing the workshops spend a considerable amount of time organizing these workshops. The fact that the workshops are not formalized as a key performance area for the Prosecutors works against institutionalization of the workshops by the NPS. Proper institutionalization would facilitate consideration of issues like percentage of time spent on organizing the workshops, and proper planning and costing of the Prosecutors’ time for holistic funding purposes. Page 35 Evaluation, Impact & Effectiveness Assessment of the CJSP B1 Report b. Project Management, Monitoring and Reporting The NPA-SOCA unit-based Project manager liaises with the Pillar to Post manager and with the Senior Public Prosecutors in managing the Project. Whilst she has power and authority over the Contractors delivering the Speak Out industrial theater, and approves the invoices, she has no authority over the Senior Public Prosecutors in the community workshop programme. She can only liaise, coordinate and negotiate with them and hope that things happen. In her efforts to effect viability of the workshops she has developed an action plan for the Project leaders. This is meant to facilitate the engagement and functioning of the Prosecutors in the Project. There are no policies or job requirements that enforce delivery beyond the expressed enthusiasm by the Prosecutors, and thus no control mechanisms for the Project Manager to ensure accomplishment of project criteria and accomplishment of the project within schedule and budget. Integrity and Accuracy of Reporting For the school programme, there are two levels of statistics collection for reporting rates. At one level the project manager gets the statistics on reporting numbers from the Contractor. These are numbers of children reporting at the school level. An informal arrangement has been entered with the schools, allowing the Contractor to record immediate case reporting as well as subsequent reporting happening within a two￾week window period after staging the Speak Out industrial thearte. This reporting is not formalized and not all the schools are approached for reporting. Some volunteer the information and poor infrastructure like absence of telephones from some of the schools impinge on the goodwill of the schools to stay in contact with the contractor for reporting purposes. Late reaction from the children means not all cases are captured. Cases not reported directly to the schools are not directly attributed to the Speak Out industrial theater. The number of cases reported within the two weeks window is a reaction measurement and should be understood as such. At another level, reporting rates are captured. This is at the court level. This data is channeled to SOCA through the courts. It is recommended that SOCA document its data collection procedures, policies, control measures and data quality assurance processes as well as limitations, where these are experienced. c. Progress against Key Performance Measures Performance measured by reach of the Public Awareness Campaign Measure Baseline Date Target Actual Community workshops conducted (coverage) 0 workshops 2002 10 workshops in 8 provinces 16 Community workshops conducted in all 8 provinces in 2002 including Gauteng in 2003 Reach out and conscientise women (reach) 0 2002 7,200 women 5, 286 (73%) Reach out and conscientise children in KZN Pilot 0 2002 32,400 school going children in 31 selected schools 51 KZN schools = 40,087 (24% above target) Reach out and conscientise children in FS 2003 32,400 in 50 schools in FS 44,982 (39% above target) Reach out and conscientise children in Mpumalanga 2003 32,400 in 50 schools in Mpumalanga 35,140 (9% above target) 2003 TOTAL FOR FS AND MPUMALANGA End of OCT 03 80,122 (more than the planned 64,800) Outstanding results have been achieved with the school campaigns since the implementation of the Speak Out Industrial theater. The quantitative output measures reflect an exceeding of expectations. These results, counted at project level, are the immediate effect of the school based activity and can be directly attributed to the approach used, that of targeting schools, providing the project with an automatic captive Page 36 Evaluation, Impact & Effectiveness Assessment of the CJSP B1 Report audience and also to the enthusiasm of the teachers who were always willing to accommodate the play in their schools, compelled by the knowledge that children are the silent victims of abuse within their communities. d. Skills Development and Change Management For the Senior Prosecutors, high-level reinforcement and motivation from NPS working together with NPA Senior Management is suggested as a positive change management factor constituting management support and drive. However this needs to be reinforced by supportive policies and procedures, for example job performance areas that formalise the mobilization of communities as one of the duties and responsibilities of the designated prosecutors. Rather than a low-key approach, basic and deliberate extensive preparation of the Prosecutors is suggested. The prosecutors' community mobilization-related abilities and training needs could be assessed and met with relevant skills development training (within reasonable expectations of what Prosecutors can do). It is important to strengthen the ability of the Prosecutors to identify and work with and through accepted community leaders. It is suggested that the training of prosecutors be incorporated in the project plan as a continuous repetitive process rather than a “one-off” event. This is a key institutionalization/sustainability driver, with budget implications. e. Notable Outputs and Successes to Date Output / Success Description Noted increased reporting rate at school level • Schools reporting sexual offences incidents immediately after the industrial theater • Reporting rate at PAC schools increased by 14% from 10% to 24% Increased openness about problems in general among school children • Children have since the Speak Out industrial theater reported cases of neglect and verbal abuse to the teachers, prompting most of the teachers in the visited schools recommending the incorporation and demonstration of manifestations of verbal and emotional abuse in the play Increased awareness and alertness among teachers • Teachers have been conscientised and have started to speak to parents on the social ills of abuse during parents meetings Institutional transformation – the project is innovative, breaking the normal traditional reactive norm of how the criminal justice system would work, by adopting an approach that locates the problem within the communities and dealing with it there, “taking justice to the people and to the children” Project efficiency is evident in the low cost associated with the PAC • The schools expressed an interest to have the industrial theater coming to the schools at least twice a year. This has budget implications Enthusiasm of the school teachers The project utilises an approach that promotes and enjoys a direct link with women and children High involvement of the Police and Prosecutors • Through the adopt-a-school campaign police are doing community outreach and collaborate with the NPA on the project A captive audience in children Accessibility of the campaign is an undisputable factor • Language is addressed by assigning actors that are conversant with languages spoken in the different Provinces. The awareness campaign utilises school and community halls accessible to the target group. The Speak Out industrial theater has been written at a level that would suit the level of the primary school children whom it targets Page 37 Evaluation, Impact & Effectiveness Assessment of the CJSP B1 Report Page 38 Output / Success Description Teachers are making use of available services, referring cases to the police (CPU) and to Social Workers • The campaign has led to an awareness among teachers of which services are available and which ones are not, in the chain of services required to facilitate reporting of the cases and reaching resolution f. Current Project Challenges, Dependencies, Risks and Issues Dependency Impact Expected Actions Fund availability often causes concerns and affects planning ability and remains the biggest institutionalization weakness and threat The SOCA unit continues to engage in fundraising efforts through the Project Sponsor. Future funding is required so as not to compromise the outputs achieved to date Budget allocation and planning is a common risk to projects in the CJSP. The CJSP partnership should consolidate the collective requirements of the PAC project and address the funding-related risk threatening to undermine the progress made, by exploring other institutionalization and sustainability measures and means Community participation is the key success area for the adult awareness campaigns and has not been fully realized. The numbers reached should be understood and against the universe of the women affected. Victims of sexual abuse and domestic violence do not want to be identified, a manifestation consistent with stigmatization of victims by society The numbers of women not reached yet pose a challenge to the project. The need is huge The multi-disciplinary approach already applied in the project should be maximized as an effort to address the socio-economic nature of the problem. Engagement of DACST, National House of Traditional Leaders and SALGA will be done during this phase to create the capacity for community leaders to communicate relevant messages to their communities Duration of the adult workshops, currently being one-day workshops is experienced as insufficient in effectively addressing the issues being dealt with Project Manager is proposing that the workshop duration be extended to 2 days for more intensive participation and to accommodate role￾play This has budget implications and more funds would be required to extend the days 4. Effectiveness Indicators SCHOOL AWARENESS CAMPAIGN: Effectiveness can be viewed at two levels. At one level, is the objective of reaching out to conscientise children? At this level the reach is significant in the numbers of children reached and demonstrates success. In one day the play is staged in an average of three schools reaching significant numbers. There is no doubt that the CJSP partnership has played a key catalytic role in conscientising women and children to break the silence. While the numbers reached are significant, it is important to consider the successes to date against the population of the many children that have not been reached yet. This signifies demand for the project and the need to expand the successful awareness raising. Effectiveness is also measured from the reports from the teachers who have witnessed recorded incidents of sexual abuse reported by children for the first time after the play has been staged. The non-formalisation of the school reporting system and the limited window for tracking reporting (2 weeks after the play) means that this data source should not be relied on for capturing reporting rates. SOCA is contemplating approaching the Police stations linked to the PAC schools, as a reliable data source for cases reported. Interviews held with six PAC schools revealed that teachers rated the effectiveness of Speak Out industrial theater as high. Immediate effectiveness is evident in the immediate reaction and also in delayed reaction of children reporting cases and breaking the silence. According to reports from schools visited, most of the Evaluation, Impact & Effectiveness Assessment of the CJSP B1 Report reported cases have been handed over to the SAPS for investigation (effect attributable directly to the PAC project). While on the whole, the play was rated as highly effective; the following areas of improvement were identified: • The children are young and tend to attach permanent identities to the play characters, associating the characters with either the good or the bad. Thus using one person to depict different characters (to minimize costs) makes it difficult for the children to comprehend the character changes, leaving confusion in their minds (also reinforcing the “split personality syndrome” of perpetrators who use a good side to lure children) • Actors’ fluency in the indigenous language is important for Primary School children to fully follow the play, especially in Provinces like Limpopo. The reporting rate indicator has been reported with increasing rates. The methodologies applied to obtain the rates need to be understood by all role-players, especially at the Project level. At the time of the evaluation this knowledge and information did not prevail at project level. See 3b above for recommendations in this regard, to bring about reasonable understanding of data quality and issues of attribution. Recommended effectiveness measures are: Raised awareness among school children and among adult workshop participants can be measured at the first level by conducting simple pre-test and post-test indexes like 5 question cards. The case studies compiled by Pillar to Post are useful measures of effectiveness and the same method could be extended to the Community workshops. 5. Sustainability Key factors for sustainability are: Page 39 Factor Status Institutional Readiness Project management and co-ordination SOCA Project Manager plays a significant project leadership role. This is done with the technical assistance of the CJSP-PMSO. CJSP￾PMSO staff has through the project transferred project management skills in specific project management areas like managing within budgets, and managing contracts. The CJSP￾PMSO has been hands on, involved in the collection of project related indicator statistics on reporting rates and conviction rates from the different data source points Poor Despite the fact that CJSP has provided training to the project managers on project management, SOCA would benefit from advanced project management skills and expertise. Monitoring expertise is also an area of weakness within NPA with the Research Unit not being involved in performance monitoring. Ways of building internal capacity in these areas should be considered as a necessary factor for institutionalization of the project Communication and change management This is an important area of necessary competence as the success of the PAC project – adult workshops rely on effective communication and buy-in and engagement of multiple stakeholders from different Departments and NGOs/CBOs. Change of mindsets is necessary to gain total commitment from the stakeholders on the project. The unwavering support of Prosecutors has to be demonstrated Change management is required to address Limited Evaluation, Impact & Effectiveness Assessment of the CJSP B1 Report Page 40 Factor Status Institutional Readiness deep-rooted cultural beliefs and myths associated with domestic violence Readiness for operational owners to manage implemented services/infrastructure Sufficient The campaigns are once off events whose outputs need to be managed for resolution by the different stakeholders that represent human rights or constitutional rights custodians like the courts. The courts are considered ready to manage the outputs of the campaigns as constitutional and legal frameworks are in place. With school children the teachers become the intermediaries. While teachers have institutionalized the responsibility, they need to be trained and given basic guidelines as the frontline soldiers for children. This requires policy commitments from the Department of Education. Schools with guidance teachers commit the services of the guidance teachers, but not all schools have guidance teachers Level of skill and capacity Limited Teachers have limited knowledge on how to offer basic support and encouragement to children. There are no Social Workers at the schools and limited Social Work resources weaken the help-chain to the detriment of the reporting victims. Prosecutors are not trained in community mobilization/organisation but are expected to work closely with local leaders to mobilize substantial numbers for the workshops Senior management support Sufficient Organization structure Limited Organisational structure of the NPS requires proper acknowledgement and naturalization of the community outreach element of Senior Public Prosecutor job for phased institutionalization and sustainability to take form Future funding The Speak OUT campaign received R1.6 million from the NPA and has received assistance from another donor for the printing of the Speak Out booklet in all languages for distribution during October 2004. The community workshops need to be Limited PAC is heavily dependent on donor funding and is in a continuous and often “catching up” fundraising mode for the project to continue its life. Evaluation, Impact & Effectiveness Assessment of the CJSP B1 Report Page 41 Factor Status Institutional Readiness integrated into the court services with proper budgeting arrangements Interdepartmental coordination Limited Interdepartmental coordination needs to translate to supportive policies that reinforce the multi￾disciplinary approach 6. Key Findings a. Project Performance School campaigns exceeded expectations. The community workshops reached significant numbers. The community workshops are dependent on an intensive community mobilization and leadership identification strategy, which requires effective coordination and systematic engagement of the SPPs accentuated by deliberate involvement and training. b. Impact and Effectiveness Schools have become aware of the prevalence of other forms of abuse (verbal and emotional abuse) and the need to address these. The schools are requesting that the play be brought to the schools at least twice each year. The demand is high. The school awareness programme has made significant differences to the lives of the children who were helped by the industrial theater to break free from their own silence and reported sexual abuse. These children’s lives have been changed for the better. To them the awareness program has made both an immediate and a long-term impact. Impact: The PAC together with the other project components of SOCA has led to the recognition of SOCA as the most innovative public programme by Standard Bank, and being awarded a National CPSI (Centre for Public Service Innovation) award. c. Institutionalisation and Sustainability Future funding and interdepartmental coordination are key variables that need to be in place for the commitment and ownership of the project by NPA to reach reasonable levels of sustainability. Evaluation, Impact & Effectiveness Assessment of the CJSP B2 Report Project Code: B2 Sub-Programme: Sexual Offences and Community Affairs (SOCA) Project: Case Managers’ Extended Pilot Project (CMEPP) 1. Background a. Project Problem Statement Currently prosecution of sexual offenses in SA is poorly managed and victims are often exposed to forms of secondary victimization. The Case Managers Extended Pilot Project seeks to ensure that there is efficient and effective sexual offenses prosecution in South Africa. Additionally, to reduce secondary victimization of complainants by ensuring that dockets are readily available and that complainants do not wait too long before their cases are heard in court. b. Project Purpose The Project Manager and the Case Managers state the project aims as: • To ensure effective and efficient sexual offences prosecution. • To reduce secondary victimization of complainants. 2. Project Definition a. Objectives To appoint four additional Case Managers at facilities that are collocated with the Thuthuzela Care Centers in order to prepare and streamline case rolls through the Criminal Justice System and thereby increase the prosecutors and magistrate’s average court hours from 4.5 to 6 hours per day and reduce Case Cycle Times from 9-24 month to 6-9 months. b. Key Stakeholders The beneficiaries of this pilot project are describes as communities and victims of sexual offences and domestic violence in the following areas: • Wynberg, Western Cape • Umlazi, KZN • Soweto, Gauteng • Bloemfontein, Free State • Galeshewe, Kimberly, Northern Cape Page 42 Evaluation, Impact & Effectiveness Assessment of the CJSP B2 Report c. Key Deliverables Milestone/Task/Activity Expected Impact Track and monitor cases All reported cases have dockets and are heard in court. Case Managers have a database of all reported cases. This makes it easy for them to track and monitor the cases Facilitate the cases Prosecutors have more time in court Assist in managing the court rolls Prosecutors are able to present cases with sufficient information Secure the attendance of witnesses at court The number of Victims appearing in court has increased. Assist in the process to improve relationships Prosecutors are trained and attend multi-disciplinary team meetings Follow-up with relevant role players Follow-up on witnesses, investigators, lawyers etc. occurs to ensure that they are present in court Issue subpoenas to witnesses required for prosecution Prosecution is more effective when witnesses attend court sessions Assist in increasing the conviction rate Conviction rate has increased at both centers. The Wynberg Case Manager reports that there is 81% conviction rate Assist in increasing average court hours Court hours have increased from a minimum of 3 to an average of 7 hours Assist in decreasing case cycle times The case cycle has decreased from 9-24 months to an average of 6 months Assist in reducing secondary victimization of survivors Because of improved court management victims do not have to wait too long in queues The information above pertains to Bloemfontein, Umlazi, KZN and Wynberg in the Western Cape. The Case Manager for Soweto resigned in January 2004 and interviews for a New Case Manager for both Soweto and Galeshewe were conducted in August 2004. 3. Project Performance a. Progress Against Project Milestones Milestones Status Comments Appoint additional Case Managers In progress At the end of September 2003 there were four Case Mangers appointed. By the end of August 2004 there were three Case Managers within the Project Prepare and streamline case rolls In progress This is happening in Wynberg, Umlazi and Bloemfontein Reduce case cycle time from 9-24 months to 6-9 months In progress Case Managers report that through this project the case cycle time has been reduced an average of 6 months Increase average court hours from 3-6 hours In progress Case Managers at Umlazi and Wynberg report that the average court hours have increased to 7 hours per day Orientation of the newly appointed Case Managers In progress Orientation takes place before Case Managers assume their responsibilities Please note that the above information pertains to Bloemfontein, Umlazi and Wynberg because there were no Case Managers at Soweto and Kimberly during information gathering for this evaluation. Page 43 Evaluation, Impact & Effectiveness Assessment of the CJSP B2 Report b. Progress Against Key Performance Measures The performance of this project is measured via the following indicators: Measure Baseline Targets Total cases reported 52,975 To reduce by 50% Cases dealt with 24,221 Cases should be dealt with within 6-9 months Total cases finalized 8,400 Cases should be dealt with within 6-9 months Total outstanding cases 44,531 Cases should be dealt with within 6-9 months Conviction rate 48% To increase by 10% Average court hours 3-6 hours 7 hours Case cycle time 9-24 months 6 months Reference: Induction presentation (August 2004) 4. Effectiveness Indicators The Case Managers Extended Pilot Project is demonstrating effectiveness in areas where it is being implemented. The Pilot project is seen as effective in that: • It brings stakeholders such as victim support programme, Child Line, Counselors and the police together. • There is positive feedback from trainees and their inputs into what future training content should be. • There is less time spent on cases and complainants are now coming to court as opposed to when complainants would report a case and not attend court proceedings. This indicates that confidence in the justice system is increasing and that Case Managers are effectively doing their work by assisting both complainants and witnesses to attend court proceedings. 5. Sustainability Key factors required for sustainability are identified in the table below: Factor Status Institutional Readiness Project management and co-ordination The project is within the NPA’s mandate to establish an efficient and effective unit which strives to reduce the victimization of women and children Sufficient within the NPA Level of skill and capacity The capacity of the project is threatened by the short-term employment contracts for Case Managers. The current Case Managers are not sure whether their contracts will be renewed At the beginning of the 2003/2004 Financial year, the project had four Case Managers. One has since resigned Senior management support The project has support at the highest levels within CJSP and NPA Sufficient Organization structure The Case Managers Extended Pilot Project is clearly defined within the organizational structure of NPA Sufficient Future funding Future funding has been identified as one of the critical areas to ensure sustainability of this project Limited Stakeholder engagement / enthusiasm Stakeholders include SAPS, NGO’s, Health workers and social workers Sufficient Page44 Evaluation, Impact & Effectiveness Assessment of the CJSP B2 Report 6. Key Findings The pilot project has been effective in increasing court hours and conviction rates as well as reducing case cycle times as a result of the multi-disciplinary process adopted, the commitment demonstrated by case managers and key stakeholder buy-in. However, Institutionalisation by SOCA is urgently needed to ensure future funding to support the retention of case managers. Page 45 Evaluation, Impact & Effectiveness Assessment of the CJSP B3 Report Project Code: B3 Sub-Programme: Sexual Offences and Community Affairs (SOCA) Project: Victim Assistance Officers’ Extended Pilot Project (VAOEPP) 1. Background a. Project Problem Statement South Africa is no exception to the notorious systematic secondary victimization that many countries are striving to eliminate from their court systems. South African courts have recorded significantly low conviction rates in sexual and domestic violent crime. In many of the cases, the victims refuse to testify or to cooperate with the state, mostly because of fear of reprisals. For this reason the SOCA unit initiated the Victim Assistance Program in 2002. The program aims at supporting/assisting and preparing the victim witnesses for court to ensure that they have all the information they need regarding the court processes and their own role in court as well as the role of other court personnel. The problem statement isolates the problem clearly as low conviction rates, high attrition rates (withdrawal of cases) and applies a clear cause-effect theory in stating the causes for the phenomenon as fear and as ignorance on court processes. The problem analysis has led to a clear and compatible intervention strategy (i.e. the conceptualisation factor). b. Project Purpose To immediately appoint four (4) Victim Assistance Project Officers (VAOs) at the Thuthuzela Care Centres, to safeguard the needs, concerns and the interests of the victims throughout their interaction with the Criminal Justice System, including the court process. Safeguarding has been defined in separate documents using other statements as “to assist victims as they go through court process and to help them reduce the psychological shock and trauma that victims can suffer by lending emotional support and practical support shortly after the incident/after the initial crisis with emotional support, counselling, advocacy, referral to local social agencies and information on matters such as the status of the investigation, the status of the accused, the court system and the right of victims within it. The direct benefits for the target group are specified as reduction, relief from suffering shock and trauma, lending the project legitimacy and strong people – level cause in the eyes of those that are funding and those implementing it. Anticipated Milestones also stated as Project Goals (March 03 report) • Reduce case cycle time from 9-24 to 6-9 months • Reduce secondary victimization by 20% • Reduce fear of survivors by 20% • Increase average court hours from 3-6 hours • Increase morale and motivation of staff With the project purpose defined as an output(tish) purpose, “appointment of VAOs” the anticipated milestones stated above become larger than the project purpose statement and present themselves as higher, overall and ultimate in nature. The milestones clearly identify the victims of the sexual offences and domestic violence as the beneficiaries of the project with the officers bringing the necessary intervention. Together with the people level results are anticipated systemic changes, for example court hours, indicating that the project is multi-layered in its approach. The CJSP project purpose is primarily funds-driven, determined and crafted in terms of what the funds are meant to achieve. Page 46 Evaluation, Impact & Effectiveness Assessment of the CJSP B3 Report 2. Project Definition a. Objectives • To contribute to the effective utilization of the legislation aimed at the reduction of all forms of violence against women and children • To ensure maximum utilization of all the services provided with regard to the 3 identified areas in each province, including the services provided by the NGOs and the CBOs • To heighten public intolerance to violent crimes against women and children and thus encourage increased reporting of these offences • To establish lines of communication between the service providers in the Criminal System and the victims of violence, as well as the broader community • To contribute to the restoration of public confidence in the criminal justice system The deliberate and logical: ACCESS-DEMAND-UTILISATION intent and chain that this project enforces, needs to be captured in all its presentation. This recognition would promote a comprehensive and holistic promotion of the project for funding purposes. b. Key Stakeholders The sub-program is targeted at the appointment of officers. The beneficiaries of the project are the communities and victims/survivors of Sexual Offences and Domestic Violence in the following selected pilot areas: 1. Mannenburg, Gugulethu and Khayelitsha – Western Cape 2. Mdantsane- East London in the Eastern Cape 3. Soweto in Gauteng 4. Umtata/Libode – Eastern Cape 5. Galeshewe - Kimberley c. Key Project Team Members Sponsor: Adv. Thoko Majokweni (SOCA Programme Sponsor) Project manager: Adv. Buyi Nkala d. Reporting of Statistic-Based Indicators The CJSP gets the statistics from SOCA. SOCA reportedly derives these statistics from surveys and polls commissioned and conducted from time to time by contracted survey and research institutions. 3. Project Performance a. Progress against Project Milestones Key Deliverables The key deliverable is the appointment of victim assistance officers to be attached to selected Thuthuzela Care centres. This key deliverable has been achieved, with the five officers appointed through the CJSP. Appointment of victim assistance officers has a maintenance side to it in broader human resource management terms, essential for project continuity. As such, the project could have been expressed and funded in totality for institutionalisation/sustainability purposes, to include milestones/processes along the lines of: • Proper induction or orientation within a defined period • Fully and properly equipped offices in which VAOs operate • Proper integration of the officers within the TCCs • Formal skills development plans existing and planned for in response to expressed training needs Page 47 Evaluation, Impact & Effectiveness Assessment of the CJSP B3 Report Successful appointment of staff does not occur in isolation from the above. The responsibility would be assigned to the Project Manager within the balanced scorecard objective of: INTERNAL BUSINESS PROCESS - IMPROVE SYSTEMS FOR EFFICIENT AND EFFECTIVE SERVICE DELIVERY. Taking responsibility for the broader appointment of victim assistance officers is a key institutionalisation factor, currently limited by the narrow definition and scope of the project. Central to the appointment of the victim assistance officers (VAOs) is the question of accessibility to the communities being served. The Thuthuzela Care Centres are by design located within public hospital settings, as victims are likely to seek medical help from hospitals after victimization. This arrangement makes the officers truly accessible. Project Performance In terms of the secondary part of the objective statement, i.e. “safeguarding of victims”, included as the latter part of the objectives, the following milestones offer an important opportunity for reviewing project effectiveness. Key Milestones Status* Comments Establish and maintain early contacts with the victim after the case is reported and guide them through the court process Service being rendered as part of daily job operation The early contact is facilitated by the one stop service centre concept with the officers available at the centres where the victims report the cases to the SAPS, receive medical examination, counselling services and are referred to the officers. VAOs reportedly also get details of reported cases from the case managers and establish contact with the victims who may have had no knowledge of the service Coordinate medical examinations and reports for evidence Service being rendered as part of daily job operation The officers work with the nurses and the doctors to achieve this coordination. The shortage of doctors is the major obstacle experienced by officers in their efforts to render speedy services. Victims wait for the doctors who are not available for up to 24 hours in the centres Offer guidance to the victim through court processes Service being rendered as part of daily job operation This is a key element of the officer’s job Maintain regular communication with the victim and the investigating officers, providing the victim with relevant information Service being rendered as part of daily job operation Communication and information giving is a key element of the VAO’s job. The officers liase between the investigating officers and the victims, soliciting vital information from the officers on progress issues and making sure that the victims understand the legal processes and formalities as applicable to their cases. This is found to be a critical factor of support that sustains the victim, contributing to continuing interest and cooperation in the case throughout the duration of the case and contributing to success in convicting the perpetrator. Without the support of the VAOs, communication breakdowns between the investigating officers and the victims occurs leading to loss of hope, fear and minimized opportunities of convicting the perpetrator. The officers feel this information is void (a key project effectiveness factor) Page 48 Evaluation, Impact & Effectiveness Assessment of the CJSP B3 Report Page 49 Key Milestones Status* Comments Work with sexual offences special prosecutor to prepare the victim for court processes seminar Service being rendered as part of daily job operation The victim assistance officers have created relationships with the case managers and the prosecutors and are able to communicate the victims’ concerns to the prosecutors, creating a vital connection that maintains the independence of the court officials. Within the centres, VAOs have managed to create a working relationship with the other officials in the TCCs applying coordination and teamwork to create effective service delivery and resolution of work obstacles. An example of successful managing of conflict involves a case where ambulance attendants were not prioritising TCC cases. This was raised by the particular VAO and resolved amicably to the benefit of the clients Provide emotional support in order to minimize secondary victimization Service being rendered as part of daily job operation Consideration of safety plans is part of the assistance strategy offered to victims. Where removal is necessary, the officers liase with relevant NGOs or Social Workers to remove victims to places of safety, effectively minimizing the chances of secondary victimisation Ensure that Sexual Offences cases are trial ready Service being rendered as part of daily job operation Substantive evidence collected timeously in a friendly environment ensuring that most cases are trial ready, is the value added by the multi-disciplinary approach. The VAOs offer the victims court preparation and orientation, ensuring their readiness as witnesses Help increase the conviction rate by 10% This is the ultimate change that VAOs contribute to, by performing at the job level. Elimination of secondary victimization; Elimination of hostile police environment; Ensuring that victims can identify the perpetrators without fear, are conditions that the VAOs facilitate to promote convictions. Coordination between the VAOs and court officials increase the opportunities for achievement of this result (milestone) At the project level, performance is measured through quantitative indicators • Number of cases reported as a performance measure of the VAO • Number of VAO reported cases dealt with • Total number of cases reported at the Thuthuzela centres (UTILISATION) • Number of cases dealt with, that were handled/assisted by the VAO, showing significant involvement of the officers • Number of cases dealt with, that were finalised, reporting significant finalisation rates • Percentage of survivors shielded from secondary victimization through their cases receiving support, reflecting increases in percentage rates • Average reporting rate of survivors of domestic violence and sexual offences at the TCCs where VAO are deployed, with increasing percentages, recorded (DEMAND) • Average court hours at the THUTHUZELA Rape Care Centres where VAOs are deployed, recording more than one hour increases (ACCESS) • Conviction rate for the VAOEPP, recorded at a 59% increase for 2003 • Reduction of secondary victimization, with reduction reported • Improved reporting rate While these are good meaningful indicators, they need to be applied with full definition of key words, formal identification of data sources and methods of calculation. Evaluation, Impact & Effectiveness Assessment of the CJSP B3 Report The victim assistance project monthly statistical report captures the following statistics: • New cases • Finalised court preparation cases • Consult and referrals • Case outcomes – guilty/not guilty/w/d For accuracy, integrity and ownership, the project should have information on how percentage rates are derived / calculated from the above statistics. Effectiveness At the level at which the project objective is pitched, i.e. appointing a number of VAOs, effectiveness of the project is significant as the objectives of appointing the officers were met within reasonable timeframes. At the level of “safeguarding” the interests and needs of the victims, effectiveness is significant from the analysis of job performance grid, done above. Quality of the services is another factor that completes the effectiveness. It should be possible to deduce reasonable evidence about quality of service, from the rate at which the cases handled by the victim officers lead to successful and speedy arrest of perpetrators and from the reduction of secondary victimization. While there is passion for the introduction of victim satisfaction surveys, caution should be taken that the quest to measure does not compromise the interests of the victims who may be traumatized at the time of receiving the services (a matter of intended excellence being an enemy of the good). Again, the dependency of the VAOs on the Uniform protocols and the successful introduction and application of these in dealing with victims as well as anticipated training of VAOs on the Domestic Violence Manual should take care of quality and standards issues and their enforcement. Qualitative analysis and reporting and the use of case studies in a manner that respects the confidentiality of the victim should be relied upon for qualitative data. The reduction of the incidence of domestic violence and sexual abuse is a result of several projects enhanced by a multi-disciplinary/multi-layered approach and should be measured at a higher level than the project. This, however, does not dispute the probability that the victim assistant project may be contributing to this higher-level result, especially given the fact that the officers are involved in a public awareness campaign, going the extra mile by being involved in preventive approaches. Case cycle reduction and conviction rates are powerful and direct measures of project effectiveness, provided that they are linked directly to cases where VAOs operate and that other variables like complexity and uniqueness of individual cases are balanced in the equation. b. Notable Outputs and Successes to Date Page 50 Output / Success Description Effective introduction of an innovative model and pilot of giving necessary support to victims Prosecutors achieving success in their own job as victims are able to testify without fear and without weakening evidence Officers creating their own networks and doing own community profile and connecting with relevant institutions; creating effective working relationships and reliable referral systems using own initiative A significant endeavour that promotes and contributes to the protection of women and children, sending a strong message of non-tolerance of sexual abuse and domestic violence The concept of one stop centres has been effectively put to practice and success gained in promoting a multi-disciplinary approach to legal and human resolution of the cases Visibility of the courts and justice is being promoted with the justice system made more accessible to people and practically contributing to improving confidence and trust in the system The inclusion of victim assistance officers in public awareness Evaluation, Impact & Effectiveness Assessment of the CJSP B3 Report Page 51 Output / Success Description campaigns by other professionals is a sign of visibility of the service and its offerings Synergies of the project with the case managers is proving to have a positive effect on management and on effective resolution of reported cases A system that effectively preserves evidence to ensure convictions and bringing to book of perpetrators • Prevents secondary victimization Established protocols at the TCC reinforce promoting the multi-disciplinary approach • When a victim enters, the site coordinator makes sure that the victim sees a nurse first, then the doctor, the police, the Social Worker, POWA or other counsellor and then VOC Statistics showing an increase in the utilization of the service by women is a good indication that demonstrates demand￾utilisation and access (if and when supported with surveys of prevalence or incidence rates) c. Current Project Challenges, Dependencies, Risks and Issues Dependency Management of Synergies/DEP Impact Expected Actions Synergies with the Case Managers pilot project VAOs and case managers work closely together Synergies with PAC There is no evidence that these synergies where the officers are embarking on Public awareness are being managed. It will be necessary to coordinate and manage these synergies when the adult PAC programmes gain momentum and messages are targeted at the same audience Challenge in successful interface of VAOs in the TCCs Isolated cases of resisting the VAOs in some TCCs surfaced and has to be managed both at the centre level and at an interdepartmental level Non-management of conflict or personality problems can have a weakening effect on the project Managerial intervention from SOCA Synergies with NGOs doing similar work NGO targeting child victims and doing similar work is an example and calls for coordination and development of formal guidelines and MOUs. Also the existence of NGOs at TCCs dealing with domestic violence cases leads to a need for discussion of possible collaboration and cooperation/complimentary services Meetings are in progress in some of the TCCs to formalise these collaborations Synergy with Uniform Protocol Different approaches of court preparation from different service providers would benefit from the anticipated launch of the uniform protocol in dealing with victims Will strengthen the project as basic essentials and standards will inform requirements of both infrastructural and human resources Issue is that poverty issues have a hindering effect on the project Often, victims do not have funds or transport to attend court hearings or to come to the centres to report cases. SAPS transport is often relied VAOs are expected to identify these issues and ensure that victims receive assistance Evaluation, Impact & Effectiveness Assessment of the CJSP B3 Report upon and there is not enough of it as their vehicles are limited Synergies with the Domestic violence manuals Need to institutionalise to minimize chances of people moving on to other jobs due to insecurity VAOs are employed as contract workers and the short term nature of their contracts causes insecurity NPA expected to budget for VAOs from next financial year 4. Sustainability A huge risk factor for the institutionalisation and sustainability of the project is the uncertainty around the extension of the VAOs job contracts. The NPA needs to raise funds to sustain the project and absorb the victim officers within its own organisational structure. The officers are paid by the CJSP and there are no plans for alternative measures to retain them. Losing the officers who have the experience of piloting the service would create a gap in institutional knowledge and capacity and would have an eroding effect to the current outputs of the project. Institutionalisation of the project will also be determined by availability of funds to expand the project to other areas, to replicate the model Other Key factors for sustainability are: Page 52 Factor Status Institutional Readiness Project management and co-ordination This factor continues to be critical as there is a lot of interdepartmental coordination to do. There are many role-players and coordination is a critical factor for sustainability Poor The SOCA/NPA does not have specialist project management skills or a program unit that drives and gives technical assistance to Programme managers, within the NPA. There is a need for continuous training and technical support on programme management Communication and change management The VAOs have demonstrated good communication skills that have helped to integrate the project at local levels. However this is being threatened by the job insecurity experienced by the current VAOs Sufficient Most of the VAOs have social work qualifications and have taken the initiative to network with other role players at the local level Readiness for operational owners to manage implemented services/infrastructure While NPA plans to appoint 35 more officers, the project and institutional readiness is weakened by the lack of concrete plans for absorption of current VAOs into NPA-SOCA permanent staff structures or by the lack of plans to effect longer-term contracts for the VAOs Limited -There is a need to effect long￾term employment measures for the VAOs to avoid loosing them to other organizations or Government Departments. SOCA needs to take over the officers from the CJSP for continuation purposes. Full programming of the project under SOCA would allow for longer contract periods and would address the job security facto. -The Project Manager needs to expand her staff capacity. She needs at least an administrative Evaluation, Impact & Effectiveness Assessment of the CJSP B3 Report Page 53 Factor Status Institutional Readiness assistance person, to be effective. -Infrastructure is limited with the technology infrastructure updated. For example no email facilities exist. The VAOs’ jobs are about communication and revolve around information – email facilities are important. Sharing offices breaches confidentiality Level of skill and capacity Suggested training on domestic violence act, court preparation, conflict management, sexual offences act and rights of victims for the VAOs Needs to reside permanently with NPA in as far Program management skills and monitoring skills are concerned Senior management support Sufficient Organization structure The SOCA structure would have to be modified to allow employment of the VAOs within the SOCA organizational structure. More job positions would have to be created for proper institutionalisation of expansion of the project– this has budget implications. The multi-disciplinary nature of the structure at the Thuthuzela Care centres introduces organisational dynamics that require careful management of work relationships among staff operating from the centres. Modifications of the SOCA structure is recommended for permanent or longer-term contracts for VAOs Future funding It is critical that the BAC continue to get funding to maintain the vital link of VAOs Limited Community Participation This is a critical factor that would promote demand and utilization of the centres and VAO services VAOs are involved in public awareness campaigns Levels and availability of skills Critical - for sustainability. The Uniform protocol for victim management will hopefully prescribe levels of skills required. This will inform the staffing patterns for the VAO project. The project employs mostly qualified Social Workers, as VAOs and their skills and competencies are considered appropriate for the job Strong 5. Key Findings a. Impact and Effectiveness Unexpected consequences: Receipt of donations of Teddy Bears by the project from the United Transport Industries and Daimler Chrysler National and International Publicity for the Mdantsane Centre through Health Centre. Impact demonstrated by case study: Nokuzola’s case bent traditional justice procedures because she had the intervention of the officer. The officer alerted the court about the fact that she was reluctant to testify because she would be asked her name and address and the perpetrator would trace her. The court changed procedures and accommodated the victim’s wish not to have her name and address announced and verified in court, as this would have made it possible for the perpetrator to trace and probably kill her as he had vowed. Evaluation, Impact & Effectiveness Assessment of the CJSP B3 Report Effectiveness Effectiveness is evident in that the assistants are serving victims in ways that meets the informational, security and emotional needs of the victims and serves to meet the requirements that facilitate meeting the ends of justice: with perpetrators convicted and secondary victimization reduced, through safety plans. Conviction rates have increased, women are utilizing the centres, victims have been removed and placed in places of safety, and victims have been prepared for court – evidence of services being rendered and victims being serviced. Page 54 Evaluation, Impact & Effectiveness Assessment of the CJSP B4 Report Project Code: B4 Sub-Programme: Sexual Offences and Community Affairs (SOCA) Project: Sexual Offences Multi-Disciplinary Training Manual (SOMM) 1. Background a. Project Problem Statement The new Sexual Offenses Bill proposes substantive and procedural changes that will address gaps that the current Laws do not contain. Current prosecutors and magistrates are not equipped with the necessary capacity to respond to this bill. This project thus seeks to produce a multi-disciplinary training manual that will enhance the skills of prosecutors and magistrates who are involved in the prosecution of sexual offenses. b. Project Purpose To develop and produce an accredited Multi-disciplinary Training Manual on sexual offenses as a practical guide to the prosecution of such offenses, in order to enhance and develop expertise to ensure professional service delivery and to reduce the prevalent incidences of sexual offences. The Project Will: • Produce a multi-disciplinary manual on sexual offenses in order to equip those involved in the prosecution of sexual offenses. • Provide a manual that is not only a practical guide covering all aspects relating to the prosecution of this crime, but also to provide an acknowledged academic work, thus giving it recognition throughout the various professions involved in the prosecution of sexual offenses. • Highlight the proposed changes brought about by the Sexual Offences Bill. • Assist in deconstructing many of the outdated stereotypes that exist concerning sexual offences so as to further highlight the need for changes to the present law. c. Project Objectives To produce 1000 Training Manuals to sensitize and strengthen the capacities of Prosecutors and Magistrates in order to reduce Sexual Offenses by 10% and to increase the Conviction rate by 10%. 2. Project Performance Anticipated Milestones Status Comments To produce 1000 Accredited Multi-disciplinary Training Manuals, resources reference file and participants’ Workbook to strengthen capacities of prosecutors and magistrates in order to: • Reduce incidents of domestic violence (secondary victimization by 10% and domestic violence by 10%) • Launch and distribute the manual • Train and implement 0% complete The content of the manual will be based on the New Sexual Offences Act which is still awaiting promulgation It is anticipated that the Act will be promulgated some time in 2005 because it has been opened for public debate leading to the broadened description or definition of rape. Additionally, NPA and stakeholders, who amongst others include, SAPS, health professionals, social workers, prosecutors and NGOs, would rather wait to avoid amendments once the Act is promulgated. The project is not yet implemented pending the promulgation of the New Sexual Offences Act. Page 55 Evaluation, Impact & Effectiveness Assessment of the CJSP B4 Report 3. Sustainability Key factors required for sustainability of SOMM are as follows: Factor Status Institutional Readiness Readiness for operational owners to manage implemented services/infrastructure The NPA has identified professionals who are waiting to be called to write the manual as soon as the Sexual Offences Act is promulgated The NPA will draw on lessons from the Domestic Violence Multi-disciplinary Training Manual that has been developed. Plans for accreditation and future funding are in place to ensure sustainability 4. Key Findings The project is not off the ground as yet and thus no key findings can be evaluated. Page 56 Evaluation, Impact & Effectiveness Assessment of the CJSP B5 Report Project Code: B5 Sub-Programme: Sexual Offences and Community Affairs (SOCA) Project: Domestic Violence Multi-Disciplinary Training Manual (DVMM) 1. Background: “In United Hands, There’s Power to Conquer Domestic Violence” a. Project Problem Statement The project responded to a concern raised by Prosecutors that as functionaries at the tail end of the Criminal Justice System, they have to interact with others to ensure collaboration that would benefit the victims of domestic violence and ensure their protection by the justice system. The focus is on promoting a multi￾disciplinary and integrated approach to the management of domestic violence cases. The manual is a tool for the Integrated Training Program that seeks to develop skills, and build the capacity of service organisations in a way that is sufficient to establish sensitivity, accuracy, uniformity, integrated support, coordination, efficiency and effectiveness in the implementation of the Domestic Violence Act. The new Domestic Violence Act created new roles and responsibilities for a number of role-players and created interdependencies, which called for multi-disciplinary training as opposed to uni-disciplinary training. b. Project Purpose The project purpose was the development of domestic violence multi-disciplinary training manuals. 2. Project Definition To produce an accredited multi-disciplinary Participant training manual with the purpose of strengthening the capacities of Prosecutors and Magistrates in order to reduce incidences of Domestic Violence and secondary victimization. a. Objectives To produce an accredited Multi-disciplinary Participants’ Workbook, Resources Reference File and a Training Manual on Domestic Violence in South Africa. Specific Objectives • To familiarise the participants with the social context within which the domestic violence occurs • To inform the participants of the laws and policies applicable to domestic violence • To provide the participants with a common understanding of the roles and responsibilities of all the players in the justice system in relation to domestic violence • To educate the participants on the need to support victims and on how to assist them in the compilation of their safety plans • To increase the participants’ awareness of the skills required to properly manage domestic violence cases The project manager cited the following objectives, compatible with the project purpose of strengthening the capacities of the Prosecutors and Magistrates. • To improve evidence collection • To improve service delivery at all points of service delivery • To increase collaboration and cooperation in the management of domestic violence cases • To improve the referral system Page The specific objectives are compatible with the “strengthening the capacities” purpose. The ultimate beneficiaries are the victims of domestic violence. The objective for the CJSP is the production of an accredited training manual on Domestic Violence. 57 Evaluation, Impact & Effectiveness Assessment of the CJSP B5 Report b. Key Deliverables A complete package consisting of • A Participants Workbook, a Resources Reference File and a Multi-Disciplinary Training Manual • A Trainers Manual c. Key Stakeholders The target group is trainers and learners from organisations and government institutions involved in the referral of victims, domestic violence victim support services and in the prosecution of the domestic violence cases. The key stakeholders are Department of Justice – Magistrates, NPA – prosecutors, Clerks of the Court, Department of Health Care workers, SAPS, Independent Complaints Directorate, Department of Social Development, Correctional services, Department of Education and NGOs and CBOs, Department of Safety and Security. d. Key Project Team Members Sponsor: Adv. Thoko Majokweni (SOCA Programme Sponsor) Project manager: Adv. Tsidi Nkambula 3. Project Performance a. Progress against Project Milestones Key Deliverables Dependency Impact Expected Actions Availability of highly recognized experts as authors A pool of writers was assembled. It successfully managed to produce the manual Existence of a National Domestic Violence Programme Committee comprising of interdepartmental representatives Working committees were established per need and for each phase of the manual development. The programme committee is the next committee to be established to manage the implementation of the training Shares a common objective of sensitizing key users and reducing direct and secondary sexual offences and domestic violence, with the Sexual Offences manual. Synergies with the Victim Assistance Officers VAO s are looking forward to being trained through the manual so as to build their capacity and skills base to benefit the victims that come to the TCCs Justice College providing the training The process is on to identify accredited trainers Page 58 Evaluation, Impact & Effectiveness Assessment of the CJSP B5 Report Key Milestones Milestones Expected Delivery Date Completion Status Comments Assembling an inter￾disciplinary team of professional writing experts March 2001 100% A pool of writers was assembled and produced the manual Preliminary review of the manual July 2002 100% This was funded by CJSP and led to further review processes 2nd Review combined with the proof reading workshop September 2002 100% Validation was combined with the language simplification process step Final review March 2003 100% Achieved through CJSP funding Testing and evaluation of the manual with stakeholders participation September 2003 100% Achieved through CJSP funding Lay out, design and packaging January 2004 75% This line item was originally provided for within the registration and accreditation of the manual budget element in the budget document dated 4 February 2003. However it has been undertaken separately and alternative funds were raised for it. Time spent on raising these funds caused a setback in completing packaging by January 2004. The layout and packaging of the manual was being reviewed at the time of this evaluation Registration and accreditation of the manual The budget for registration and accreditation, including printing for submission and for launching (20 copies) is R39, 000. The budget amount needs to be informed by proper costing, derived from a comprehensive analysis of the current accreditation and registration processes. This will facilitate informed decision making and agreement on the way forward Publication of the manual The publication of the manual will only happen after the manual has been made to be unit￾standard based Implementation of the manual Preparing for training is in process. Project Manager has drafted an MOU that would ensure ownership and commitment of resources and services to avoid financial blockages. More funds are needed for implementing the training From the above analysis, one can conclude as follows. - The overall scope of the project goes beyond the CJSP project as it includes publication and actual implementation. - Proper costing, including costing of input processes along the project critical path needs to be done to inform accurate budgeting. (For example, expenses relating to the accreditation team operations) This is necessary to avoid delays or constraints that may threaten erosion of the significant outputs achieved so far. - Counterpart-funding arrangements, where these apply, should be obtained and formalized early in planning the project. - Amendments to the original budget should be communicated and formalized. Page 59 Evaluation, Impact & Effectiveness Assessment of the CJSP B5 Report b. Notable Outputs and Successes to Date Output / Success Description SOCA has introduced a significant change in the way it does business by developing the manual, taking lead and ownership of a project traditionally based in the world of Education Establishment of the Accreditation team as a sign of continuing commitment and as a strong factor of sustainability that comes with the existence of an accredited manual Lay out and design of the manual emphasized durability, inclusion of a theme portraying unity and adaptability to legal changes and modifications and as such sustainability Manual creates the platform and basis for service providers partnerships, for collaboration, for advocacy and for complimenting each other and for eradication of duplication Successful mobilization of all stakeholders including those perceived not to be major stakeholders, from the beginning of the process • Buy-in and ownership as stakeholders were involved from the development of the concept. A working committee was put together, the National Domestic Violence monitoring committee was used to sell the concept. Upon completion of mission, committees were replaced with another one that was focused on accomplishing the next mission. For example the training committee has replaced the monitoring committee. This helps to focus the functions of the committee and avoids redundancy. The Programme committee will replace the training committee to steer implementation Project had the backing of Parliament in the Justice portfolio which mooted the idea of an integrated committee in 1999 Keeping more than 15 writers on the project from start to end A complete innovation A new manual in South Africa where there was no relevant literature on the subject Fast procurement and handling of logistics by the CJSP. Support from the PMSO and availability for program review, program management skills transfer and PMSO visibility throughout the development phase 4. Effectiveness Indicators The effective question is “How well or the extent to which the project contributed to the project purpose/objectives?” In answering the question, is the recognition that the manual was developed and has been achieved as an output. It consists of 5 modules, with the content validated as relevant and adequate and serving the needs of the main stakeholders. Further planning and costing is necessary to realize the accreditation element. Page 60 Evaluation, Impact & Effectiveness Assessment of the CJSP B5 Report Effectiveness will also be addressed at the implementation stage, when the project is in a position to measure the effect and impact of the actual training. Quality control is evident in the measures taken to make sure that the manual is user friendly, uses simple but professional language. Several proof reading exercises focusing on language, level and quality of content dominated the project management process. Validation of the manual has been conducted. The following are some of the quality principles and criteria that governed its development: • Content addressing rural and cultural issues • Participation of all role-players in developing content • Writers drawn from all disciplines to make the manual truly multi disciplinary Performance Measures at a higher level: • Reduction of the incidence of Domestic Violence • Reduction of secondary victimization The project is not collecting information on these indicators yet. Due to the fact that these are ultimate results to which the project is anticipated to contribute, caution, detail and definition need to be applied in selecting indicators that would directly capture the causal link between training and the reduction in the incidence of domestic violence and in secondary victimization. Since service providers would be the ones coming into direct contact with the victims, formalization of implementing organisations as data sources and documentation of attribution variables would be necessary. At the training level quantitative indicators to measure reach, coverage and increase of participant knowledge can be applied. 5. Sustainability Other Key factors for sustainability are: Page Factor Status Institutional Readiness Project management and co-ordination This factor continues to be critical as there is a lot of interdepartmental coordination to do. There are many role-players and coordination is a critical factor for sustainability. Active participation and ownership of the project by the target group are key factors that should inform the coordination process, to ensure buy-in into the protocol Limited The Project managers were trained by CJSP on project management. This continues to be an area of need for personal development. SOCA does not have a technical assistance unit for programme management and this is a risk factor for institutionalization and sustainability of the project. Internal unit support in this area proves necessary Communication and change management Communication remains critical as different stakeholders are involved in the project and are venturing out of Departmental comfort zones to interdepartmental and into Govt-NGO partnerships. Planning the next step, i.e. accreditation also requires effective communication on the budget implications Readiness for operational owners to manage implemented services/infrastructure The NPA has seconded the Project Manager and she is playing a significant management role, overseeing all the project activities. Managing cross-country writers is not an easy job, but considerable control and good planning emerged and contributed to the successful achievement of the immediate project objective – production of the NPA is continuing with the process of planning for the training. However the process of developing the manual still needs to be rounded off with getting it accredited. While there is evidence that the accreditation 61 Evaluation, Impact & Effectiveness Assessment of the CJSP B5 Report Page Factor Status Institutional Readiness manual process is being pursued, joint and open review of the budget implications and planning thereof is necessary. Implementation funds have to be secured. Training plans are being developed. A MOU drafted with SOCA leadership will govern the institutionalization process. NPA is the leading organisation currently. Plans to identify trainers are in process Level of skill and capacity The special skill needed for the manual development was writing. Expert trainers and writers from South Africa and from Canada were engaged. Sufficient The Project manager created a point of centrality for the writing process. She was involved in every step acting as manager and a control point. She also was the chief writer, coordinator, liaison point, scheduling working sessions across the two countries Senior management support Sufficient Organization structure Working progress - a structure that is multi-disciplinary is being formed to take the process further. The programme Committee will be the working structure and a revolving leadership arrangement has been introduced Future funding Critical for continuation and sustainability Limited The fact that there is uncertainty about the sufficiency of the budget for the accreditation, registration and publication of the manual is a risk factor for institutionalization. At the implementation level all role-players are expected to commit funds and resources as lead organisations and as owners of the project Engagement /enthusiasm of stakeholders Working Progress Supportive policy to ensure sustainability of the manual is being finalized. Departments and other role-players are expected to commit to adapt regulations and policies and to commit resources to the continuing application and institutionalization of the manual. NGOs representation on the project has been secured. A board created by the MOU 62 Evaluation, Impact & Effectiveness Assessment of the CJSP B5 Report Page Factor Status Institutional Readiness constituted by Director Generals and Departmental Heads and National Directors for continuation – a sustainability factor. The MOU is a tool for institutionalization 6. Key Findings a. Project Performance Project supported by a well thought through project process plan. b. Impact and Effectiveness Immediate impact and effectiveness is that the manual has been developed and exists. The process still has to be completed with accreditation of the manual. Long term impact and effectiveness will be measured at the level of training and strengthening of capacities. 63 Evaluation, Impact & Effectiveness Assessment of the CJSP C2 Report Project Code: C2 Sub-Programme: Justice College Capacity Building Project: Impact Assessment Project (IAP) 1. Background a. Project Problem Statement As part of a larger DOJCD restructuring process the Justice College carried out an impact study to assess its relevance, impact and effectiveness, efficiency, sustainability and potential, its curricular, training methodologies and technologies as well as their impact measures, with a view of repositioning the Justice College to effectively strengthen the core and support operations and services of the Department of Justice. The College faced a situation in which without a statute, it will not be the institution of choice and will have no right to operate and could be substituted by other institutions that will perform the training business. The problem statement describes the problem in detail, locating it within the broader DOJ restructuring process. The specific problem of possible redundancy of the College is outlined clearly. b. Project Purpose As part of a re-positioning exercise, the Justice College undertook the impact assessment to enable it to re￾position itself for enhanced service delivery and to measure its impact on capacity building and training initiatives so as to sustain improved service delivery. The purpose is regarded as central and compatible with the overall restructuring exercise. The ultimate purpose of improved service delivery by the College is clear within the project purpose and there is compatibility between this purpose and performing an impact assessment whose outputs would inform the purpose of repositioning for improved service delivery. Anticipated Outcomes: Re-positioned Justice College (the IAP informs the repositioning) Improved monitoring of impact and service delivery Improved planning and impact assessment 2. Background a. Objectives • To assess the needs of and build capacity of the Justice College in order to reposition it to effectively serve the developmental needs of the DOJCD, the Judiciary, the NPA and other quasi judicial entities and stakeholders • To develop an assessment tool, to evaluate the impacts and relevance of the Justice College’s offerings b. Key Stakeholders The target group for the impact assessment is the Justice College itself. The major stakeholders are the DOJCD, the Judiciary and the National Prosecuting Authority who will benefit from improved service delivery and training by the College. Ultimately, the public that would benefit from a strengthened criminal justice system and the reward for the justice system role-players is positive image and confidence in it. For the purposes of the impact assessment the key target group is the College itself and the problem statement captures the specificity of the problem for this target group. The long-term benefits are clear. There is a direct benefit for the College and for the broader criminal justice system in the exercise/project. Page 70 Evaluation, Impact & Effectiveness Assessment of the CJSP C2 Report c. Key Project Team Members Sponsor: Adv. C. Van Riet Project manager: Karel Kruger Consultant: Nokusa Consulting 3. Project Performance a. Key Deliverables Deliverables/Milestones Completion Status Comments Action research completed 100% The original plan stated that the project start date as from June 2002 to November 2002. The dates were moved back to Oct. 2003. And in actual fact the project started later than planned in August 2003 and was finished in April 2004. The extensions were managed and were formally agreed upon with CJSP-PMSO Submission of research report to IAP Steering committee 100% The institutional impact assessment was finalised and has been accepted by the IAP Steering Committee and by the Management of the Justice College. Active involvement by the Project Coordinator, rigorous coordination and balancing interests/priorities of key role-players are some of the important manoeuvres that made the project succeed. There is ownership of the impact assessment report within Justice College. Key enabling factors were • Professionalism of the external consultants • A project charter/work plan that regulated tasks and timeframes • Cooperation of the respondents • A project communication strategy that ensured maximum buy-in and involvement • Support from the PMSO • Availability of funding Assessment tool submitted to the Justice college 100% The on-line tool to assess the training and capacity building done by the College was submitted by the Consultants in April 2004. Pending is training of the Justice College and partners in the application of the tool Note: Implementation of the impact assessment is not part of the CJSP. For future implementation, availability of funds remains one of the critical factors. Page 71 Evaluation, Impact & Effectiveness Assessment of the CJSP C2 Report b. Notable Outputs and Successes to Date Output / Success Description The impact assessment is a forward￾looking exercise • The spin-off from the impact assessment is that some of its recommendations will inform the visioning and strategic planning of the College The assessment confirmed the place of the Justice College within a transforming justice system and came up with concrete steps on how the process of repositioning the College can be taken forward • The process has resulted in a higher-level decision being taken on the autonomous status of the College. Changes within have unfortunately affected the level of completion of this decision as the matter has been hauled back into a review process. There is full understanding within all stakeholders that the status quo requires higher-level decision-making 4. Sustainability An impact assessment exercise is carried out to inform issues of repositioning, institutionalisation and sustainability plans. The recommendations that came out of the impact assessment exercise are viewed as having met this expectation. The question that begs an answer is the institutional readiness to implement and thereby institutionalise the recommendations. One of the critical factors for institutionalisation of the outputs from the IAP is availability of funds, to implement the recommendations and put the findings to use. This requires further fundraising efforts on the part of the Justice College. An output of the impact assessment was the development of training measurement tool, which has been submitted to the Justice College by the consultants who developed it. The Project Manager has the responsibility to operationalise the tool by first training the College and other partners on its application. An overarching and directive factor with the sustainability of the outputs of the impact assessment is the political reality that while the Justice College took the initiative to promote the assessment and have been active participants in all the phases of the project, control of the process is presently affected by the pending Ministerial decision on the autonomy and independence of the College. This decision has an overbearing and a directive effect on how the Justice College implements the recommendations of the impact assessment and on sustainability. Broader organisational structure can only be planned within a significant amount of certainty. The College as the project sponsor has no ability or authority to exert control over this decision. 5. Key Findings The impact assessment was a useful exercise that informed and served the interests of the Justice College in terms of knowing and identifying the gaps in its relevance, impact and effectiveness, efficiency, sustainability and potential, its curricular, training methodologies and technologies as well as their impact measures. As a result of the impact assessment the College has at its disposal concrete recommendations and suggestions on how to improve and reposition its capacity building agenda. It is imperative that the College, in order to sustain the outputs of the assessment, prepare a work breakdown structure or plan that would assist them implement the accepted recommendations. This should be done within an environment that realistically tracks and takes into consideration the pending political decisions. Achievement of this balancing act is regarded as an on-going challenge for the project. Page 72 Evaluation, Impact & Effectiveness Assessment of the CJSP C3 Report Project Code: C3 Sub-Programme: Justice College Capacity Building Project: Visioning and Strategic Planning (VSP) 1. Project Definition a. Project Purpose To enable the Justice College to re-position itself for enhanced service delivery. b. Project Objectives To produce a vision and Strategic Plan for the new Justice College of South Africa to actually reposition itself according to the Impact Assessment Report: to move from its present position, structure and operational activities to a position and structure which will enable it to function optimally and therefore implement the findings of the Impact Assessment Project. c. Key Project Team Members Project sponsor and coordinator: Adv. Cecille van Riet 2. Project Performance The project is on hold pending formalisation of a legal instrument, policy formulation and decision-making on the new Budgeted National Justice College of South Africa by Cabinet and approval thereof by Cabinet, evolving from the recommendations of the Impact Assessment Project. Page 73 Evaluation, Impact & Effectiveness Assessment of the CJSP C4 Report Project Code: C4 Sub-Programme: Justice College Capacity Building Project: Magistrates’ Mentorship Program (MMP) 1. Background a. Project Problem Statement Magistrates Courts are at the forefront of society’s interaction with the constitution. With the Magistrates’ Courts hearing about 90% of cases that come before South African Courts, the primary responsibility for ensuring the protection of human rights and the realization of constitutional values lies with these Courts. In the past newly appointed Magistrates were put on probation and their performance was evaluated without the necessary on-the-job training. In instances where magistrates were mentored, untrained mentors who were not interested in accommodating and managing diversity destroyed them. The problem statement clearly stipulates the responsibilities facing the Magistrates and the failure of the system to prepare them for protection of human rights and realization of constitutional values. The statement traces the cause-effect problems that are within the scope of the project. The causes may be beyond the scope of this project, but the documents screened for this assessment do not acknowledge these other variables, creating the impression that mentorship alone will lead to empowered Magistrates. Interviews with the Project Manager revealed awareness on the interdependencies and causal linkages between this project and Re Aga Boswa and the Integrated Case Flow Management projects as related projects (interdependent) projects in the Magistrates capacity building. This theory needs to be captured in the project conceptualisation documents for consistency purposes. b. Project Purpose The project builds the capacity of less experienced Magistrates to be better prepared to fulfil their primary responsibility of ensuring the protection of human rights and the realization of constitutional values within the South African Courts. 2. Project Definition a. Objectives Magistrates who act as mentors to newly appointed magistrates are appointed by Cluster heads, Regional (district level) and Provincial Chief Magistrates. The project exposes mentors to a two and a half days mentorship-training workshop. According to the project manager, the project plan has a total of 3 mentorship workshops, making it important for the Justice College to implement institutionalisation plans that fund the project beyond the planned workshops. The project objectives are “To facilitate the sharing of experience in the magistracy, with a view to develop the skills of the particularly less experienced magistrates.” The project objectives are directly linked to the CJSP objectives of building capacity in order to strengthen the criminal justice system. A cause and effect logic outlined in the problem statement about problematic past experience of magistrates destroyed by untrained mentors, gives a clear indication of how mentorship is important as a skills development mechanism and how this route can become destructive if mentors are not trained. The benefits for the mentees are clear. The benefits for the participant magistrates being trained as mentors could be phrased more explicitly in the project objective statement. b. Key Deliverables • Training workshops to train magistrates as mentors for less experienced Magistrates Page 74 Evaluation, Impact & Effectiveness Assessment of the CJSP C4 Report c. Key Stakeholders The participating mentor magistrates are the target group for the mentoring skills development. Broader job-related capacity building is aimed at the less experienced Magistrates. d. Key Project Team Members Sponsor: Adv. Cecille Van Riet Project manager: Belinda Molamu 3. Project Performance a. Progress against Project Milestones Key Milestones Status* Planned End Date Actual End Date % Complete Comments • Undertake and complete a needs assessment Decided not to undertake a formal needs assessment 2003 N/A N/A A formal needs assessment beyond the expression of need by the Magistrates would have been used as a systematic rational means of informing and determining course content and approach. The trainers themselves, to inform the training materials and get the training to reach its full potential, would have done this. This observation is made in the light of the concern raised by the course participants about the heavy industrial and corporate orientation of the mentor course modules • Compile the mentoring manual and material Complete 100% The training material was developed by the external consultancy responsible for the training and was approved by the Project Manager. The compilation did not include a pilot or validation step • Market and communicate the Project Ongoing Ongoing for life of project This is done through getting information to newly appointed Magistrates through the Magistrates Association Body • Complete the arrangements and logistics Ongoing Ongoing This is done by the Project Manager with CJSP-PMSO assistance and was successfully completed for the first workshop held in August 2004. Funding available for these line items • Conduct the seminar Ongoing The first workshop was held in Irene in August 2004, covering modules on mentoring theory, essential mentoring tasks, effective communication, self insight, understanding the mentee and personal development planning Page 75 Evaluation, Impact & Effectiveness Assessment of the CJSP C4 Report Page 76 Key Milestones Status* Planned End Date Actual End Date % Complete Comments • Evaluate and report on progress Ongoing at monthly CJSP meetings Done by the Project Manager. On site visits, formal group report backs and follow ups would enrich monitoring and assessment of progress b. Project Management, Monitoring and Reporting The Justice College Project Manager who reports to the Justice College Project Sponsor and to the CJSP does Project Management. Reporting is done using the project milestones at the monthly CJSP meetings and through regular internal meetings at the CJSP. Reports compiled by the Project Manager are used for accounting purposes to the CJSP and to the Justice College. The Mentors will submit quarterly Progress reports to the Magistrates Commission. The Project Manager plans to build in policies that would make it possible for the College to access the reports. This is necessary for continued monitoring and responsibility by the College. The project successfully realized its key milestone of holding the first training in August 2004 through the hand on management style applied by the project manager and through the cooperative and collaborative approach of the Justice College project team and management. Post-training monitoring is important to monitor progress being made with the actual capacity building. This is important as the process has to be managed and as lessons learned from practical application can be used to improve the training of mentors. Key project steps like needs assessment and piloting training material were left out either because they were no longer deemed necessary or because they had not been factored in the budget. These steps or elements are important with any innovative training and are essential steps for making any training relevant to the needs of the trainees. For example the needs assessment would have captured the realities of the Magistrate mentor – mentee environment and informed the workshop approach and methodology. A case in point is the evaluative observation made by the participant magistrates that what may constitute standard and accepted practice in mentor-mentee relationships in the business world (power and hiercharchical status) is fundamentally opposed to the requirements of observing mentee independence, enforced by the constitution in the judicial world. This points to the importance of an all-embracing approach, including needs assessments as part of developing and implementing a training activity, to inform methodology and training materials development. The participants made the following recommendations in the focus groups held during the evaluation: • Arrange for participant group report backs • Formalise the mentorship by issuing broad guidelines for the mentorship • Introduce the mentees to the mentorship through an orientation that outlines and aligns expectations of all role-players and that creates a common understanding of “what it is in it for all” from the beginning • Incorporate much more justice systems-oriented theory and try and adjust the setting for the workshops from industrial to practical scenarios within the judiciary c. Progress against Key Performance Measures The project uses the following performance measures Measure Baseline Date Target Actual (2004) Number of mentors trained – participant registers 0 (a pilot of the training programme was undertaken) 40 planned 35 mentors trained (This is a significant change considering the 0 baseline and the innovative nature of the project) Evaluation, Impact & Effectiveness Assessment of the CJSP C4 Report Page 77 Measure Baseline Date Target Actual (2004) Number of workshops held 0 August 2004 3 1 Learning level evaluation of the quality of the training This is assessed through evaluation of the workshop by participants Number of mentees mentored Data not available yet The project could consider applying the following measures to track the extent and quality of the mentorship-based capacity building exercise. Measure Baseline Date Target Actual (2004) Pre-test and post-test instruments to measure increase of mentorship skills at the workshops A qualitative measure Level of satisfaction of the mentees A qualitative measure Index-based survey instrument, limited to few key questions Type of skills gained A qualitative measure Site visits to determine mentorship progress/ face to face interviews with mentees and mentors A qualitative measure, not included as a budget item in the current budget Limited success stories A qualitative measure Note: It is necessary that for the impact of the training to be measured effectively, monitoring mechanisms be put in place to document elements like the level of satisfaction, type of skills gained, case studies by the mentees illustrating their effectiveness and swiftness. Higher Level Measurement There is an indirect inference made between capacitated magistrates and conviction rates and case cycles, ultimately leading to increased confidence in the justice system. The use of these higher-level measures requires documentation of indicator definitions, documentation of assumptions, calculation methodologies and attribution issues. Sorting and documenting these would allow more confident use and reference to the data showing increasing conviction rates and confidence. (The review of the surveys or data sources from which the percentages used by CJSP, was not be undertaken as part of this assessment). The conviction rate is a very complex indicator, easily influenced by other confounding factors such as crime rates and increasing number of appointed magistrates. Use of this indicator requires its adequate contextualisation and sufficient definition. d. Skills Development and Change Management The project is about skills development, equipping the Magistrates with skills to mentor their newly appointed colleagues. Change management is a key factor within this project, given the challenge introduced by the expectation on Magistrates to expand their responsibilities and the challenge it brings to the traditional hierarchical approaches within the judicial system. The project brings to the fore a need for conscious and constant examination of the ethics of power management and affirmation and diligent observation and respect of the independence of the less experienced colleagues. Evaluation, Impact & Effectiveness Assessment of the CJSP C4 Report e. Notable Outputs and Successes to Date Output / Success Description A substantial number of Magistrates have been trained • Magistrates trained included those from remote rural areas where the biggest need is and there was an even rural/urban spread. The group was representative including women in its midst Enthusiasm and high motivation and commitment among the trainees Magistrates have certainly been equipped with new skills and knowledge The training addresses issues of self awareness important for Magistrates who are expected to cast an image of integrity within their communities both as mentors and as arms of the justice system, thereby contributing to the legitimacy of the justice system as an institution Through the Magistrates Commission, the Magistrates are active participants in the planning of the project and decide on policy issues Ownership and participation are strong drivers for institutionalisation CJSP has given technical assistance and support with Project Management and created a conducive environment for project success. The CJSP gives constant feedback and guidance to the project manager These factors are regarded as a contributing factor to the success of the project Funding has made the project possible. The downside is the current total dependency on donor funding f. Current Project Challenges, Dependencies, Risks and Issues Dependency Impact Expected Actions Funding and uncertainty on its availability impacts on the ability to plan ad set clear targets in advance for the project Targets are based on fund availability rather than on need Mentorship would work better if the mentor and mentee were in one Magistrate office The reality of scarcity of resources and budget constraints works against the realization of this ideal Availability of trainee mentors is affected by other priorities This affects the actual numbers of participants that actually show up for the workshops The Project Coordinator does not control the choice of participants Bias in selection cannot be controlled by the Project Manager Guidelines on selection criteria may minimize although not totally eradicate the problem The need to balance tracking progress of the project through measuring indicators like conviction rates and case cycles, with allowing the magistrates to exercise independent intellect in individual cases and allow quality resolution of cases to still have a dominant place in the system, is a challenge Performance measures should constantly reside within the identified problems and should be reviewed for change when the nature of the problem and project objectives changes. This calls for clear objective statements and clear definitions of the indicators and their dimensions The workshop material and theory is viewed as heavily set in an industrial setting and lacking in judicial practicalities This removes and distances the training content from the reality that mentorship of magistrates happens within a context where hierarchical status does not apply, but rather experience-based value applies. This reality introduces different dynamics to the mentor-mentee relationship and calls for much more contextualised training Page 78 Evaluation, Impact & Effectiveness Assessment of the CJSP C4 Report 4. Effectiveness Indicators Effectiveness of the project needs to be viewed against the following extract. The project objective is “To facilitate the sharing of experience in the magistracy, with a view to develop the skills of the particularly less experienced magistrates”. Facilitation is a means towards the skills development and is the core project activity. This is achieved through the training of magistrates as mentors. From the first workshop, the overall perspective is that the workshop has empowered the participants to a great extent, giving them practical skills to be able to mentor less experienced colleagues. The workshop content is relevant and comprehensive. The participant magistrates feel the workshops have made a significant difference in their capabilities, steering them from a position of total unpreparedness to being better prepared, better informed to tackle the job. Effectiveness at one level can be measured by measuring reach; meaning that targets for mentorship are set in advance and performance progress is reviewed by comparing targets to the actual reach of the project. At another level, effectiveness can be measured by measuring comparing the universe of newly appointed magistrates to the actual numbers mentored. This requires baseline information on numbers of newly appointed magistrates per year (project implementation year). 5. Sustainability The mentorship programme currently relies on donor funding. Funding is a critical factor towards institutionalisation of the project by DOJ. Key factors for sustainability are: Factor Status Institutional Readiness Project management and co-ordination A Justice College official, who receives enough support and assistance from the College in her responsibilities, manages the project and this allows her to take total control and direction of project. Coordination continues to remain a key factor in managing the project and ensuring successful attendance and participation by Magistrates. At a higher level coordination involves further understanding and tracking of the political decisions on the future of the Justice College, to be able to factor necessary adjustments to future plans The Project manager should be able to undertake onsite visits to monitor progress of the project post the training as part of project management. This component of project management has budget implications that require broader planning Communication and change management Change management is required to facilitate continuing sustenance of mentorship within the judicial. There is a need to formalise mentorship as part of the day-to-day Magistrate job elements and to incorporate performance in this area as part of the Magistrates key performance areas Communication is strengthened by the ability of the Project Manager to utilize the Magistrates Association as a channel to publicise information and get feedback from the Magistrates on the project Limited Magistrates in general still view the requirement to mentor colleagues as a “nice to do” task rather than an imperative. Job policy adjustments or influence needs to be considered by the Justice College as an integral element of institutionalisation of mentorship. Communication is strong Page 79 Evaluation, Impact & Effectiveness Assessment of the CJSP C4 Report Page 80 Factor Status Institutional Readiness Readiness for operational owners to manage implemented services/infrastructure The mentor magistrates are ready to start the mentorship. However workloads, distances of over 90 kilometres to be travelled for meetings in some instances, are a few hindrances that the project has to acknowledge Limited Level of skill and capacity Training of mentors has been outsourced, meaning that the training skill remains external to the organisation. The advantages of reducing the cost of training by having an internal unit to train mentors need to be weighed against quality and feasibility variables. The pool of trained mentors needs to be expanded to be able to reach and service more major centres and to reach rural settings Limited still (in terms of numbers of trained mentors) Senior management support Justice College supports the project Sufficient Organization structure Mentorship is not recognized as a formal part of Magistrates’ job description Poor. The opportunity to accomplish significant results, would be enhanced by an organizational job structure that formalizes mentorship Future funding Critical for institutionalisation of the project. There is a need to fund the implementation of the mentorship programme Limited The project relies heavily on donor funding and the implementation of the mentorship should be fully functional for the project to be considered as fully effective. Otherwise it is training magistrates for a capacity building strategy that cannot be realized Engagement /enthusiasm of stakeholders The mentees need to be oriented in order to gain their buy-in and full understanding of responsibilities, expectations and dependencies Limited. Currently only the mentors get the orientation Evaluation and Monitoring The Justice College has developed a tool to measure the impact of its capacity building and training. At the time of the evaluation, the College had not started using the tool This is a step in the right direction. The operationalisation of the tool has to be fast-tracked for its benefits to be significant. Limited Senior Management Support The Justice College staff and senior managers give excellent support and create a good working environment Excellent at the Justice College level 6. Key Findings a. Project Performance This is a clearly successful project, with the first workshop successfully carried out. Evaluation, Impact & Effectiveness Assessment of the CJSP C4 Report b. Impact and Effectiveness The workshops have exceeded expectations in that Magistrates view them as a platform for them to share and exchange experiences and to learn from each other. Immediate impact will be better measured after the magistrates have started the mentorship. It would be important to track and capture the experiences of the mentored magistrates in the course of doing their jobs and those of the mentors in their application of the knowledge gained from the training workshop. Suggested measures: 1. Number of mentors implementing models from the workshop, qualified by an analysis of apparent non￾implementation c. Institutionalisation and Sustainability Institutionalisation is an imperative considering that all newly appointed Magistrates must be mentored. A strong sustainability factor is that the project is creating an available pool of mentors that can be utilised continuously by the Justice College. This creates sustainability at the capacity building programme level. Page 81 Evaluation, Impact & Effectiveness Assessment of the CJSP D1 Report Project Code: D1 Sub-Programme: Outsourcing the Management of Monies in Trust Project: Management of Monies in Trust (MMIT) Page 82 1. Background a. Project Problem Statement This project is a response to reports revealing that there is massive corruption regarding the management of government Trust Funds. b. Project Purpose “To produce via the PPP insourced and outsourced processes options and timely, solutions based on Service Level Agreements with contractors, solution providers and stakeholders to accept, effectively manage and release monies in trust”. The stakeholders are both internal and external. Internal stakeholders are the finance unit of DOJCD, the Ministry, Judiciary and NPA. External Stakeholders include affected government departments and the Treasury as the facilitators of the PPP process. Buy-in processes are being conducted simultaneously with the feasibility study. This has led to insufficient attention being given to buy-in from stakeholders. c. Project Objectives “To outline and implement rational processes based on agreed PPP principles as well as accepted Service Level Agreements, between the Department of Justice and Constitutional Development, stakeholders and prospective contractors.” The ultimate beneficiaries are users of the end product of this process (e.g. women collecting maintenance payments via bank ATMs). The end product will be rendered by the institution that will be contracted in line with the PPP Project Life Cycle (e.g. financial institution). The direct benefits for the end-users are clearly indicated in that trust monies will no longer “disappear” and access will be enhanced with a downstream indirect impact on poverty alleviation within communities. 2. Project Performance a. Progress against Project Milestones Anticipated Milestones Status Comments • Produce a Feasibility study 90% complete Transaction Advisor has submitted the Feasibility Study Report - Awaiting approval by the Treasury • Complete and present Option Analyses Report 90% complete Transaction Advisor has submitted the Feasibility Study Report - Awaiting approval by the Treasury • Secure Treasury Authorization Level I (feasibility) 0% complete This phase is depended on the approval of the feasibility study report • Secure Treasury Authorization Level II (RFP) 0% complete Pending RFPs • Secure Treasury Authorization Level III 0% complete Pending RFPs Evaluation, Impact & Effectiveness Assessment of the CJSP D1 Report The Feasibility study has been completed and the Project Team is awaiting approval of the Options Analyses Report before it can secure Treasury Authorization Level I and proceed to the next stage. The project is 6 months behind schedule as the Project Team is still addressing issues of the first quarter of the financial year. b. Project Management, Monitoring and Reporting The project prepares complex reports monthly for the project which are mostly used to communicate progress to stakeholders and as marketing material for potential private partners (e.g. retail banks, business community) to keep their appetite alive. The PPP project lifecycle also requires MMIT to adopt a very formal reporting process for Treasury (a key stakeholder for this project). c. Current Project Challenges, Dependencies, Risks and Issues Challenge/Risks/Issues Action being Taken There is a change in leadership, that is, the new Minister and her Deputy were recently appointed The Minister is reviewing all CJSP projects including MMIT This project requires very high-level involvement and commitment Issues affecting progress have been raised within CJSP Buy-in from all stakeholders is crucial The Project Team and the Transaction Advisor are addressing issues of interdependencies among stakeholders 3. Sustainability Discussions with the project sponsor and manager suggested the key requirements for sustainability to be MMIT budget allocation from Treasury, the need for people to begin implementation of the PPP, the provision of training/capacity building (given the badly underfunded department) and finally increased efficiency via the institutionalisation of modernised systems. The key sustainability factor is summarised in the table below: Factor Status Institutional Readiness Readiness for operational owners to manage implemented services/infrastructure PPP Project Life Cycle is a mandatory mechanism that already addresses issues of sustainability e.g. to obtain Treasury Approval I, the feasibility study should demonstrate affordability and value for money Issues of skill, capacity and infrastructure are what this project is addressing through a PPP 4. Key Findings The feasibility study is complete, although behind schedule, and is awaiting formal approval of the Options Analyses Report before Treasury Authorisation Level I can be secured. The project is equipped with quality project management skills and high levels of commitment are exhibited by the project team. The project depends heavily on stakeholder buy-in and thus the project sponsor must ensure that buy-in processes are ensured on an ongoing basis. Page 83 Evaluation, Impact & Effectiveness Assessment of the CJSP E1 Report Project Code: E1 Sub-Programme: Transforming the Judiciary Project: Judicial Officers Association of SA’s International Association of Judges (JOASA’s IAJ) Conference 1. Background JOASA, which represents 1113 magistrates from all ranks in South Africa, was in September 1990 accepted as a full member of the IAJ, which represents magistrates and judges globally. The IAJ aims to: • Safeguard the independence of the judiciary • Increase and perfect the knowledge and understanding of judges by interaction with colleagues from other countries • Study judicial problems of a regional, national and/or international interest • Improve the knowledge and co-operation of judiciaries across borders The African Region of the IAJ was established in 1993 and meets annually to: • Bring a better understanding of the judicial world to African institutions • Ensure that the law becomes a living reality for all African magistrates and judges • Improve and facilitate co-operation between judicial officers and democratic institutions • Support efforts for the creation of common democratic African legal space within the NEPAD framework a. Conference Problem Statement The annual IAJ conference was to be held in South Africa during June 2003. Given the needs of both SA and other African countries, the theme chosen was “How to reduce judicial backlogs in your countries”. The relevance of the conference to SA is that the SA Judiciary, faced with numerous backlogs, needs to explore ways of streamlining and transforming itself. In addition, SA has experienced a number of problems relating to human rights abuses, most noticeably with respect to female genital mutilation. The Judiciary constitutionally needs to explore ways of how it can contribute to safeguarding the culture of human rights. b. Conference Purpose • To explore and discuss Judicial backlogs and their potential solutions • To discuss ways of transforming the Judiciary and safeguarding the culture of human rights 2. Activity Definition a. Objectives • To explore reasons for judicial backlogs across the African region and recommend potential approaches for overcoming the backlogs • To discuss the prevalence of female genital mutilation and the role that judicial officers should play in eradicating the crime b. Key Deliverables • Documented solutions to Judicial backlogs • Resolutions for transforming the Judiciary • Written papers which have yet to be published Page 84 Evaluation, Impact & Effectiveness Assessment of the CJSP E1 Report c. Key Stakeholders • 97 Judicial officers from across the African Region of the IAJ were invited, with attendance by 70 from: – South Africa – Morocco – Tunisia – Egypt – Senegal – Guinea – Ivory Coast – Nigeria – Togo – Cameroon – Mali – Tanzania – Zimbabwe – Kenya • The Department of Justice • Business Against Crime • USAID d. Key Project Team Members Project coordinator: Connie Molwantwa Steering Committee Chairperson: Cagney Musi Committee member: Mr Maumela 3. Project Performance The CJSP advanced funds for this conference on the basis of an expressed need and thus the project should be viewed as an activity rather than a project from CJSP’s perspective. The two primary milestones were to plan and hold the Conference. All these were completed and the Regional African Group of the IAJ Conference was successfully held on the 13th to 16th June 2003 at the Birchwood Executive Hotel, Johannesburg. The following outputs were generated from the conference: Output / Success Description Resolutions documented as conference outputs • Set performance standards for processing of court cases including timelines • Provide sanctions in cases of non compliance with deadlines • Implement alternative dispute resolution mechanisms • Mobilise more resources for courts and ensure optimal utilization • Provide adequate statutory powers and effective measures to officers • Appoint sufficient officers and provide appropriate training • Develop protocols/practice manuals/notices for structured cooperation between court case processing role players • Officers to play a pivotal role in case management to promote good administration Publishable papers • Three papers dealing with gender mutilation, transformation and case backlogs Page 85 Evaluation, Impact & Effectiveness Assessment of the CJSP E2 Report Project Code: E2 Sub-Programme: Transforming the Judiciary Project: SA Judges Symposium (SAJS) 1. Background a. Symposium Problem Statement South African Judges and Magistrates have not convened for a period of approximately 70 years. Given that it is nine years since constitutional democracy in SA, significant changes have taken place within the SA judiciary (“a legal watershed”). It was thus felt urgent and necessary for a “meeting of judicial minds”. b. Symposium Purpose • To review the functioning of the judiciary over the past nine years • To plan the way forward for the judiciary with a view to improving the administration and delivery of justice 2. Project Definition a. Objectives • To hold a symposium to review the functioning of the judiciary and to outline a plan for the way forward b. Key Deliverables • Symposium statement and four publishable working papers • Resolutions for transforming the Judiciary • Moving display and video tapes of three constitutional cases • Opening of the Constitutional Court c. Key Stakeholders • The SA Judiciary – 350 South African Judicial Officers (i.e. Judges and Magistrates) as well as 34 Lower Judicial Officers from JOASA (4), ARMSA (9), Regional Court Presidents (8) and Cluster Heads (13) were invited to attend the Symposium proceedings – 174 judicial officers actually participated in the proceedings • External stakeholders – 233 delegates were invited for the opening session including the SA President, Deputy Minister of Justice, Ambassadors of USA, Netherlands, Sweden, Finland, Germany and Canada – 172 delegates were invited to the Symposium Banquet • The Department of Justice • Business Against Crime • USAID d. Key Project Team Members Project coordinator: Deputy Chief Justice Langa 3. Performance a. Progress against Milestones The SA Judges Symposium was held on the 16th to 18th July 2003 at the Birchwood Executive Hotel, Johannesburg. Page 86 Evaluation, Impact & Effectiveness Assessment of the CJSP E2 Report Key milestones consisted of planning the Symposium, holding the proceedings and preparing/publishing the Symposium output statement and papers (i.e. the deliverables). No key issues were identified in the execution of the initiative and there was a high level of senior stakeholder buy-in and commitment to the project. Symposium delegates rated the success of the Symposium as being above average, with good overall organization, relevant and practically applicable content and good quality speakers. Small problems were experienced with transportation arrangements and a few individuals felt that some key issues were not dealt with or even side stepped, as the structure of the discussions did not always lend itself to people being forthright enough. The opportunity to network and exchange ideas was nevertheless seen as long overdue and highly valuable. In particular, the comparisons between “old” and “new school” judges were seen as advantageous to all. b. Project Management, Monitoring and Reporting Reporting was accomplished via the delivery of the Symposium statement and working papers. No information was made available on the quality of project management and monitoring for the project. c. Progress against Key Performance Measures The key performance indicator for this initiative is the successful achievement of milestones and the translation of the symposium expressed needs and outputs into twelve tangible projects that collectively contribute to the transformation of the Judiciary. d. Skills Development and Change Management The Symposium increased general levels of awareness and understanding of the issues within the judiciary and some of the proposed solutions. e. Notable Outputs and Successes to Date The most notable outputs are the key deliverables and in particular the tangible symposium resolutions. The Chief Justice and Heads of Court were mandated to implement the meeting resolutions. The additional twelve projects within the sub-program “Transforming the Judiciary” emerged as a direct result of Symposium outputs. The table below outlines the key outputs related to the deliverables. Deliverable Tangible Outputs / Resolutions Four Publishable Symposium Papers -Papers were not made available during the evaluation but press articles entitled “Tension as Top Judges Discuss their Future” and “Courts have to Serve Society, Top Judge Says” were provided Symposium Statement: Vision of the Constitution Role and function of the courts Accessibility of justice Representivity of the Judiciary Independence of the Judiciary -Courts now required to declare invalid, legislation or conduct inconsistent with the Constitution -Decide cases impartially and according to law -Ongoing education of judges -Adoption of a code of conduct for judges -Administer justice to all -Commitment to improve accessibility, user friendliness and efficiency of courts by identifying causes of failures and partnering with arms of state to address them -Urge government to identify and correct the legal aid system in collaboration with the judiciary -Judiciary now 34% black (vs. 2% in 1994); magistracy 50% -Now 25 females judges versus only 2 in 1994 -More to be done to increase racial and gender representation -Importance of independent judiciary in SA constitutional order -Interdependence of the executive, judiciary and legislature (all court Page 87 Evaluation, Impact & Effectiveness Assessment of the CJSP E2 Report Page 88 Deliverable Tangible Outputs / Resolutions Single Judiciary Public scrutiny Working conditions Administrative autonomy of the Judiciary orders to be enforced by all arms of the state) -All courts must be independent (enhanced protection needed for lower order courts) -Accessibility to public scrutiny is important; thus accurate communication to broader society is required via facilitation of media work (e.g. providing summaries of case proceedings and judgements) -Basic facilities are inadequate for many judges and magistrates; needs addressing urgently -Judiciary needs effective say in its own budget and expenditure control Moving Display and Video Tapes of 3 Constitutional Cases Assistance was provided by the CJSP-PMSO to Orchre Communications in developing the moving display and video tapes to be installed at the SACC museum -USAID provided an additional $30,000 for this purpose f. Current Challenges, Dependencies, Risks and Issues No key issues were identified in the execution of the symposium planning. Key dependencies for this initiative are essentially the twelve other projects within the sub-program as they were developed as a direct result of the symposium resolutions. 4. Effectiveness Indicators Effectiveness will be measured by the extent to which the resolutions made at the Symposium are realized through the successful implementation of the projects within the sub-program. The overall impact realised should be a transformed judiciary. 5. Sustainability The institutionalization of evolving issues is expected to take place via the implementation of the twelve downstream projects. Symposium delegates feel that little evidence exists to date to support that institutionalization is actually beginning to take place and that a more formal structure to monitor the output/results is needed. Some suggested an annual event to report back on decisions taken and results achieved would be worthwhile, whilst others alluded to another form of formalized structure to enable judges and magistrates to articulate problems and issues. Delegates also felt that more engagement between Judges and Heads of Court was needed to ensure effective issue resolution in future. The findings suggest that an ongoing communication strategy is needed to keep all stakeholders informed of the progress on the relevant CJSP sub-program projects. 6. Key Findings Overall, this initiative was successful in the achievement of its proposed objectives of holding a Symposium and developing a plan for the way forward for the SA judiciary. There was a high degree of stakeholder support and buy-in and a fairly high level of attendance by key judicial officers and external invited delegates. In particular the quality of administrative organization, relevancy of content and quality of speakers were highlighted as positive project performance indicators. Evaluation, Impact & Effectiveness Assessment of the CJSP E3 Report Project Code: E3 Sub-Programme: Transforming the Judiciary Project: Leadership and Management Development Program for Female Judicial Officers (LMDPFJO) 1. Background • The IAWJ was established in the USA in 1991, but the association soon identified the need for extending its membership internationally as women judges experience similar challenges around the world, independent of their local environment. • South Africa officially joined the IAWJ in August this year. The SA chapter was officially inaugurated on 8 August – timed to coincide with Women’s day (work began in November last year on establishing the branch, but members felt it important to leverage sentiment around Women’s month and 10 years of democracy). • The most important objective for the SA branch of the IAWJ is the development of women leadership and management for judicial officers in South Africa. a. Project Problem Statement There is a lack of women in leadership roles in the judiciary. This is perceived to be a function of: • The legacy of the previous dispensation: – There was only one female judge before 1994 – Today there are 28 women judges (out of a total of 210) – As a result, most are young or less experienced – In theory, the judiciary would like to value a leader on potential (not just experience). This is difficult to achieve in practice, however • Senior / experienced judges have far more influence on the judiciary (“make their presence felt”) and some show a reluctance to learn / adapt to new ways • “Junior” judges lack the confidence and comfort to take a leadership role b. Project Purpose To bring together a number of SA female judicial officers to: • Facilitate and encourage those with interest and a potential to join the judiciary leadership and management echelons • Exchange information and research on issues of vital concern to women judicial officers 2. Project Definition a. Objectives The CJSP project restricts its scope to the execution of the first IAWJ conference. Project team members however see the conference as simply the starting point and that the scope is actually much broader: • Development and implementation of initiatives to increase women leadership in the judiciary – Motivation of women – Management skills (including how to discipline) – Interpersonal behaviour • Development and implementation of initiatives that address women’s issues in the justice process – Awareness amongst the judiciary of gender issues amongst the justice system’s beneficiaries – Community outreach programs to adapt judicial practices to community gender needs – Female members of the judiciary have an important role to play in achieving the department’s goal of gender equality through changing attitudes of justice officials and the judiciary Page 89 Evaluation, Impact & Effectiveness Assessment of the CJSP E3 Report • In essence the project serves to kick-start the relationship between the high court judges and lower court bench in order to: – Facilitate knowledge sharing – Develop judicial leadership and development – Instill overall confidence in justice – Address transformation issues – Form the SA Chapter of the IAWJ and increase membership of this international body – Move forward as a single judiciary in SA b. Key Deliverables • Execution of the first IAWJ Conference • Report on IAWJ Conference findings • Establishment of the SA office of the IAWJ • Increased membership of the SA Chapter of the IAWJ (from 15 to a target of 45) c. Key Stakeholders • All women within the judiciary - beneficiaries • Department of Justice - Project Sponsor • Business Against Crime • USAID d. Key Project Team Members Project Coordinator: Judge Mokgoro Steering Committee Chairperson and IAWJ SA President: Judge Lucy Mailula Project Leader/Facilitator and IAWJ SA Secretary: Belinda Molamu Steering Committee Member and IAWJ SA Vice President Programs: Judge Theron Steering Committee Member: Valerie Qgiba Steering Committee Member: Judge O‘Regan IAWJ SA Deputy President: Ms J Wessels IAWJ SA Treasurer: Judge J Traverso IAWJ SA Vice President Publications: Ms S Naidoo 3. Project Performance a. Progress against Project Milestones The key project milestones included: • Appointment of working committee • Identification of Seminar content and schedule • Planning and execution of logistics • Identification and booking of speakers • Notice to participants • Conducting of seminar • Reporting on progress The following progress was made towards achieving these milestones: • The Conference was held • Project team members rated the participation by delegates as excellent: all provinces were well represented; women’s interest groups were enthusiastic; the content was relevant to the objectives (topics assigned by steering committee) and speakers were highly proficient (both judges and NGOs). • Delegates rated the Conference as very good to excellent in terms of administrative and logistical arrangements as well as the quality of presenters. Page 90 Evaluation, Impact & Effectiveness Assessment of the CJSP E3 Report – Small glitches were experienced with a power failure and some accommodation issues but they did not impact the delegates’ overall success rating. – Most delegates felt it was a valuable platform for interaction between the judicial officers, contributing towards the eradication of a silo mentality within the judiciary. – Magistrates in particular felt it gave them hope and encouragement for their development in the future. – Individuals felt there would be merit in hosting such an event annually. • A Working Group on Women in the Judiciary has been established (the first meeting was held on 20 August consisting of executive committee and provincial representatives). Standing committees are to be set up to deal with specific issues identified • 55 Additional people joined the SA Chapter of the IAWJ, the SA office was established at the Justice College and a website layout is now in place. Outputs remaining post the Conference include: • A report on findings to be transcribed from the recorded proceedings and taken to the Vice president of Publications and Programs who will review issues raised and determine priority for training programs going forward • A debriefing meeting is still to be held • A newsletter / publication is to be launched • An investigation into existing international leadership development programs for women judges (to leverage best practice, e.g. by exposing judicial officers to international conventions and encouraging them to practically apply what they learn in the local courts) or alternatively, tailoring local programs targeted at business to meet the needs of women judges (e.g. training program on jurisprudence of equity to be conducted before end November 2004). • Prioritisation of potential initiatives to ensure appropriate use of available funds Key enablers for achieving the project objectives included: • Establishing an office with dedicated support staff who are focused on the vision and willing to work hard to achieve it • Provision of valuable ongoing advice, consultation, technical and administrative support by the PMSO. The PMSO assisted in keeping the team focused and provided constructive report feedback and follow up on key issues • Establishing specialised research capabilities to ensure initiatives are correctly targeted • Committed project team members • Committed external stakeholders in terms of high level support (rather than detailed direct involvement) for the LMDPFJO – Has the support of the Presidency and the SA Chapter of the IAWJ has been invited to partake in the government’s women dialogue process – The Chief Justice has also endorsed it and the program is vested in the Chief Justice’s office – The Gender Directorate has agreed to provide support in achieving gender transformation – Working closely with Chapter 9 institutions (e.g. the Gender Commission) b. Project Management, Monitoring and Reporting Monitoring • The working group needs to establish a more formal methodology for monitoring / self evaluation – The current project used Steering Committee meetings and a Workplan with identified deadlines and targets to evaluate progress. Monthly meetings were used to monitor project progress and identify achievements and challenges – Key targets were to recruit as many new members to the IAWJ as possible, to get accepted invitations to the conference by as many of the most relevant women judicial officers as possible, and to ensure that conference content and speakers are in line with project objectives • Performance measures are a key item on the agenda of the next working group meeting Page 91 Evaluation, Impact & Effectiveness Assessment of the CJSP E3 Report – It must be noted that the program itself does not make decisions to appoint women to leadership roles but acts in a supporting role in terms of writing motivation/endorsement letters and selecting women with potential to attend conferences/development programs Meetings Key meetings held during the course of the project include: Description Attendees Frequency Working Committee meetings to plan and schedule the program, set conference date and plan program and logistical arrangements Led by Constitutional Judge Mokgoro Twice c. Progress against Key Performance Measures Key performance indicators were essentially the completion of milestones within set timeframes and an increase in the number of women joining the SA Chapter of the IAWJ. d. Skills Development and Change Management Skills development was not a key objective of the conference. However, in the longer-term the IAWJ has as one of its goals the desire to increase the development of women in the judiciary. This will be achieved through the mentoring and coaching of women and providing ongoing conference training. In addition, the IAWJ will endorse the promotion and development of worthy women with potential. e. Notable Outputs and Successes to Date Output / Success Description Increased membership of the IAWJ within SA 55 additional members of the SA Chapter of the IAWJ as a direct result of the conference Improved communication, awareness and understanding Sharing of issues and views on women development and transformation assisted in improving relations between female judicial officers, network building and gaining a more unified perspective Working Group established for ongoing sustainability Individual standing committees to be established to explore ongoing women’s issues: • Transformation of the judiciary in terms of increased promotion and development of women and changing mindsets of men towards women in the judiciary • Exploring training and development needs of women (e.g. leadership development programs) f. Current Project Challenges, Dependencies, Risks and Issues Funding was a major issue for the project: • The business proposal did not request enough funds • They were not confident that their request would be approved and so were too conservative in their request for funds • Also, costs were underestimated • Have learnt that funds are not a constraint and will thus request more appropriate funding. The project was successful in getting people to begin thinking about key issues and began discussions between lower and higher court judicial officers. The Steering Committee was not as effective as it should have been • Most members lacked conference planning and coordination skills, with the result that a limited few were left to execute the required tasks to meet project milestones and deliverables (felt overwhelmed by what needed to be done). Page 92 Evaluation, Impact & Effectiveness Assessment of the CJSP E3 Report • It was felt that the Steering Committee members should ideally all be located centrally for optimal working relations. • The most support is leveraged from the individual that puts the business plan together as he/she has the highest level of commitment to ensuring objectives are met; the project leader thus had significant preparation work to handle alone prior to meetings. • Steering Committee members were often uninformed as to the requirements of the funders – CJSP support in this regard was critical and highly valued. 4. Effectiveness Indicators No explicit effectiveness indicator was communicated for the project but the long term effectiveness baseline should be the change in number/percentage of female judicial officers. The baseline number at project inception was 26 female judges out of 204 (13 are black) and 428 female magistrates out of 1662. 5. Sustainability and Institutionalisation Key factors identified by the project team to ensure project sustainability include: • Finding the right methods for change management (to impact behaviours of the judiciary when it comes to gender issues & talent evaluation) – Some of the male judges feel the program is not timely as there are still racial issues that need addressing prior to gender issues – Still an element of distrust prevalent between the magistrates and judges • Correct structure to support institutionalisation – Provincial representatives on executive council are there to facilitate the setting up of regional councils for practical implementation and support of provincial initiatives • Collaboration with other institutions to provide timely and relevant training programs (Justice College; JCE) • Commitment of resources on an ongoing basis to support IAWJ initiatives (e.g. research & publications) • Finding the right methods to gain senior judiciary support for developing women leadership & making the judiciary responsible for talent as a community (beyond direct reporting lines) • Development of programs to continue the transformation and development process and maintain the focus, including training/mentoring and support programs • Future funding is to be covered by membership fees (R300 registration plus R600 per annum per member) plus the opportunity to get more funding for specific projects as and when identified Key factors identified by conference delegates include the following: • Project sustainability will be enhanced by ongoing demonstrated support by the Minister, State President and Chief Justice. • Although there is still a long road ahead before transformation is realised, women are beginning to feel that this issue is beginning to get the attention it deserves. • True sustainability will be measured by how many more women are considered, interviewed and hired into relevant senior positions within the judicial hierarchy. 6. Key Findings a. Project Performance This project was operationally highly successful in achieving its stated objectives. It was well executed and tangible outputs were delivered. The outstanding project issues and learnings include: • Key measures must be adopted by the Working Group and its underlying Standing Committees to ensure ongoing effective performance management. These measures should link directly to key goals, e.g. number of training and development programs to be launched/hosted; number/percentage of women judicial officers to be trained/endorsed/mentored, etc., number of communication initiatives to be Page 93 Evaluation, Impact & Effectiveness Assessment of the CJSP E3 Report undertaken amongst members and the judiciary as a whole; satisfaction survey scores amongst existing members • The conference report is still outstanding which must be completed as a matter of urgency as it will highlight the ongoing training and development requirements for prioritisation • A newsletter publication is still to be launched which will form an important communication link between the work of the IAWJ and the rest of the judicial officers • Leveraging international and local best practice and ensuring an integrated approach amongst various relevant stakeholders is important to prevent wasted effort and ensure coordinated focus on key objectives b. Impact and Effectiveness • Effectiveness measures relevant to ongoing initiatives in this area should be the number of women successfully hired into senior positions within the judiciary. Although the IAWJ does not directly control this, indirectly it must contribute to this effectiveness measure either through training, development and mentoring or through change management initiatives to foster a cultural mindset throughout the judiciary. To date one woman magistrate and member of the SA Chapter of the IAWJ has been appointed as a judge. c. Institutionalisation and Sustainability Sustainability is to be achieved via the SA office of the IAWJ and its paid up members. The Working Group is the key mechanism by which Conference outcomes will be sustained and institutionalized at the regional/provincial level. A few suggestions for continued success in this regard include: • The Working Group should as one of its key roles develop a set of guidelines for operations of the decentralized standing committees to ensure that a common integrated approach is adopted. • The Working Group should develop an integrated view of its proposed journey for achieving its aims in the areas of – Change management amongst both men and women within the judiciary and magistery – Communication to all relevant stakeholders on progress achieved and future areas of focus – Degree of interaction and collaboration with other relevant and interested parties (e.g. Gender commission) so that each may leverage off the others sources of information, people and funds. Page 94 Evaluation, Impact & Effectiveness Assessment of the CJSP E4 & E5 Report Project Code: E4, E5 Sub-Programme: Transforming the Judiciary Projects: Towards a Unified South African Judiciary (USAJ) Single Rule Making Authority (SRMA) 1. Background a. Project Problem Statement Currently the judiciary is seen in two bodies, namely the High Judiciary and Lower Judiciary. The Constitution requires that the judiciary functions as one body. Two projects within the Transforming the Judiciary Sub-program, “Towards a Single Unified SA Judiciary” (USAJ) and “Single Rule Making Authority” (SRMA), seek to transform the judiciary into a single body. The existing bodies have each their own structures which guide them (e.g. the Judicial Services Commission and the Magistrate Commission). Fully operating rules, principles and proposals towards a single unified judiciary are expected to assist in this regard. b. Project Purpose • To enable unified rule making in the judiciary • To define a common set of rules to govern a single and unified judiciary • Source and engage proposals towards a unified judiciary 2. Project Definition a. Key Deliverables • Establish a single rule making authority for the judiciary • Review of existing rules and formulation of unified rules for the judiciary • Call for proposals and establish a mechanism to engage and interrogate such towards a single judiciary in SA • Establish a unified judiciary to represent judicial officers in SA b. Key Stakeholders • Judiciary – Higher Court Judiciary – Lower Court Magistrates c. Key Project Team Members Projects’ co-ordinator/facilitator: Dr Biki Minyuku (CJSP Program Director) Project co-ordinator (SRMA): Judge Ngcobo (Judiciary) Project co-ordinator (USAJ): Judge Ngoepe (Chairman of Magistrate’s Commission) 3. Project Performance a. Progress against Project Milestones The projects have essentially not started and are thus behind schedule. The table below summarises the key milestones and performance status: Page 95 Evaluation, Impact & Effectiveness Assessment of the CJSP E4 & E5 Report Project Key Milestones Status Expected End Date Comments Establish committee to review rules Completed J. Ngcobo and J. Ngoepe will be collaborating to ensure alignment between projects Amend both Civil and Criminal Procedures Completed Certain procedures prevented Judge Ngcobo from fast-tracking the review process timeously Appoint 6 researchers to review & analyze rules In progress Await decision from DOJCD to provide resources to appoint researchers Review, analyze and transform rules Initiated 2005 Set up an inclusive committee Hold brainstorm session SRMA Implement Single Rule Making Authority and related rules - 2005 Request and gather proposals from relevant stakeholders on mechanisms to achieve judicial unification - SRMA and LMDPFJO are examples of how successfully judges and magistrates can work and leverage joint insights for these project teams USAJ Establish committee as single representative body of both judges and magistrates - Suggested process of completing SRMA first in order to use the rules as a basis for a unified judiciary to operate as a single body (i.e. rules to serve as governing principles for engagement) b. Project Management, Monitoring and Reporting Dr Minyuku and Cecille van Riet are acting as a link between the overall sub-program to transform the judiciary, overseeing the interdependencies of the initiatives in that sub-program and the executive. No formal project reporting or monitoring process is currently in place. This will be determined once the researchers are appointed and the projects commence work on their most significant deliverables. c. Progress against Key Performance Measures Performance measures are not relevant at project level (program level more appropriate). 4. Effectiveness Indicators Effectiveness measures are not relevant at project level (program level more appropriate). 5. Sustainability Credibility and respect are important enablers for success in the USAJ project. To achieve this, a change in mind-set on behalf of both parties (judges and magistrates) is needed to work together towards a common purpose, as required by the constitution. In addition credibility is hampered by the fact that some magistrates are not adjudicating cases and as a result aren’t actually performing the function of judicial officers. This factor can be addressed through the provision of training in civil matter adjudication. Page 96 Evaluation, Impact & Effectiveness Assessment of the CJSP E6, E7 & E8 Report Project Code: E6, E7, E8 Sub-Programme: Transforming the Judiciary Projects: Interim Advisory Council/Board (IAC/B) Restructuring and Transformation Committee (RTC) Judicial Education and Training Committee (JETC) 1. Background a. Project Problem Statement • There is a constitutional imperative for judicial independence; • There is a need to provide training and development for judicial officers; • A structure is required whereat training and development can be coordinated whilst ensuring independence The JETC and RTC were absorbed into the IAC, which now functions as a single body with broadened legal, judicial, magisterial and prosecutorial representation. b. Project Purpose • Establish an interim board to make recommendations on where judicial education and training should take place in light of the constitutional imperative for independence. More specifically it aims to: – Develop a legal instrument for the establishment of an entity – Restructure the current organisation into three faculties with an independent forum /committee – Conduct strategic planning for the new College – Define ongoing training needs for judicial officers 2. Project Definition a. Objectives • Practice effective leadership • Establish respective Authority, Council and Committees • Institute efficient Judicial structures and processes • Assign Judges as Conveners and/or Project Champions • Appoint relevant authority or committee • Help define Protocols/Appropriate Terms of Reference • Prioritise issues and convene regular meetings • Engage in consultations (where necessary) with key stakeholders • Provide the necessary administrative and logistical support • Submit monthly progress reports and quarterly evaluations b. Key Deliverables The key project deliverables for the broad committee representing judges, magistrates, prosecutors and independent legal advisors are outlined in the table below: Page 97 Deliverable Contribution to Project Objectives Status Draft legal instrument Establish Justice College as independent body Pending approval by Minister by end August Implemented legislation • Restructured College into 3 faculties Establish independent council and faculty boards Conduct strategic planning On hold pending legislation Evaluation, Impact & Effectiveness Assessment of the CJSP E6, E7 & E8 Report Page 98 Deliverable Contribution to Project Objectives Status • Strategic plan Recommendations on restructuring required Provide list of intended transformation projects Business plan for the College On hold pending legislation Coordinating committee Curriculum for existing judges Curriculum for new judges Curriculum for aspirant judges Delivery of current year’s training programmes for all clients c. Key Stakeholders • Justice College • Department of Justice • The Judiciary • The National Prosecutions Authority • Business Against Crime • USAID d. Key Project Team Members Project sponsor: Deputy Chief Justice P Lange Project coordinator: Judge Johann Kriegler Committee member: Advocate Cecille van Riet Committee member: Dr Biki Minyuku 3. Project Performance a. Progress against Project Milestones The outputs of the impact assessment project, the Canadian program, independent consultant (Nokusa) findings, training study for high court judges performed by the JETC and the IAC’s research all recommend the same conclusion – that the Justice College be established as an independent statutory body with a separate council/board consisting of three faculties (faculty prosecutorial training, judicial training and general training) each with its own board to ensure independence. Three separate colleges would have been costly and unnecessary from an infrastructural (current infrastructure including library and language labs are adequate to support all needs) and HR/admin perspective. The IAC project experienced some key changes to the original scope as a result of the change in Minister and Deputy Minister. The new Deputy Minister did not initially agree with the project recommendations. As a result additional time was expended in briefing him of the work done to date to support the validity of the recommended decision. The project is now awaiting legislative approval to proceed to the strategic planning stage. This was expected to take place more than a month back but to date no decision has been forthcoming. Key project phases for the IAC are outlined in the table below. Evaluation, Impact & Effectiveness Assessment of the CJSP E6, E7 & E8 Report Milestones Delivery Date Actual Delivery Date Status Civil Procedures amended • Draft legal instrument • Draft press release • Approve and deliver press release Due one month back (July 2004) Hasn’t happened (planned meeting with Minister for early September) In progress Delivery at risk Implement new structure • Restructure College into 3 faculties with independent boards • Conduct visioning and strategic planning session Due end September Not begun In progress Delivery at risk The project was too small to warrant a detailed Workplan – it was really only a collection of small meetings with key people. b. Project Management, Monitoring and Reporting Given their tight schedules and judicial independence, Dr Minyuku and Adv Cecille van Riet have been requested to monitor and report progress on the IAC, which was achieved primarily by means of regular communication and small meetings (driven by the Exco members). Two key meetings were held by the IAC. The first managerial meeting looked at the legislation upon which the statutory board is based. This meeting comprised of representative members from the Higher and Lower Judiciary, Prosecution, Law Society of SA, the General Council of the Bar of SA, Association of Law Deans, the CJSP-PMSO, the DOJCD’s CEO and members of the Judicial Coordinating Committee. Judge Johann Kriegler was confirmed as the Interim Judge Dean of the New National Justice College of SA, consisting of 4 Independent Faculty Boards. A second key meeting was held with the Minister and Deputy Minister in order to convince them of the necessity for a legal instrument and cabinet memorandum to be established. c. Progress against Key Performance Measures The IAC members feel that key project measures are only relevant at the level of the impact assessment project. However, measuring progress for this project does entail monitoring the extent to which legislative approval is obtained and the extent to which other project deliverables are met (i.e. the setting up of three faculties and drafting of a strategic plan). On balance the IAC was successfully implemented as a small and cost effective project with senior management buy-in focused on lobbying and addressing an important constitutional imperative. Page 99 d. Skills Development and Change Management The capacity building objective for the JETC project, and more broadly for the Justice College as a whole, is to skill court officials rather than the Department itself. In this context, the Skills Development Act has no jurisdiction over judicial education. Skills gaps do not appear to exist within the project team itself but rather amongst its downstream beneficiaries. Current infrastructure is adequate to support ongoing needs (building, offices, etc) but computer stations will be needed for the judge researcher/secretary and for the purposes of acquiring international education material. In terms of service / infrastructure delivery, the intention is to provide relevant needs-based training to all clients (magistrates, judges, masters of high court (court officials), prosecutors, clerks of the court, interpreters and registrars). These needs are to be identified and planned as an output of the project, which will only be realized post Ministerial approval. Ongoing training will then be funded by the Treasury as part of the budget for running the College. Training for high court judges is budgeted at R2,87m which has not been included in the approved project budget as it was not initially envisaged as a need. Evaluation, Impact & Effectiveness Assessment of the CJSP E6, E7 & E8 Report e. Notable Outputs and Successes to Date The most notable output from the IAC project is that despite the initial negative predisposition of the Deputy Minister towards the project and its intended outcomes, the project team were able to engage with him to the extent that it now appears as though the project objectives will be met (i.e. the legislation is very likely to soon be approved). This was made possible by the intense commitment shown by senior judicial officers within the project team. f. Current Project Challenges, Dependencies, Risks and Issues The primary issue impacting project success is the Deputy Minister delaying the legislation approval process. This delay was perceived to be related to reluctance on the Deputy Minister’s part to diminishing the power of the department. However, the issue was timeously addressed with an appropriate degree of urgency via a process of key people making an initial courtesy visit to update Deputy Minister De Lange on the project progress followed by a meeting to convince him of the findings and recommendations in terms of the constitutional imperative. 4. Effectiveness Indicators As for performance indicators, no explicit effectiveness indicators were established for the project. Setting up the New Justice College of South Africa will be a major achievement for SA with potential impact for the entire SADEC region. It will also ensure properly trained judicial officers who can perform their jobs more effectively (i.e. one can’t have transformation without education). 5. Sustainability In terms of project institutionalization, the College has the personnel required and a suitable Acting Dean Judge has already been identified. The two key factors required for sustainability are legislation and funding, as outlined below: Factor Relevance Capacity to Institutionalize Legislation Project success depends on approval and sign-off by Minister • Ready and able once approval is achieved Funding Need R2,87m to train high court judges • Have identified a retired constitutional court judge as Dean and have facilities and people available but need funds for material and courses • Training needs will be identified by the board with College management assuming ongoing responsibility for institutionalization 6. Key Findings a. Project Performance On balance it is our opinion that the IAC was successfully implemented as a small and cost effective project with senior management buy-in focused on establishing the New National Justice College of South Africa. A key lesson learnt from this project, which is directly applicable to other projects, is to take smaller bites at a time (plan smaller pilots) to determine whether or not the concept works in practice prior to a broader pilot roll-out (will prevent large re-engineering projects which are financially more risky). Page 100 Evaluation, Impact & Effectiveness Assessment of the CJSP E6, E7 & E8 Report b. Impact and Effectiveness • No effectiveness measures were found for the project. A relevant measure for ongoing initiatives in this area is an increased level of trust from all of the College’s clients as a result of its independence. This in practice should be indicated by increasing numbers of people attending and requesting training initiatives from the body. However, that said, effectiveness is best measured for the entire sub-program of judicial transformation rather than at the individual project level as all the projects collectively contribute to the desired outcome. c. Institutionalisation and Sustainability The IAC is awaiting cabinet approval for the New Justice College of SA. Institutionalisation will then be addressed via the operational budget of the College. Page 101 Evaluation, Impact & Effectiveness Assessment of the CJSP E10 Report Project Code: E10 Sub-Programme: Transforming the Judiciary Project: Aspirant Judges Training and Development Program (AJTP) 1. Background a. Project Problem Statement The judiciary is currently under capacitated and is still not representative of the diverse SA society. This project seeks to address two issues concurrently, firstly by increasing the pool from which Judges can be appointed and secondly by training and mentoring legal practitioners who aspire to become Judges. b. Project Purpose To identify, select, train and develop aspirant Judges in South Africa with a view to increasing a Pool from which Judges can be appointed. 2. Project Performance The following table summarizes the Key Milestones for the AJTP project: Anticipated Milestones Status Comments Assign Project Coordinator 100% complete A coordinator is assigned and has managed to implement the project Identify trainers 100% complete Trainers and mentors have been identified Develop the Training and Development Program in stages/phases 100% complete The first training program has been developed. Mentorship is in progress Identify and select participants In progress Continuous process. Three cycles have been completed Invite the participants and conduct training and development in stages / phases In progress Three cycles have been completed Internal Assessment and External Evaluation In progress Internal assessments and an external evaluation for the three cycles have been completed Currently, 24 Aspirant Judges have been trained and 16 of them are appointed as Acting Judges. 3. Effectiveness Indicators Feedback received from Trainees indicates that the project is effective and immediate impact is evident. The following statements are quotes from trainees’ comments as gathered via telephonic surveys: • The training and development of aspirant Judges is through practical mentorship programmes. Participants believe that mentorship is working very well and should be encouraged. • The project provides Trainees with an opportunity to consider their options to become future judges. • The experience is invaluable. • Appreciation of the guidance, practical insight and general advice garnered from assigned Mentors. • Trainees are all, but 2, practicing attorneys who now have an insight into the bench from a different perspective. Page 103 The CJSP is pleased that through this project the Judiciary will have a larger pool of Judges. Additionally, Acting Judges portray confidence and independence in handling cases assigned to them. Evaluation, Impact & Effectiveness Assessment of the CJSP E10 Report 4. Sustainability The key factors relevant to the evaluation of project sustainability are as follows: Factor Status Institutional Readiness Project management and co-ordination The project is still at the first stages of implementation and showing signs of success The Judiciary is ready as an institution to continue with the development of the training programme Level of skill and capacity Expert trainers and writers from South Africa and from Canada were engaged for this purpose Sufficient Existing Judges conduct training and are engaged in the development of the project. Senior management support Trainers and mentors are currently engaged in the implementation of the project Sufficient Future funding Critical for continuation and sustainability Plans are in place to ensure that there is future funding for the project Engagement /enthusiasm of stakeholders Stakeholder enthusiasm is very high among the aspirant Judges who are undergoing the training programme Sufficient Page 104 Evaluation, Impact & Effectiveness Assessment of the CJSP E11 Report Project Code: E11 Sub-Programme: Transforming the Judiciary Project: Training the (Trainer’s) Trainer (TTT) 1. Project Definition a. Project Purpose To identify a CJEFJ fellow to attend the NJE Conference with a view to developing a home-grown Train the Trainer Programme and to help train South African Judges with an ability and interest in training. b. Alignment with Sub-Programme Purpose This project is in line with the Sub-programme’s goal of transforming the South African Judiciary and developing capacity to lead and sustain an effective justice system. Through this project the judiciary ensures that training and development of existing and new judges is readily accessible and meets their needs throughout the transformation process. c. Key Stakeholders The project’s target group is those Judges who are identified as the most competent to train others while transferring skills and knowledge gained throughout their tenure as Judges. 2. Project Performance a. Progress against Project Milestones Anticipated Milestones Status Comments Assign Project Coordinator 100% complete A coordinator has been assigned and is implementing the project Assign Candidate (CJEFI Fellow) to attend the NJF’s Train the Trainer Conference 100% complete Two Judges attended the NJF’s Train the Trainer Conference Accumulate knowledge and expertise In progress Continuous process Compile and submit report on conference 100% complete Report has been submitted Present programme to Judicial Education and Training Committee for adoption and implementation 100% complete Implementation is pending adoption Jointly develop the South African Train the Trainer Programme In progress Train the Trainers Pending Internal assessment and External Evaluation Pending The project is currently in the implementation stage/ phase. The Training of Trainers programme is being developed after Judge Kriegler and Judge Brand attended the CJEFI Fellow Conference in December 2003. Two Judges have been identified to date as being qualified to fulfill the role of Trainers. Page 105 Evaluation, Impact & Effectiveness Assessment of the CJSP E11 Report b. Notable Outputs and Successes to Date The Judiciary has two Trainers who are qualified to train other trainers. 3. Effectiveness Indicators It is too early to measure the effectiveness or impact of the project because the two Trainers to date have not trained any additional trainers. 4. Sustainability Factor Status Institutional Readiness Readiness for operational owners to manage implemented services/infrastructure Two Judges have been trained as Trainers who can train additional trainers. The Induction Programme the Newly Appointed Judges scheduled for January 2005 will be used as both a training session for additional Trainers and Continuing Education for Judges The Judiciary is ready to institutionalize this project Project management and co-ordination Two Judges have been trained as Trainers who can train additional trainers The Judiciary is ready as an institution to manage and coordinate the training programme Level of skill and capacity The current Trainers were appointed based on their track record in terms of skills and experience There is sufficient skills and capacity to induct and train the Newly Appointed Judges and ensure Continuing Education within the Judiciary Senior management support Trainers are currently engaged in the implementation and further development of training programmes within the project Sufficient Organization structure The project is incorporated into the plans of the Judiciary Sufficient Future funding Critical for continuation and sustainability Plans are in place to ensure that there is future funding for the project Engagement /enthusiasm of stakeholders Stakeholder enthusiasm is very high Sufficient Levels and availability of skills Currently there is a limited pool of Judges in South Africa. This project seeks to address this problem by ensuring that the newly appointed Judges receive training that is required Plans are in place to develop the project further to ensure that training continues amongst Judges Page 106 Evaluation, Impact & Effectiveness Assessment of the CJSP E12 Report Project Code: E12 Sub-Programme: Transforming the Judiciary Project: Newly Appointed Judges’ Orientation Programme (NAJOP) 1. Project Purpose The aim of the project is to systematically induct newly appointed Judges into the South African Judicial System. 2. Project Performance Anticipated milestones Status Comments Assign Project coordinator 100% complete The Coordinators has been appointed and is implementing the project Develop Orientation Programme 90% complete Implementation is pending Identify Trainers 100% complete Trainers identified - implementation is pending Invite participants In progress By the end of July there were only 5 newly appointed judges. The number has now increased to 14 Induct participants October 2004 Induction has been rescheduled to January 2005 after the October 2004 session was postponed Evaluate the programme Initially the first training was scheduled for July 2004 then postponed to October 2004. In July there were only five newly appointed Judges to be oriented and a decision was taken that it would not make financial sense to train such a small number of participants. The Judicial Services Commission has since appointed nine more Judges. In October 2004 the Judicial Services Commission set again and appointed more Judges. Training of all the Newly Appointed Judges will now take place in January 2005. 3. Effectiveness Indicators The orientation programme will only start in January 2005 thus its effectiveness cannot be measured. 4. Sustainability Factor Status Institutional Readiness Readiness for operational owners to manage implemented services/infrastructure This project is linked to the Training of Trainers Project (TTT). Four Judges have been trained as Trainers who can train additional trainers. For example, in October 2004 the Newly Appointed Judges induction programme will be used as a training session for additional Trainers The Judiciary is ready to institutionalize this project Project management and co-ordination By the end of August 2004 the Judicial Services Commission had appointed New Judges whose induction will take place in January 2005 The Judiciary is ready as an institution to manage and coordinate the training programme Page 107 Evaluation, Impact & Effectiveness Assessment of the CJSP E12 Report Page 108 Factor Status Institutional Readiness Level of skill and capacity The current Trainers were appointed based on their track record in terms of skills and experience There is sufficient skills and capacity to induct and train the newly appointed Judges Senior management support Trainers are currently engaged in the implementation and further development of training programmes within the project Sufficient Organization structure The project is part of the organizational plans of the Judiciary Sufficient Future funding Critical for continuation and sustainability Plans are in place to ensure that there is future funding for the project. Engagement /enthusiasm of stakeholders Stakeholder enthusiasm is very high Sufficient Levels and availability of skills Currently there is a limited pool of Judges in South Africa. This project seeks to address this problem by ensuring that the newly appointed Judges receive the training that is required Plans are in place to develop the project further to ensure that training continues amongst Judges Evaluation, Impact & Effectiveness Assessment of the CJSP E13 Report Project Code: E13 Sub-Programme: Transforming the Judiciary Project: Continuing Education for Judges Training Programme (CEJTP) 1. Project Purpose This project aims to provide social context and relevant continuing needs-based education for South African Judges. 2. Project Performance Anticipated Milestones Status Comments Assign Project coordinator 100% complete The Coordinator has been appointed Determine training needs In progress The Continuing Education for Judges will be based on Topical Issues as they emerge internationally and within South Africa. The topics for the training are still to be determined Develop training programme Pending Identify trainers Pending Identify participants Pending Invite participants Pending Conduct the needs-based education programme Pending The Interim Advisory Board towards the New National Justice College of South Africa is expected to identify Topics for training based on social issues that present themselves within South Africa and internationally. Should there be a need to train South African Judges on certain issues related to this challenge, the Interim Advisory Board will first have to determine the need for such training. 3. Effectiveness Indicators The project will be implemented in conjunction with the Newly Appointed Judges’ Orientation Program. Effectiveness of the project can only be measured once the project is implemented. 4. Sustainability Factor Status Institutional Readiness Readiness for operational owners to manage implemented services/infrastructure This project depends mainly on the training needs that have yet to be identified The Justice College is ready to institutionalize this project Project management and co-ordination The Interim Advisory Board of the New National Justice College of South Africa is currently responsible for this project The Judiciary is ready as an institution to manage and coordinate the training programme Page 109 Evaluation, Impact & Effectiveness Assessment of the CJSP E13 Report Page 110 Factor Status Institutional Readiness Level of skill and capacity The current Trainers will be engaged once the training is developed There is sufficient skills and capacity to develop and conduct the training Senior management support Training needs are still to be determined Sufficient Organization structure The project falls within the Judiciary Sufficient Future funding Critical for continuation and sustainability Plans are in place to ensure that there is funding for the project. Engagement /enthusiasm of stakeholders This cannot be measured as the training has not commenced In progress Levels and availability of skills Trainers will be drawn from the Training of Trainers Programme Plans are in place to develop the project further to ensure that training continues amongst Judges Evaluation, Impact & Effectiveness Assessment of the CJSP E9 Report Project Code: E9 Sub-Programme: Transforming the Judiciary Project: Development and Implementation of the Prosecutor-Led Plea-Bargaining Training Programme (DIPBTP) 1. Project Purpose The aim of the project is to appraise the magistrates and immerse prosecutors in alternative forms of justice with a view to reducing case-backlogs in the South African courts. 2. Project Performance Anticipated Milestones Status Comments Assign prosecutorial and magisterial Project Coordinators 50% complete The Judiciary has appointed a coordinator whilst the NPA has recently appointed a new coordinator Determine training needs 0% complete Pending Develop training programme 0% complete Pending Identify trainers 0% complete Pending Identify participants 0% complete Pending Invite participants 0% complete Pending Conduct training 0% complete Pending Internal assessment and external evaluation 0% complete Pending The project is behind schedule primarily because of poor commitment and accountability from NPA project managers. During the final quarter of the 2003/2004 financial year the project had not commenced. The CJSP still has to make a decision regarding the way forward on this project although implementation plans have been developed and are ready. 3. Sustainability Factor Status Institutional Readiness Project management and co-ordination The project has not been implemented yet because it depends on joint coordination by the Judiciary and the NPA. The Judiciary has appointed a Coordinator while the NPA has recently substituted the former Project Coordinator with a new person who will ensure project inception. Implementation plans are in place and a decision will soon be made on how to take the process further A New Project Coordinator has been recently appointed Senior management support The NPA is addressing issues of Implementation A New Project Coordinator has been recently appointed Future funding Project plans with budgets have been developed Sufficient Engagement /enthusiasm of stakeholders The CJSP and the NPA are jointly engaging stakeholders to ensure implementation and sustainability of the project Sufficient Levels and availability of skills The Judiciary and the NPA are committed to the development and implementation of the Prosecutor-led Plea-bargaining Training Programme Plans to ensure sufficient skills levels are in place Page 102 Evaluation, Impact & Effectiveness Assessment of the CJSP F1 Report Project Code: F1 Sub-Programme: Capacity Building within the DOJCD Project: Establishing DOJCD Program Management Support Unit (PMSU) 1. Background a. Project Problem Statement Sections 38(a) (i) of the PFMA and 3.2.7(a) of the Treasury Regulations defines the CEO and CAO’s as responsible for ensuring “risk-free as well as effective, efficient and transparent financial and (project) management systems and internal controls” within the DoJCD. In this regard, the DoJCD has received successive negative internal audit report findings on donor funded projects (with the exception of the CJSP). The DoJCD is implementing a number of development programs, each being managed separately by the various business units of the department. These projects are largely managed by line managers seconded to projects. The DoJCD currently lacks the capacity to manage project funding. In addition, there is a shortage of specialist project design and management skills in the DOJCD. The CJSP-PMSO currently provides program management of USAID donor funds and has given some project and financial management support to the projects that fall within the CJSP. These projects represent a sub￾set of the development program and projects being implemented by the DoJCD and NPA. The CJSP-PMSO only concerns itself with USAID funded initiatives, whereas the broader DoJCD development programs receive department and other donor funding. b. Project Purpose To establish the DOJCD’s Program Management & Support Unit, thereby providing “an integrated and synergistic approach towards core and donor funded project designs and developments within DOJCD (to avoid unnecessary slack, duplication and resource wastage) and towards effective management of relevant cross-cutting issues, project interfaces and interdependencies across all DOJCD’s Business Units and strategic partners.” 2. Project Definition a. Objectives • To establish a Program Management & Support Unit (PMSU) that ensures “the consolidation, synergistic designs and development, appropriate incubation and / or piloting as well as the effective implementation, management, monitoring and impact evaluations of DOJCD’s core and donor funded projects and their related activities”. • To ensure transfer and development of project management skills within DOJCD by the CJSP-PMSO. b. Key Deliverables • Design and approval of PMSU function in the DOJCD • Determine, recruit and appoint PMSU staffing requirements. • Develop PMSU skills through assessments, development interventions and skills transfer c. Key Stakeholders • National Prosecuting Authority • DoJCD business units Page 111 • CJSP PIC, POC and PMSO Evaluation, Impact & Effectiveness Assessment of the CJSP F1 Report d. Key Project Team Members Sponsor: V Pikoli (Girector-General DoJCD) Project manager: Dr B Minyuku Steering Committee: Adv. Simon Jiyane: MD – Court Services Business Unit, DOJCD; Ms Jackie Ngeva: MD – Human Resources Business Unit, DOJCD; Mr. Max Budeli: Chief Audit Executive, DOJCD; Adv. Patric Mtshaulana: Advocate of the Supreme Court; Mr. Moss Hadebe: Chief Executive Officer, Nokusa Consulting; Dr. Silas Ramaite: Deputy National Director of Public Prosecutions, NPA; Dr. Biki S. V. Minyuku: Program Director, CJSP; PMSU Chief Operating Officer (to be appointed) 3. Project Performance a. Progress against Project Milestones Key Milestones Status Planned End Date Actual End Date % Complete Progress Assessment Establish a steering committee Completed Dec-03 Jul-04 100% G Approval of blue-print for the PMSU Completed Dec-03 Jul-04 100% G Recruit Chief Operating Officer and Financial Manager On hold Apr-04 Jul-04 20% R The Director-General having a rethink about the PMSU Skills transfer & development Sep-04 R CJSP-PMSO exit May-05 R The project has been allocated R 2,690,000 from CJSP, none of which has been spent to date. b. Notable Outputs and Successes to Date A blue print for the PMSU has been developed and approved by the steering committee. The blue print states the following as the role of the PMSU: • Coordinate all DOJCD Projects and their related Development and or Donor Funds • Determine / establish project needs, explore gaps, perform feasibility studies, design, develop and write Funding Proposals to help implement and sustain new and existing DOJCD Projects • Resource and provide a single point relevant technical support services to all DOJCD projects to help nurture and sustain existing and new DOJCD projects towards fruition and maturity • Provide intellectual and project leadership within DOJCD vis-à-vis project management principles and standards • Thoroughly exploit and effectively manage synergistic intra as well as cross projects interdependencies • Supervise, manage and oversee staff, the budget and day-to-day operations of the DOJCD’s Project Management Support Unit • Formally and periodically Report and accordingly Account to the D-G, funders and all the relevant stakeholders • Develop and effect rational performance measures for all DOJCD’s projects • Help train and develop capacity within DOJCD in project management principles, implementation and monitoring tools • Monitor, conduct, undertake and periodically report on the progress, performance appraisals and projects evaluations of all DOJCD Project Managers as well as their projects, respectively Page 112 Evaluation, Impact & Effectiveness Assessment of the CJSP F1 Report c. Current Project Challenges, Dependencies, Risks and Issues The key issue and potentially risk in the project is the delay in finalizing the structure of the PMSU, in particular, the level of the head of the PMSU. The PMSU blue print, as approved by the steering committee, recommends that the head of the unit be pitched at Chief Operating Officer level. The Director-General is considering an alternative level for the head of the PMSU. This change would also impact the way funds are applied in the PMSU. The Director-General and the Steering Committee have not as yet been able to meet to work through this obstacle. While meetings have been scheduled, they have been cancelled due to the unavailability of the Director General. This is preventing the project from moving forward and places delivery at risk. d. Project Relevance The successes and challenges of the CJSP and its projects indicate a high need for the PMSU to continue and enhance the service of the CJSP-PMSO: Learnings Challenges • Effective donor funding management resulting in clean audit findings and structure process for requests for USAID funds • The CJSP team has developed an intimate understanding of the operations and strategy of the DoJCD • PMSO value-add in terms of technical and strategic mentorship to project managers • Ability to provide a highly skilled resource (Dr Minyuku) to drive the high priority Transforming the Judiciary program • Fragmented approach to project management, both in the CJSP and DoJCD, often leads to poor alignment and resource / scheduling conflicts • Many projects have limited or uncertain sustainability due to funding issues. Some project managers are uncertain of or lack confidence in the funding process of the DoJCD. Lack of funding may require reprioritization of DoJCD activities • Most projects grapple with an effective change management approach for the complex organization structure arising from the separation of powers • Stretched project managers, seconded from line functions, have placed some projects at risk due to lack of capacity and limited professional experience as project managers 4. Sustainability Below are the key factors for sustainability of the PMSU Factor Status Sustainability Organisation structure As this project is about delivering strategic project leadership capability, structure is an important factor for sustainability. As discussed above, there is uncertainty about the structure of the PMSU Limited The steering committee is the valid governance to address the organization issue, however, it is not doing so as yet Availability of skills The DoJCD currently does not have skills that can be redeployed to the PMSU. The DoJCD is not an employer of choice for program / project management skills, hence it is not well placed to attract and retain talent. The CJSP￾PMSO has program management skills that can be transferred to appointed personnel. CJSP￾PMSO skills transfer will only be effective if the appointed personnel are competent and experienced in program / project management. Thus the ability to attract and retain will be critical Limited There are representatives from business on the steering committee that can guide the DoJCD in this aspect. Continued uncertainty about the structure of the PMSU is, however, preventing concrete actions towards identifying a sustainable solution to capacity building Page 113 Evaluation, Impact & Effectiveness Assessment of the CJSP F1 Report Page 114 Factor Status Sustainability Future funding USAID has approved funding for one year of operation of the PMSU. The steering committee has approved inclusion of the PMSU in the MTEF of the DoJCD Cannot evaluate Do not have information that confirms MTEF has been allocated 5. Key Findings The resourcing and completion of the PMSU will have the highest impact on overcoming the challenges being experienced by DoJCD and CJSP projects. The extent and strategic level of these challenges certainly justifies the proposed level and capacity of the PMSU. The project must thus be considered a high priority initiative in the CJSP. Continuing delays in resolving the uncertainty about the structure and way forward place the successful implementation of the PMSU at significant risk. To place the project back on track towards meeting its objectives, the following actions require urgent participation of the senior stakeholders of the project: • Steering committee meeting with the Director-General – The above role-players must prioritize making a decision of the way forward for the PMSU. • Fast-track the appointment of the PMSU Chief Operating Officer – A competent and experienced Chief Operating Officer will inject the necessary momentum and capacity into the establishment of the PMSU. • Appoint a task team to find and implement innovative solutions for recruiting talent into the PMSU – This action can be done in parallel with the appointment of the COO. Actions that the task team should consider include: o Determining the extent of the challenge for government departments to establish internal project management and consulting capability by researching what models and strategies have been successful and unsuccessful in government/parastatals in SA o Determining best practice from above o Exploring the secondment of people from other government departments and parastatals that have been successful (to accelerate the establishment and benefit from their learnings) o Developing a compelling employee value proposition Evaluation, Impact & Effectiveness Assessment of the CJSP F2 Report Project Code: F2 Sub-Programme: Capacity Building within the DOJCD Project: Establishing the DOJCD Forensic Audit Unit (FAU) 1. Background a. Project Problem Statement The DOJCD, as part of government’s fight against fraud and corruption, identified shortcomings and lapses in the governance processes of the department. While there existed internal audit and investigations skills in the department, they were not specifically focused on forensic investigations and risk management required to identify, report and prevent fraud and corruption. b. Project Purpose The project purpose is to create forensic audit capacity and skills within the Department to enable it to deal with internal fraud and corruption. Initially it is expected that the unit will provide internal capacity on investigations related to the history of weak controls. Once controls have improved and risk detection tools are in operation, it is expected that there will be fewer incidents of fraud and corruption. It is likely that, at this point, the capacity of the FAU will focus on controls, policy and analysis. 2. Project Definition a. Objectives • Establish an internal forensic audit unit to (a) investigate suspected fraud and corruption (b) analyse data to identify suspicious activities (c) support courts and DOJCD in identifying weakness and gaps in internal control processes. • Build and develop capacity in the unit. b. Key Deliverables • Risk assessment of the DOJCD, focusing on fraud risk. • Design, resourcing and implementation of the Forensic Audit Unit. • Training and development of skills in the unit c. Key Stakeholders • DOJCD – Courts – Office of the CFO – Internal Audit • NPA – Special Investigations Unit (SIU) investigators assigned to high risk courts investigation d. Key Project Team Members Project manager: Max Budeli (Director: FAU – since March 2004) Service providers: KPMG, PWC, Gobodo Investigative Accounting and Maxima Global Page 115 Evaluation, Impact & Effectiveness Assessment of the CJSP F2 Report 3. Project Performance a. Progress Against Project Milestones Key Milestones Status Planned End Date Actual End Date % Complete Risk of Delivery Establish a steering committee Completed Mar-04 Feb-04 100% G Fraud risk assessment Completed Apr-04 Sep-04 50% A Generic risk assessment conducted and now being adapted into fraud framework Full implementation and capacitation of the Forensic Audit Unit - Includes training and development of internal specialists In progress Jun-04 Mar-05 50% A Incorrect focus on court investigations led to late start. The unit is established but audit methodology design, capacity building and organization alignment are outstanding. Budget issues are also slowing down progress Complete investigation into courts at risk In progress Jun-05 Jun-05 30% A Debate between SIU and DOJCD on deadline. SIU propose 2006 date b. Project Management, Monitoring and Reporting A steering committee has been established and is chaired by Max Budeli. A quorum of the steering committee has not been meeting regularly. Dr Minyuku provides guidance to the project, but is not in a position to make the decisions required to ensure progress on key topics such as organization structure and funding. The appointment of the project manager as the chair of the steering committee is considered a flaw in the governance structure as an independent, senior chair is required to drive attendance and decision making. c. Progress against Key Performance Measures Performance measures have not been set at project level. Performance measures are however set for each investigation conducted by the team. The unit forms just a part of the value chain responsible for fraud and corruption management. The unit does not believe that it directly influences the reduction in fraud and corruption. d. Skills Development and Change Management Currently skills are developed through transfer of skills from more experience investigators and on-the-job informal training. Skills gaps and plans to address these have been identified. The plans largely rely on outsourced training programs. Funding is currently not available to conduct the training programs. The establishment of the unit has been communicated to all stakeholders and the interventions of the unit thus (investigations, gap analysis) have been well received. e. Notable Outputs and Successes to Date The unit is established and is providing an efficient service to the investigations it is conducting. This is despite the significant challenges it faces to fully meet the purpose and objectives of the project. It has also been able to conduct some preliminary data analysis that identified areas of risk for further investigation. Skilled staff in the office of the CFO have been integrated into the unit and are transferring skills to other members of the unit. The unit has employed two junior staff members and now has a complement of 9 full￾time employees. Page 116 Evaluation, Impact & Effectiveness Assessment of the CJSP F2 Report f. Current Project Challenges, Dependencies, Risks and Issues Incorrect Application of External Service Provider Resources The external service providers, Maxima Global and Gobodo Investigative Accounting, were appointed to support the design, resourcing and implementation of the Forensic Audit Unit. They have, however, instead been applied to the investigations being conducted on specific courts. The CJSP budgeted to pay the service providers’ fees. As a result of the incorrect application of the service providers, however, the CJSP rejected the DoJCD’s request for payment of R1.2m in fees. The DoJCD has thus used its own funds to pay the service providers. The DOJCD will now use its line management to deliver the components that were supposed to be delivered by the service providers. The Department and the CJSP are discussing ways to apply CJSP funds in the FAU. Organizational Alignment Prior to the FAU being established, the office of the CFO and the SIU collaborated to conduct the investigations of the courts identified as high risk. Initially, the SIU allocated 26 investigators to the investigations – this has grown to 42. The SIU, in the past, simply received payment from the DOJCD for the cost of the investigations. This cost has not been planned in the DOJCD budget. Despite this, the SIU has effectively institutionalized the investigator capacity on the assumption that the DOJCD will fund their costs. The SIU have not committed to the FAU deadline of June 2005 to complete the investigations. The FAU anticipates that it will no longer need capacity from the SIU once the investigations are completed. There is no clear plan for the redeployment of the capacity in the SIU after June 2005. The cost of the 42 investigators is R7.5m pa. The FAU has a budget of R3m, which covers its own staff and costs. In the past, the DOJCD simply paid funds across to the SIU. In this financial year, the DOJCD has made the decision to not pay the SIU until decisions have been made around organizational and operational alignment. A discussion document and proposed structure was drafted in June 2004, but awaits discussion at a steering committee. In the interim, the DOJCD CFO is making a submission to Treasury for additional budget to cover the cost of the SIU resources. 4. Effectiveness Indicators Fraud and corruption losses should be the measure of impact. Unfortunately, the DOJCD does not have the data or reporting ability currently to measure these losses. 5. Sustainability The key factors for sustainability of the FAU are: Factor Status Sustainability Infrastructure Currently, inconsistent data limits ability to analyze and report on fraud and corruption. Implementation of Justice Yellow Pages, the Moneys in Trust project and the Basic Accounting System will solve this Sufficient The financial systems and processes will however take 5-10 years to be fully implemented Level of skill and capacity The skills development program for the FAU has been identified. The DOJCD and other government departments are working with the University of Fort Hare to develop a learnership program. Funding for the program has not been included in the MTEF for the FAU Limited It is expected that donors will fund training. There is no clear plan to obtain funding and this is certainly not a long term solution. The various government departments will have to collaborate on the program Page 117 Evaluation, Impact & Effectiveness Assessment of the CJSP F2 Report Organization structure The organization structure issues mentioned above refer Limited The organizational issues threaten the identity, funding and performance of the forensic audit unit. While a proposal has been developed, there has been no senior management attention to resolve these issues Future funding Apart from the request to cover the SIU costs, there appears to be no clear business plan in place for the future funding Limited At current capacity, the unit will be able to support the DOJCD’s basic forensic investigation needs. Further funding will be required to develop the skills to the level that fraud and corruption are managed pro-actively 6. Key Findings The Forensic Audit Unit is already impacting fraud and corruption management within the DOJCD, despite the delays in project progress and the funding and organizational issues it is experiencing. It is, however, not operating at its potential, which is required if government’s expectation of zero-tolerance of fraud and corruption is to be met. To achieve its potential, the following actions are recommended: • An independent chairperson should be appointed to the steering committee. The chairperson should drive the steering committee to resolve the organizational structure issues impacting the project. • The steering committee should review the current business plan and submit a proposal to the CJSP-PMSO and DOJCD funding processes. The FAU should consider defining performance or effectiveness measures at unit level rather than individual investigation level. These measures could serve as an important tool for staff motivation and morale. Page 118 Evaluation, Impact & Effectiveness Assessment of the CJSP G1 Report Page 119 Project Code: G1 Sub-Programme: External Anti-Corruption Initiative Project: Supporting BAC-SA’s Specialised Commercial Crimes Court Initiative (SCCCI) 1. Background a. Project Problem Statement The effect of commercial crime on the economic sector includes direct loss of R40bn annually, 30% of business failures and loss of international investment. 82% of businesses are victimised. Only 65k of 330k incidents are reported. Due to the dysfunctional judicial system conviction rates are low (only 6%), sentences are inappropriate, cases are not investigated & prosecuted successfully, resources are managed and utilized poorly and there’s a backlog of cases. In addition practices of good governance and business ethics cannot be enforced, and a desperate private sector is seeking inappropriate solutions. b. Project Purpose • To create an effective, integrated judicial environment in which perpetrators of complex commercial crimes are convicted quickly and efficiently with appropriate sentences. This will be achieved through: – Specialization in commercial cases – Co-location of and teamwork between investigators and prosecutors – Competent and effective staff – Redefined business processes – Proper management of workflow, productivity and quality • To improve the integration and alignment between the preventive mechanisms and the judicial process on an economy-wide basis through developing relationships with the private sector organizations and actively integrating them into the process. 2. Project Definition a. Objectives • Increase the number of commercial crime cases completed • Increase the conviction rate of reported cases • Increase the ratio of convictions to acquittals • Reduce case process time • Secure appropriate sentences • Increase skill levels • Reduce staff turnover • Increase court utilization • Reduce backlog of cases b. Key Deliverables • Establishment of commercial crime court centres in Pretoria, Johannesburg, Durban, Port Elizabeth and Cape Town. The Specialised Commercial Crime Unit (SCCU) also plans to deliver courts in Bloemfontein and Germiston, but these are outside the scope of the CJSP. Scope of delivery for each centre: – Feasibility study – Business establishment (accommodation, equipment & fittings) – Set-up Infrastructure – Governance structure – Systems and procedures – Operational & productivity measures Evaluation, Impact & Effectiveness Assessment of the CJSP G1 Report Page 120 – Library – Staff development – Training & mentorship program • Establishment of FEDISA / CIT centre (also outside the scope of the CJSP) • Effective Integration of private sector resources in the overall process to improve performance and skills target. c. Key Stakeholders • National Prosecuting Authority SCCU (national & provincial) – Project Sponsor • South African Police Services – National Head Office – Provincial Commissioners – Commercial Branch Commanders • Judiciary – Regional Court Presidents – Chief Magistrates – Lower Court Management Committee • Department of Justice and Constitutional Development (national & provincial) – Court Services – ISM • Business Against Crime (BAC) d. Key Project Team Members Sponsor: Chris Jordaan (Head of SCCU) Project leader: Tom Bouwer (BAC) Project manager: Karen Borcher (BAC) 3. Project Performance a. Progress against Project Milestones Overall, the project has achieved its milestones where adequate funding is in place. The project benefits from the establishment of the Specialised Commercial Crimes Unit in 1999, which provides focused leadership and sponsorship to the project. The dedicated project team from BAC play a hands-on role in the project, ensuring limited reliance on the delivery from already stretched support functions in the Department of Justice and Constitutional Development and NPA. In addition, they have been active in change management and integration of the various stakeholders towards achieving the vision for the project. Available budget (both current and future) in the NPA for prosecution staff remains the single biggest obstacle towards meeting the timelines for the remaining milestones. For future milestones, there is uncertainty as to whether budget will be approved. For initiatives in the implementation phase, overall pressure on the NPA to stay within its 2004 budgets has lead to delays in approving spending on these initiatives. Key Milestones Status* Planned End Date Actual End Date % Complete Risk of Delivery Pretoria commercial crime court Institutionalized Dec 2003 100% G Johannesburg commercial crime court 2nd phase Dec 2004 Dec 2004 90% G Some delays in appointing staff, but posts now filled Evaluation, Impact & Effectiveness Assessment of the CJSP G1 Report Page 121 Key Milestones Status* Planned End Date Actual End Date % Complete Risk of Delivery Durban commercial crime court 1st phase Q2 2005 Q3/Q4 2005 15% A Delays in establishing premises place this milestone behind schedule Port Elizabeth commercial crime court 1st phase Q3 2005 Q3 2005 15% G The court completed its first case on 19 July 2004 Cape Town commercial crime court Feasibility Q4 2005 Project on hold 5% R Business case approved, but lack of available budget in the NPA has put this on hold * Commercial crime courts implementation has following distinct phases: Feasibility - Assess whether commercial crime in area justifies a specialized court - Prepare and process business plan to establish court t t r - Obtain budgetary & implementa ion approval 1 st phase - Pilot specialized court - Deliver all infrastructure, processes, management struc ures and skills development - Obtain approval for roll-out 2 nd phase - “Roll-out” – growth phase to achieve required capacity - Ongoing change management, refining and mento ship Institutionalised - BAC hand over to NPA b. Progress against Key Performance Measures Results achieved in Pretoria show that performance has well exceeded expectations and targets since the implementation of the specialized commercial crime court there in 2000. These results can be directly attributed to the commercial crime courts initiative - the project approach focuses on the entire criminal justice process and enables an integrated organization and case flow management approach. Results achieved at the Pretoria Commercial Crime Unit: Measure Baseline (1999) Target Actual (2003) Improve productivity / number of cases completed 15 per annum 50% increase 283 (1900%) Reduce case time Av. 30 months 35% Av. 14 months Increase conviction rate of cases reported 6 per 100 15 per 100 (international standard) 23 per 100 Increase conviction rate of cases tried 70 per 100 90 per 100 95 per 100 Increase pre-trial settlements 10% 50% 50% Pass appropriate sentences Mainly suspended sentences Prison, no fine Prison or fine No suspended sentences Increase skills levels Reduce staff turnover 50% per annum 10% per annum 2.5% per annum The Johannesburg Commercial Crime Unit, in its first year of operation, shows promise of similar results: • Cases completed: monthly average of 17.6 cases per month • Conviction rate per case tried: 95.6% • No resignations in the last year Evaluation, Impact & Effectiveness Assessment of the CJSP G1 Report Page 122 c. Skills Development and Change Management Key deliverables of the project include staff development and delivery of a training and mentorship program. These are customized for each commercial crime court established and are targeted at both prosecutors and investigators: Staff development: • Determine training needs – Job profiling to identify skills specialization – Proficiency testing • Career path planning • Development of training program Execution of training and mentorship program • With support of private sector institutions, – Lecture program – Training manuals • Training impact assessments • Continuous review and assessment of staff development plan (as above) • Day-to-day interactive assistance and mentorship from project team d. Notable Outputs and Successes to Date Output / Success Description Case management approach key lever in performance results achieved so far Case management approach sees each case as a project: • Less doubt in cases – more effective sentencing • Clearer evidence – more pre-trial settlements Reduction in staff turnover Staff attrition has been effectively eliminated due to: • More focused work effort • Stronger career and personal development • Result-orientation motivates staff Effective change management Hands on involvement of the project team has delivered a working collaboration model: • Management committees (with multi-stakeholder representation) in established courts have assumed full executive and strategic ownership of the courts: • Environment encourages creative solutions to problems (e.g. JHB branch created a web-based facility to counter 419 scams) • All stakeholders experience same training and mentorship programs Plea bargaining legislation Commercial crimes courts focus on pre-trial settlements has driven significant progress on development of legislation on plea-bargaining e. Current Project Challenges, Dependencies, Risks and Issues NPA Budget Constraint Implementation in Cape Town has been placed on hold due to lack of funding for prosecution staff. Evaluation, Impact & Effectiveness Assessment of the CJSP G1 Report Page 123 Lack of Administrative Support within the Project Team The BAC project team has insufficient capacity to deliver the level of support it has been giving s crime courts. BAC intend appointing a project administrator to increase capacity. f. Alignment with Other DOJCD Initiatives The commercial crime court centres model is being implemented independently to the initiatives aimed at restructuring of court services. The current project design does not include activities to manage dependencies on projects such as Re Aga Boswa and Integrated Case Flow Management. The standardized models being delivered by these projects are, hence, not incorporated in the commercial crimes court centres model. The project management team do not believe these projects are relevant to the commercial crimes courts initiative. 4. Effectiveness Indicators The project has not set effectiveness measures and hence impact cannot be commented on. 5. Sustainability Factor Status Sustainability Institutionalization With the establishment of the SCCU in 2002, the NPA effectively institutionalized the CCCI. This was based on the success of the Pretoria commercial crime court Good President Mbeki’s commitment to the nation to deliver commercial crime courts this year will ensure the SCCU remains institutionalized Project management and co-ordination While SCCU senior management and staff play a significant project leadership role, all project management and related technical skills are delivered by BAC staff. BAC staff develop and implement project plans and change management. They play a critical role in developing business plans for roll-out Limited/Sufficient The BAC has budgeted to continue providing project management support until 2005. Should there be continued delays in allocating funding for the roll-out, there is a risk that BAC may withdraw its team. In that scenario, it is unlikely that the SCCU would be able to provide its own project management competence (due to lack of funds) to complete the roll-out Communication and change management Experience at the sites implemented has shown that the success of the commercial courts model lies in the buy-in and engagement of multiple stakeholders from different reporting lines. The BAC project team and SCCU senior management have partnered to ensure effective stakeholder involvement throughout the active sites. A change management approach has been developed that can be adapted for new sites Sufficient SCCU senior managers have, through experience, acquired change management competence and experience. The combination of BAC and SCCU management involvement optimizes buy-in and acceptance, however, sufficient skills and learnings have been transferred to enable the SCCU to adequately deliver change management without the BAC Evaluation, Impact & Effectiveness Assessment of the CJSP G1 Report Page 124 Level of skill and capacity The resourcing model proven by CCCI has a formula of 1 Court : 10 Prosecutors : 5 investigators. Specialist skills are scarce and take at least 2 years to develop. The SCCU competes for these skills against private sector forensic audit firms and other specialist units in the NPA (eg. The Scorpions.) When recruiting, the SCCU rarely gets enough applications for posts, let alone qualifying candidates. The SCCU is also unable to secure sufficient junior prosecutors for development. Apart from ensuring good succession planning, the junior prosecutors provide important research and knowledge management services to their senior colleagues Poor The AFSA BAC trust has provided funding to the NPA so that it can employ contract private prosecutors. This is, however, a short term arrangement. There is a risk that increasing workload will move the SCCU into firefighting mode and will reverse the improvements in morale and staff retention achieved thus far Training and development The project has developed a skills development and mentorship program for all parties involved in the commercial crimes courts. The BAC team have been managing the program actively and provide training and mentorship for the process and work planning components of the model. Specialist skills are obtained through outside providers and Justice College Limited The overall approach, as described in 3d, to training has proven successful. The SCCU’s ability to sustain it post implementation will depend on: - Sufficient funding - Development of a sustainable induction program - Development of sustainable mentorship & on-the-job development capacity Organization structure In the established courts, governance structures have been established, with representation from all stakeholders: - Steering boards - Management teams - Clearly defined roles & responsibilities Good The current structure has the support of stakeholders involved and provides an environment for focus on common performance measures, alignment, process refinement and knowledge and capacity building Future funding The NPA have budgeted for the full capacitation of the Pretoria and Johannesburg courts. Budget will only be approved for full capacitation of the Durban and Port Elizabeth courts once the 1st phase implementation has been successful. Allocation of budget for Cape Town has been placed on hold (for 1 year) Limited/Poor The full roll-out of commercial crime courts is at risk due to lack of funding. There is no clear plan to appeal for sufficient budget, which needs to be addressed at a senior management level. The President’s commitment, however, should compel NPA leadership to negotiate funding 6. Key Findings a. Project Performance The Specialised Commercial Crimes Courts Initiative is well designed, with a proven project management approach. Its stakeholders are engaged and take an active role in the implementation of the project. The Pretoria commercial crimes court is institutionalized and its performance has well exceeded expectations. Lack of funding in the NPA has placed implementation in Cape Town on hold. b. Institutionalisation and Sustainability The governance structure and project-based case management approach of the courts have proven to be successful in exceeding expected performance against the objectives of the project. The BAC project management team provides the required hands-on support to drive implementation. The change Evaluation, Impact & Effectiveness Assessment of the CJSP G1 Report Page 125 management approach, led in partnership by the BAC team and SCCU senior management, has led to the necessary changes in mindset and operational behaviour. Funding and talent management are key risks to the sustainability of the Commercial Crime Court Centres. Both require the clear support of senior management in the NPA to: • Guarantee funding in the MTEF for the commercial crimes courts • Support SCCU senior management in implementing a bespoke talent management strategy to enable the SCCU to attract and retain the scarce, specialist skills it requires. The project could consider leveraging NPA representatives in the CJSP-POC to campaign within the NPA for institutional funding of the SCCU. The project team should review the impact of interdependencies with other DoJCD projects such as Re Aga Boswa and Integrated Case Flow Management. This review should include changes to project design to leverage improved standards and policies in Court Services. Learnings from the commercial crime court centres projects could also be shared with project managers of these projects. DOJCD CRIMINAL JUSTICE STRENGTHENING PROGRAM (CJSP) Tel: (012) 315 1261 Momentum Building West Tower Fax: (012) 315 1863 Cnr. Pretorious & Prinsloo Streets Pretoria 0001 10 August 2004 To All Stakeholders and Management involved the Criminal Justice Strengthening Program and its projects Interim Performance Evaluation & Impacts and Effectiveness Assessment The CJSP has appointed a consortium of consultants to conduct an independent interim evaluation and assessment of the overarching impacts and effectiveness of the CJSP. The consortium comprises: Goals and Performance Analysts (GAP), Decipher Consulting and Linkages Development Agency. The evaluation will play a critical role in assisting the CJSP management and funders to understand whether the CJSP projects are achieving their goals and making an impact, how potential challenges can be overcome and how projects can be tailored to achieve their maximum potential. The expected outputs of the evaluation are: an evaluation of the alignment with the DOJCD vision and mission; an evaluation of performance and achievement of targets by all 28 projects of the CJSP; an effectiveness assessment of the projects; an impact assessment of the CJSP program and its offerings where possible; an assessment of the relevance and currency and sustainability of its offerings; and an assessment of the potential for institutionalisation of the Program. The evaluation will rely heavily on qualitative research, supported by analysis of available performance data. In order to obtain a comprehensive and holistic view, extensive participation by representatives from all relevant stakeholder groups will be required. Depending on the nature of each project, project stakeholders will be required to participate in one or a combination of the following activities: • Self-administered questionnaires • Focus group interviews • Individual interviews • Project field visits 1 2 The CJSP requires completion of the evaluation and assessment by the end of August 2004, since the cycle of the program ends at the end of September 2004. To meet this deadline, interviews, field visits and questionnaires must be completed by 24 August. This is a high priority exercise for the CJSP, with very tight timelines that need to be adhered to. Please ensure that you make time available in that period to complete questionnaires and meet with the consultants. You will be contacted shortly by a representative of one of the appointed consultants regarding your participation in the evaluation. Kind regards, Dr. Biki S. V. Minyuku CJSP-PMSO Director PROGRAMME LEVEL LIST OF QUESTIONS - Sub-Program Sponsors / PMSO STRATEGY ISSUES No Question Answer (Click the relevant box or shade by hand and write/type in your answer in the space provided) Notes 1 In your opinion what is the overall level of contribution of the 28 projects to the CJSP objectives? Please give reasons for your response Reasons: 2 Are some projects not aligned with CJSP goals? If so, which ones and why do your feel so? 3 Are there any initiatives or projects that should be introduced? If so, which ones and why do you feel so? 4 What external dependencies exist which may impact the success of CJSP initiatives (e.g. external factors, stakeholder relations, legislation, etc) and how are these managed at the program level 5 Please describe the strengths and weaknesses of implementing the CJSP and the extent to which these facilitate or hinder the achievement of goals/objectives/key deliverables? GOVERNANCE AND MANAGEMENT No Question Answer Notes 6 Rate the performance of the PIC. Please explain the reason for your rating Reasons: 7 Rate the performance of the POC. Please explain the reason/s for your rating Reasons: 8 Rate the performance of the PMSO. Please explain the reason/s for your rating Reasons: 9 How effective do you believe this tiered structure is in achieving effective management of the CJSP? 10 Describe the unique value each forum/committee adds to decision making within the program 11 At a high level, how is program performance monitored (i.e. who is accountable and what mechanisms are used)? 12 What program level meetings exist and who attends these? 13 How effective are these meetings in meeting program objectives? Please give reasons for your response Reasons: 14 How would you rate the performance of the project implementation process overall? Please give reasons for your response Reasons: 15 In your opinion, how effective is the current reporting (integrity, accuracy and quality of insights offered)? Give reasons for your response Reasons: 16 On average, how quantifiable do you believe project measurables are? Why do you feel so? Comment on project measures: 17 Rate the extent to which critical project issues are raised timeously and given the appropriate level of attention? Poor contributionLimited Average Good Excellent contribution Poor performanceLimited Average Good Excellent performance Poor performanceLimited Average Good Excellent performance Poor performanceLimited Average Good Excellent performance Very poor Poor Average Good Excellent Very poor Poor Average Good Excellent Very poor Poor Average Good Excellent Very ineffective Limited Average Good Highly effective Very poor Poor Average Good Excellent Not quantifiable Limited Average Good Extremely quantifiable 18 What are the challenges experienced with program management and coordination? And how can these be improved? Please elaborate. OVERARCHING IMPACT No Question Answer Notes 19 What is the expected impact of the CJSP? 20 In your opinion, to what extent has the impact been achieved? Use the scale alongside to rate and explain your rating? Reasons: 21 What have been the CJSP’s most significant areas of impact? (Give examples) 22 What are the key enablers to success? SUSTAINABILITY No Question Answer Notes 23 What is the DOJCD strategy/plan for sustaining/institutionalising the CJSP? 24 Rank the following elements in terms of what you believe to be most critical in ensuring the program's sustainability/institutionalisation: Assign numbers 1 to 14 to each element where 1 = most critical and 14 least critical; use each number only once Project management and coordination Infrastructure requirements Community participation/partnerships Other community structures/initiatives Related opportunities Future funding Overcoming challenges Risk mitigation plans Communication and change management Engagement/enthusiasm of stakeholders Levels of skill and capacity/availability of skills Senior management support Appropriate organisation structure Others (please specify) 25 Rate the program's overall level of readiness in terms of how the CJSP is positioned with respect to each of these elements. Project management and coordination Infrastructure requirements Community participation/partnerships Other community structures/initiatives Related opportunities Future funding Overcoming challenges Risk mitigation plans Communication and change management Engagement/enthusiasm of stakeholders Levels of skill and capacity/availability of skills Senior management support Appropriate organisation structure Others (please specify) 26 What can be done to address the identified gaps in institutional readiness in the most critical areas? Very poor impact Limited Average Good Excellent impact Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness Poor readiness Limited Average Good Excellent readiness 1 2 3 4 5 6 7 8 9 10 11 12 13 14 27 What risk mitigation plans has the CJSP put in place to date to enable institutionalisation? ACHIEVEMENT OF TARGETS No Question Notes 28 In your opinion, to what extent have the EASE (Effective, Accessible, Swift and Efficient) targets been achieved by the program as a whole? Please give reasons for your response Reasons: CAPACITY No Question Notes 29 Is there a skills development plan and if so, to what extent is it being implemented? 30 How effective do you believe the transfer of skills from CJSP to DOJ&CD staff has been? Please elaborate with reasons for your rating Reasons: 31 Describe the level of involvement of the stakeholders / government depts / NGOs in the CJSP Very ineffective Limited Average Good Very effective Poor achievementLimited Average Good Excellent achievement PROGRAMME LEVEL LIST OF QUESTIONS - PIC / POC STRATEGY ISSUES No Question Answer (Click the relevant box or shade by hand and write/type in your answer in the space provided) Notes 1 In your opinion what is the overall level of contribution of the 28 projects to the CJSP objectives? Please give reasons for your response Reasons: 2 Rate the extent to which the CJSP is aligned with the DOJCD's vision and mission? Please elaborate with reasons for your rating Reasons: 3 What external dependencies exist which may impact the success of CJSP initiatives (e.g. external factors, stakeholder relations, legislation, etc) and how are these managed at the program level 4 Please describe the strengths and weaknesses of implementing the CJSP and the extent to which these facilitate or hinder the achievement of goals/objectives/key deliverables? GOVERNANCE AND MANAGEMENT No Question Answer Notes 5 How effective do you believe the tiered structure (PIC, POC, PMSO) is in achieving effective management of the CJSP? Please give reasons for your rating Reasons: 6 Describe the unique value each forum/committee adds to decision making within the program 7 How would you rate the performance of the project implementation process overall? Please give reasons for your response Reasons: 8 On average, how quantifiable do you believe project measurables are? Why do you feel so? Comment on project measures: 9 What are the challenges experienced with program management and coordination? And how can these be improved? Please elaborate. OVERARCHING IMPACT No Question Answer Notes 10 What is the expected impact of the CJSP? 11 In your opinion, to what extent has the impact been achieved? Use the scale alongside to rate and explain your rating? Reasons: SUSTAINABILITY No Question Answer Notes 12 What is the DOJCD strategy/plan for sustaining/institutionalising the CJSP? 13 What do you believe are the most critical factors in ensuring the program's sustainability/institutionalisation? 14 How would you rate the program overall on its level of readiness for sustainable institutionalisation? ACHIEVEMENT OF TARGETS No Question Notes 15 In your opinion, to what extent have the EASE (Effective, Accessible, Swift and Efficient) targets been achieved by the program as a whole? Please give reasons for your response Reasons: CAPACITY No Question Notes Poor contributionLimited Average Good Excellent contribution Poor alignment Limited Average Good Excellent alignment Very poor impact Limited Average Good Excellent impact Very ineffective Limited Average Good Highly effective Very poor Poor Average Good Excellent Not quantifiable Limited Average Good Extremely quantifiable Poor achievementLimited Average Good Excellent achievement 16 Is there a skills development plan and if so, to what extent is it being implemented? 17 How effective do you believe the transfer of skills from CJSP to DOJ&CD staff has been? Please elaborate with reasons for your rating Reasons: 18 Describe the level of involvement of the stakeholders / government depts / NGOs in the CJSP Very ineffective Limited Average Good Very effective PROJECT LEVEL LIST OF QUESTIONS SECTION A: PROJECT OVERVIEW No Question 1 Project name: 2 What is the current phase of the project 3 In your own words, please describe why this project is being implemented 4 In your opinion, in what ways is the project addressing and/or not addressing the identified problems or goals. Please describe in detail 5 Please give suggestions on how weaknesses or gaps can be addressed (include new initiatives or ideas; or what should be removed from project scope) 6 Please list the key deliverables of the current phase of the project. How were these deliverables expected to contribute to solving the problems identified? How have they actually contributed to solving these problems? 7 Have effectiveness indicators been identified ? No Question 8 Please list effectiveness indicators and extent to which they have been realised. 9 Indicator 1 10 Indicator 2 etc 11 List the groups of people that you perceive as benefitting from the project 12 Describe the level of involvement of project stakeholders in the project. What has been done to obtain their buy-in and / or ownership of the outputs of the project? 13 Comment on accessibility of services provided by the project, in terms of: Location Language Inclusivity User friendliness Other Answer PROJECT DESIGN & MANAGEMENT PROJECT DESIGN & MANAGEMENT Answer Not achieved Achieved Exceeded Not achieved Achieved Exceeded Planning Implementation Review No Question 14 Rank the following elements in terms of what you believe to be most critical in ensuring the project's sustainability/institutionalisation: Project management and coordination Infrastructure requirements Community participation/partnerships Other community structures/initiatives Related opportunities Future funding Challenges Risk mitigation plans Communication and change management Engagement/enthusiasm of stakeholders Levels of skill and capacity/availability of skills Senior management support Organisation structure Others (please specify) No Question 15 Rate the program's overall level of readiness in terms of how the CJSP is positioned with respect to each of these elements Project management and coordination Infrastructure requirements Community participation/partnerships Other community structures/initiatives Related opportunities Future funding Challenges Risk mitigation plans Communication and change management Engagement/enthusiasm of stakeholders Levels of skill and capacity/availability of skills Senior management support Organisation structure Others (please specify) No Question 16 Please indicate your level of satisfaction with the governance structures of the CJSP (PIC, POC, PMSO) Explain the reason for your rating INSTITUTIONALISATION PROGRAM LEVEL INTERACTION Answer INSTITUTIONALISATION Answer Reason Answer 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance 1 2 3 4 5 Not Satisfied To some extent Satisfied To a great extent Very Satisfied 17 What effect/impact (positive and negative) does the current CJSP structure have on the achievement of the goals, objectives and key deliverables of your project? 18 What are the challenges experienced with program management and coordination within the CJSP? And how can these be improved? Please elaborate. SECTION B: PROJECT PROGRESS No Question 19 Is there a project work plan for the current phase of the project? If no, please give reasons why. 20 List the key milestones of the current phase Planned Date Due Date % Completion Status Milestone 1 Milestone 2 Milestone 3 Milestone 4 Milestone 5 Milestone 6 Milestone 7 Milestone 8 No Question 21 What are the main issues being experienced in achieving milestones 22 Are issues raised timeously? 23 Is the appropriate / responsible person notified of issues? 24 Are all impacted parties aware of issues and engaged in their resolution? Please explain 25 Are issues effectively prioritized and resolved? 26 Please give examples / instances of well managed issues 27 Please give examples / instances of poorly managed issues ACHEIVEMENTS AGAINST MILESTONES IMPLEMENTATION (only answer if the project is in implementation phase) ACHEIVEMENTS AGAINST MILESTONES Answer Answer 1 2 3 4 Ahead On track Behind, but delivery not at risk Delivery at risk 1 2 3 4 Ahead On track Behind, but delivery not at risk Delivery at risk 1 2 3 4 Ahead On track Behind, but delivery not at risk Delivery at risk 1 2 3 4 Ahead On track Behind, but delivery not at risk Delivery at risk 1 2 3 4 Ahead On track Behind, but delivery not at risk Delivery at risk 1 2 3 4 Ahead On track Behind, but delivery not at risk Delivery at risk 1 2 3 4 Ahead On track Behind, but delivery not at risk Delivery at risk 1 2 3 4 Ahead On track Behind, but delivery not at risk Delivery at risk No Question 28 During implementation, what changes were made to the scope of the project and why? Please list all changes made and how those changes were effected? 29 Please list factors outside of the scope of the project that influence project implementation. Explain how these factors influence the project and how they are being managed. Answer No Question 30 Describe the key enabling factors for success of the project and the extent to which they are in place 31 Who is responsible for the performance monitoring of the project? 32 Describe the methods / tools for performance monitoring(eg. Reports, meetings) 33 Which performance monitoring methods provide most value in providing information for guidance / decision-making to the project? Explain why 34 Name some other monitoring methods that could add value (if any) 35 Please describe the skills development plan for the project. 36 Please describe how skills are being transferred by the project to the DoJCD No Question 37 What skills gaps currently exist in the project team and how should these be addressed? 38 Are there adequate human resources to sustain services or infrastructure delivered by the project? If not, please share your ideas or suggestions to resolve this. 39 Target 1 Target 2 Target 3 Target 4 Target 5 Target 6 Target 7 Target 8 Please list all targets relevant to your project and indicate the extent to which they have been achieved Answer IMPLEMENTATION (only answer if the project is in implementation phase) IMPLEMENTATION (only answer if the project is in implementation phase) Answer Not achieved Achieved Exceeded Not achieved Achieved Exceeded Not achieved Achieved Exceeded Not achieved Achieved Exceeded Not achieved Achieved Exceeded Not achieved Achieved Exceeded Not achieved Achieved Exceeded Not achieved Achieved Exceeded No Question 40 To what extent has the project contributed to the results achieved against performance measures set by the project? Please explain. 41 What suggestions do you have that will improve project implementation? 42 Please share some of the lessons learnt and stories of success No Question 43 Please describe the key results or outputs from the project 44 Should there be further steps taken to consolidate the value gained from these results? If yes, please explain. 45 Please list any outstanding deliverables from the project 46 Explain how ongoing responsibility for the outputs of the project (eg. Services, manuals,policies) have been handed over to a DOJCD owner for ongoing management. Reason: IMPLEMENTATION (only answer if the project is in implementation phase) Answer CLOSED / REVIEW (if project is closed) Answer 1 3 4 5 No Contribution To some extent Adequate contribution To a great extent Most significant contribution 2 INTERIM EVALUATION AND ASSESSMENT OF THE CRIMINAL JUSTICE STRENGTHENING PROGRAM (CJSP) PROJECT STAKEHOLDER QUESTIONAIRRE NOTES: i ii Name Project name: Project role SECTION A: PROJECT DESIGN AND MANAGEMENT No Question Answer 1 In your own words, please describe why this project is being implemented 2 In your opinion, and to what extent is the project contributing towards addressing the identified problems or goals. Please comment on your rating Comments 3 Please give suggestions on how weaknesses or gaps can be addressed (include new initiatives or ideas; or what should be removed from project scope) 4 In your opinion, What is working well? What is not working well? 5 What would you do again? 6 What would you do differently? Please rate your satisfaction with the level of involvement of project stakeholders. Consider the activities taking place to obtain their buy-in and their ownership of the outputs of the project. Comments 7 Comment on accessibility of services provided by the project, in terms of: Location Language Inclusivity User friendliness Other SECTION B: IMPLEMENTATION PROGRESS Based on the phase of the project, please complete as follows: Planning phase: Sections A & D Implementation phase: All sections Closed / review phase: Sections A, C & D Please consider the following questions in relation to the specific project in the CJSP that you are personally involved in. 1 2 3 4 5 No Contribution To some extent Adequate contribution To a great extent Most significant contribution 1 2 3 4 5 Not Satisfied To some extent Satisfied To a great extent Very Satisfied No Question Answer 8 Please list factors outside of the scope of the project that influence project implementation. Explain how these factors influence the project and how they are being managed. 9 Overall, how would you rate progress towards achieving the important milestones for this project? (if the project is in the planning phase, please rate based on the activities required for planning) 10 Describe the key enabling factors for success of the project and the extent to which they are in place (eg. Funding, technology etc) 11 Which performance monitoring methods used in the project provide most value in providing information for guidance / decision-making to the project? Explain why 12 Name some other monitoring methods that could add value (if any) 13 What are the main issues being experienced in the project? 1 14 Are issues raised timeously? 15 Is the appropriate / responsible person notified of issues? 16 Are all impacted parties aware of issues and engaged in their resolution? Please explain 17 Please give examples / instances of well managed issues 18 Please give examples / instances of poorly managed issues 19 What suggestions do you have that will improve project implementation? 1 2 3 4 5 Unsuccessful To some extent Successful To a great extent Extremely successful SECTION C: RESULTS No Question Answer 20 To what extent has the project contributed to the results achieved against performance measures set by the project? Please explain. Reason: 21 Please rate the success achieved in implementing the skills development plan for the project 22 Please rate your satisfaction with the transfer of skills from the project to DoJCD staff Comments 23 Are there adequate human resources to sustain services or infrastructure delivered by the project? If not, please share your ideas or suggestions to resolve this. 24 Where the project is closed, should there be further steps taken to consolidate the value gained from these results. If yes, please explain. SECTION D: SUSTAINABILITY & INSTITUTIONAL CAPABILITY No Question Answer 25 Please indicate your level of satisfaction with the governance structures of the CJSP (PIC, POC, PMSO) with respect to their influence on the project. Explain the reason for your rating Reason 1 2 3 4 5 Unsuccessful To some extent Successful To a great extent Extremely successful 1 2 3 4 5 No Contribution To some extent Adequate contribution To a great extent Most significant contribution 1 2 3 4 5 Not Satisfied To some extent Satisfied To a great extent Very Satisfied 1 2 3 4 5 Not Satisfied To some extent Satisfied To a great extent Very Satisfied SECTION D: SUSTAINABILITY & INSTITUTIONAL CAPABILITY No Question Answer 26 Rank the following elements in terms of what you believe to be most critical in ensuring the project's sustainability/institutionalisation: Project management and coordination Infrastructure requirements Community participation/partnerships Other community structures/initiatives Related opportunities Future funding Challenges Risk mitigation plans Communication and change management Engagement/enthusiasm of stakeholders Levels of skill and capacity/availability of skills Senior management support Organisation structure Others (please specify) 27 Rate the program's overall level of readiness in terms of how the CJSP is positioned with respect to each of these elements Project management and coordination Infrastructure requirements Community participation/partnerships Other community structures/initiatives Related opportunities Future funding Challenges Risk mitigation plans Communication and change management Engagement/enthusiasm of stakeholders Levels of skill and capacity/availability of skills Senior management support Organisation structure Others (please specify) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance Poor assistance Limited Average Good Excellent assistance