EVALUATION OF THE LES ASPIN CENTER FOR GOVERNMENT ANTI-CORRUPTION AND GOOD GOVERNANCE IN AFRICA GRANT PROGRAM FINAL REPORT DECEMBER 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Richard Blue (Social Impact, Inc.), Lisa Slifer-Mbacke (Management Systems International), and Kelly Heindel (Social Impact, Inc.) Research Assistance provided by Kristine Johnston and Amr Moubarak (Social Impact, Inc.) EVALUATION OF THE LES ASPIN CENTER FOR GOVERNMENT ANTI-CORRUPTION AND GOOD GOVERNANCE IN AFRICA GRANT PROGRAM FINAL REPORT ON OF THE LES ASPIN CENTER FOR GOVERNANCE OF MARQUETTE UNIVERSITY GRANT PROGRAM FINALT DRAFT DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document was submitted by Social Impact, Inc., with Management Systems International, to the United States Agency for International Development under: USAID Contract No. RAN-I-00-09-00019-00, Task Order No. AID-OAA-TO-10-00048 Evaluation Services Indefinite Quantity Contract Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA, 22201 Tel: (703) 465-1884 Fax: (703) 465-1888 info@socialimpact.com ACRONYMS AFR Africa Bureau AI Appreciative Inquiry DG Democracy and Governance FEC Federal Elections Commission FGD Focus Group Discussion FY Fiscal Year GAO Government Accountability Office IRI International Republican Institute KI Key Informant LAC Les Aspin Center MP Member of Parliament MSI Management Systems International NDI National Democratic Institute NED National Endowment for Democracy NGO Non-governmental Organization PMP Performance Monitoring Plan RCT Randomized Control Trial SI Social Impact SOW Scope of Work TI Transparency International USAID United States Agency for International Development USG U.S. Government WAODEG West Africa Organization for Democracy and Economic Growth ACKNOWLEDGMENTS We would like to express our appreciation to USAID, the Les Aspin Center and Marquette University for their support throughout this evaluation. Cephas Lerewonu, Meghan Lefeber and Father Timothy O’Brien from the Les Aspin Center have provided invaluable assistance, information and interviews throughout our work. Jeremy Meadows and Ryan McCannell of USAID Africa Bureau, Office of Sustainable Development (AFR/SD), whose office commissioned this evaluation, have provided consistent support, guidance and comments. In addition, Maryanne Yerkes of DCHA/DG and the contracting officer technical representative for the Les Aspin program provided helpful background information and comments on our work. Finally, we want to thank the Les Aspin African Fellows Program participants for taking time to answer our questions and provide us with valuable insight into their experience and work. Map of Program Locations TABLE OF CONTENTS Executive Summary............................................................................................................................................................. i Background.......................................................................................................................................................................... 1 Purpose, Scope and Methodology.................................................................................................................................... 2 Findings and Conclusions.................................................................................................................................................. 5 Effectiveness of Partner’s Approach to Achieving Results (SOW Question 1).................................................. 6 Findings...................................................................................................................................................................... 6 Effectiveness Conclusions .....................................................................................................................................17 Participants’ Ability to Utilize Skills or Information Learned During the Program (SOW Question 2).......19 Findings....................................................................................................................................................................19 Participants’ Ability to Utilize Skills or Information Learned During the Program Conclusions.............21 Impacts on Governance Accountability Systems in Africa (SOW Question 3 and partial Question 4) .......22 Findings....................................................................................................................................................................22 Impacts on Governance Accountability Systems in Africa Conclusions.......................................................26 Sustainability and the Extent to which it has been built into the program (SOW Question 5)......................26 Findings....................................................................................................................................................................26 Sustainability Conclusions.....................................................................................................................................32 General Conclusions........................................................................................................................................................32 Overall Recommendations..............................................................................................................................................34 Specific Recommendations for each SOW question ..................................................................................................35 ANNEXES........................................................................................................................................................................37 Annex A: Scope of Work ................................................................................................................................................38 Annex B: Program schedules and agendas..................................................................................................................41 Annex C: Documents Reviewed ...................................................................................................................................53 Annex D: Data Collection Instruments........................................................................................................................54 Annex E: Table of Performance Indicators.................................................................................................................64 Annex F: FY 2010 Les Aspin Program Budget...........................................................................................................66 Annex G: Success stories and analysis........................................................................................................................... 67 Annex H: Les Aspin Center’s Response to Draft Evaluation Report......................................................................74 2 TABLES AND CHARTS Map of Program Locations............................................................................................................................................... 1 Table 1: Exposure Hours................................................................................................................................................... 2 Table 2: Alumni Survey Respondents by Country ........................................................................................................ 4 ................................................................................................................................................................................................ 4 Table 3: Participant Data 2001- 2011* ............................................................................................................................ 6 Table 4: Les Aspin’s expenditure analysis (FY 2007, 2009, 2010) ............................................................................11 Chart 1: Proportion of Major Cost Items in FY2007 ................................................................................................. 12 Chart 2: Proportion of Major Cost Items in FY2009 ................................................................................................. 12 Chart 3: Proportion of Major Cost Items in FY2010 ................................................................................................. 13 Table 5: Amount, Percent of Expenditure, and Cost Per Participant for Les Aspin and USAID Comparison Program.......................................................................................................................................................15 i Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report EXECUTIVE SUMMARY The Les Aspin African Fellows Program first began in Kenya with funding provided by the USAID/Kenya Mission in 1996. The program extended to include Ghana with additional funds received from the USAID/Ghana Mission in 1998. Funding for the program moved to central funding from Washington in 2001, which allowed the Les Aspin Center (LAC) to increase the total number of countries involved in the program to six. The focus of the program remained equally split among three West African countries (Ghana, Mali, and Nigeria) and three in East Africa (Kenya, Tanzania, and Uganda). Kenya and Ghana remained central countries in each sub-region and serve as the orientation point for the East and West Africa cohort, respectively. Initially, the East Africa Fellows Program focused on electoral process and conflict resolution, which was appropriate for the period moving up to and just following the contentious 1997 national elections in Kenya. The West Africa program focused on anti-corruption and good governance. However, the Les Aspin program developed a common focus of good governance and anti-corruption for both the East Africa and West Africa cohorts in 2008. The 2008 program also expanded to include leadership development. Despite the shift toward a common focus for both the East and West Africa program, the LAC continued to keep the West Africa and East Africa cohorts separate in their program, each taking consecutive study tours to the U.S. The overall goal of the Les Aspin program under the current, centrally funded grant from USAID is to promote accountable-governance systems and reduce corruption in Africa. The program selects participants from multiple sectors, including media, civil society, and government. The LAC provides criteria to the USAID Missions for the identification and selection of participants, which includes gender equity—defined as equal numbers of men and women selected—and a national balance of participants, not just from capital cities but from secondary cities and towns throughout the country. Participants are also selected based on age and must be between twenty-five to fifty-five years of age. In addition, the selection criteria ask that the participants be chosen from government and civil society organizations that work in the administration of justice. The criteria suggest that participants can also come from media, or from other sectors working on accountability issues. Currently, the program lasts five weeks. According to the Les Aspin Center, when faced with the need for cost-savings, the program was reduced from six weeks to five in 2006. The first week of each program (East and West Africa) begins with one week in Africa, providing an orientation and general discussion around the definition of corruption and anti-corruption efforts within the host country. For the West Africa program, this orientation week is held in Ghana, and for the East Africa program, in Kenya. Participants then travel to Washington, D.C. for a week of site visits and presentations on the U.S. political process. They then travel to Milwaukee, Wisconsin, where they are hosted by Marquette University and provided with a variety of presentations on the campus, as well as meetings with local government officials (city council members, staff of the mayor’s office, and judges). The participants then return to Washington, D.C. for two additional weeks, which include site visits and presentations by various U.S. Government (USG) organizations and think tanks. A number of these, including the Office of Government Ethics and the U.S. Government Accountability Office (GAO), focus specifically on ethics and anti-corruption. The group also tours the White House. Upon completion of the program at the end of the five weeks, the participants return to their respective countries. The training framework developed by the LAC states that the training will be “in an experiential and multi-dimensional format” and will “provide participants with skill-enhancement training on how corruption can be managed.” Finally, it states that, at the end of the training, “participants will also learn how to utilize the training resources and experiences to plan and develop results-based strategies for the implementation of accountability procedures.” ii Table 1: Exposure Hours LOCATION LECTURE SITE VISITS DISCUSSION/WORKSHOPS1 TOTAL Accra 17 hours 2 hours 6 hours 25 hours Wisconsin 12 hours 13 hours2 11.5 hours 36.5 hours Washington, D.C. 34 hours 11 hours 30 hours 75 hours TOTAL 63 HOURS 26 HOURS 47.5 HOURS 136.5 HOURS The program presenters vary widely, from government officials working in the area of anti-corruption to academicians, civil society organizations, and think tanks. There are a few portions of the program that follow traditional “training” approaches, in terms of formal training modules, exercises, and clear lesson objectives. These include the strategic planning workshop conducted in Ghana and the conflict-resolution training in Milwaukee. The majority of the presentations, in Africa and the U.S., are standard, lecture-style presentations—a speaker presents for twenty to thirty minutes, with a question-and-answer session at the end. Throughout the program, very few professional trainers or facilitators are used, in either Africa or the U.S. Site visits are included, allowing participants to observe activities in situ, in addition to the presentation; however, interaction with key officials at the sites is inconsistent. Some visits were just ‘look and see,’ in contrast to the tours of the U.S. Congress and the White House, where the participants actually interact with officials who provide information about the working system of the entity. Table 1 above is an estimated breakdown of the number of hours 2011 participants spent in lecture, discussion groups and site visits. These hours were determined from the sessions listed on the schedules provided to us by the Les Aspin Center. Certain lecture sessions do create discussions or other group work, but this varies widely and is not listed on the workshop schedules. PURPOSE, SCOPE AND METHODOLOGY PURPOSE In September 2011, the Les Aspin center will have completed its tenth year of programming under the grant provided by United States Agency for International Development Africa Bureau (USAID/AFR) in Washington.3 1 It was not always clear from the Les Aspin schedule if a session was a lecture or a workshop. In those cases, the evaluation team assumed (based on their observation of numerous sessions throughout the program), that any session lasting an entire day was a workshop. Any partial day session was assumed to be a lecture unless otherwise specified. At this point, USAID will have given nearly $5 million in support to the program. Given that this large and sustained support has not been evaluated to date, and that USAID has an increased interest in evaluation and measurement of impact, USAID/AFR commissioned this evaluation to assess the effectiveness of the Les Aspin program in achieving its objectives over the last ten years. These objectives 2 Eight of these thirteen hours of “site visits” would more appropriately fall under a “cultural experience” category. The schedule describes the day’s activities as “Cultural Experience: Milwaukee. Visit to sites of cultural and historical importance.” 3 A no-cost extension through June 2012 has recently been provided to the Les Aspin Center to complete final reports and close out the project. iii Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report include increasing individual and institutional capacity of participants and strengthening the accountability systems in target countries. METHODOLOGY Key Evaluation Questions The evaluation team used the following Scope of Work (SOW)4 questions to guide their analysis: 1. How effective is the partner’s approach to achieving results? 2. Do the program participants have the ability to utilize skills or information learned during the program to affect corruption and good governance in their home countries? 3. Do participants’ experiences in the program help to strengthen accountability systems and the U.S. government’s effort to enhance the lives of the people of Africa? 4. What is the (cost-benefit) value of this program to the objectives of the U.S. government and do the anti-corruption and good governance programs directly contribute to the goals of USAID and to the U.S. government’s interests in promoting accountable governance systems and the capacity to sustain the accountability systems in Africa as the Les Aspin center claims? 5. How has the partner built sustainability into the program for long-term impact? Could or should these sustainability have been strengthened? Design and Methods Used The team used a mixed-methodology approach, combining quantitative survey data with qualitative in-depth interviews, focus groups, and program observation to strengthen the validity of the findings. This approach allowed the evaluators to obtain and document training results beyond the initial reports by participants about a training experience. The evaluation took place from October 2010 through November 2011. This allowed the team to examine the full program implementation cycle, including site visits to Marquette University in Milwaukee, Wisconsin, as well as to the Les Aspin Center in Washington D.C. and Accra, Ghana for the 2011 classes. Additional site visits to Accra and Nairobi, Kenya for the alumni refresher courses followed. The alumni refresher courses were conducted for the first time this year, at the suggestion of USAID/Washington. USAID suggested combining the East and West Africa programs into one program and then adding an “alumni refresher” course as the second program in 2011. According to USAID staff, the idea was that this would provide insight into what has been accomplished by the program (through its alumni) and allow Les Aspin to reconnect with its participants.5 GENERAL CONCLUSIONS In the full version of this report, the following, general conclusions have been augmented by specific conclusions based on the findings for each of the SOW evaluation questions investigated by the evaluation team. The findings supporting these general conclusions are presented in the body of this report. 4 See Annex A for full SOW. 5 See Annex B for program schedules and agendas. iv The Selection Process 1. In spite of weaknesses in program design and implementation identified in the Findings section, the Les Aspin program has identified and motivated a potential critical mass of African leaders who could be utilized in USAID and other donor efforts to strengthen good democratic governance. This is a valuable asset for African governments, civil society, and USAID. 1. On balance, the selection process has fulfilled the purpose intended, including mix of public and private participants, gender balance, and geographic distribution among the six participating countries. However, there is considerable variation in the process and transparency and clarity with regard to the initial selection process needs to be strengthened. Participants were often surprised they had been selected and initially were not fully informed about the program objectives and logistical arrangements. Program Effectiveness 1. Participants clearly value the program and the opportunity to go to the USA and learn about U.S. systems and processes for promoting good governance and controlling corruption. However, recall of specific skills learned or of their application in African settings was much weaker. General research into adult, skill learning supports the conclusion that unless skills are used in the work place on return, they will eventually wither. The same applies to motivational and attitudinal changes promoted by Les Aspin program. Skills used were those taught through interactive workshops and which were applicable to the trainees’ immediate workplace. 2. Although the original proposal and the program theory emphasized interactive learning as the key distinctive feature, the actual program is heavily reliant on lectures for some segments of the thirty￾and-one-half days of exposure. This reliance, combined with the absence of specific program￾module-training objectives, lesson plans, and follow-up learning assessments serve to undermine the overall effectiveness of the program. 3. The LAC invites feedback from participants on an ad hoc basis, but has no systematic program evaluation process in place by which outcomes and impact periodically can be measured, and through which informed program improvements can be made. 4. The program’s standard presentation approach reduces the potential for learning, and does not capitalize on the depth of experience and knowledge brought by the participants. This approach signifies treatment of participants as “students,” rather than key stakeholders in an adult-learning program. 5. Based on the evidence available, the program, on the whole, is cost effective, compared to one other USAID-funded program with some similar features. There has been no increase in the grant level over the ten-year period. Les Aspin has found ways to absorb rising transportation and lodging costs, largely by cutting the program’s duration, reducing staff travel, and cutting lodging costs by requiring participants to share hotel rooms. 6. The program has no systematic monitoring and evaluation process in place to guide LAC leadership in coping with rising costs and changing circumstances in Africa. Without such a system in place, decisions about cost cutting or program re-orientation appear to be based more on anecdotal evidence, leading to a focus on administrative and logistical cost reduction measures, rather than a broader examination of the effectiveness of various program components. v Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report Follow-up and Sustainability 1. The program fails to meet the challenge of fostering and supporting sustainable results, through, for example, follow-up activities, either within countries or through networking across the regions. Individual trainees have taken initiatives, such as those mentioned in the success stories,6 2. Other sections of this report have emphasized the lack of follow-up support for LAC alumni to pursue good governance-anti-corruption efforts on their return to their home countries. Any continuation or expansion of the program will require both a serious review of the effectiveness of, and necessity for, all phases of the existing program, as well as a realistic cost projection of the steps necessary to strengthen longer-term asset utilization and effective contributions to USAID and host country goals and objectives, vis a vis strengthening good governance and controlling corruption. but in the main, no active follow-up program exists that would support networking, knowledge management, or specific initiatives—perhaps through a small grant program. While additional funding does not guarantee sustainability, the presence of a follow-up program with some funding would provide the kind of support needed to activate and mobilize the substantial human assets created by the Les Aspin Center Program. USAID Involvement 1. With the exception of USAID/Kenya, USAID mission participation in the participant selection process has been useful and well regarded. 2. USAID's aggregate involvement is not based on much continuous interaction or participation beyond the initial recruitment process. As a long-running grant program, USAID has seen the program as generally useful, but the program operates outside of the Missions’ development strategies; therefore, the program does not benefit from much support beyond the administration of the grant. 3. USAID has no policy or approach for capitalizing on the momentum for action that the program creates amongst its participants. This is a lost opportunity for USAID to demonstrate a sustained commitment to supporting outstanding leaders in these countries and gain a network of supporters for its democracy and governance work. 6 See Annex G 1 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report BACKGROUND The Les Aspin African Fellows Program first began in Kenya with funding provided by the USAID/Kenya Mission in 1996. The program extended to include Ghana with additional funds received from the USAID/Ghana Mission in 1998. Funding for the program moved to central funding from Washington in 2001, which allowed the Les Aspin Center to increase the total number of countries involved in the program to six. The focus of the program remained equally split among three West African countries (Ghana, Mali, and Nigeria) and three in East Africa (Kenya, Tanzania, and Uganda). Kenya and Ghana remained central countries in each sub-region and serve as the orientation point for the East and West Africa cohort, respectively. Initially, the East Africa Fellows Program focused on electoral process and conflict resolution, which was appropriate for the period moving up to and just following the contentious 1997 national elections in Kenya. The West Africa program focused on anti-corruption and good governance. However, the Les Aspin program developed a common focus of good governance and anti-corruption for both the East Africa and West Africa cohorts in 2008. The 2008 program also expanded to include leadership development. Despite the shift toward a common focus for both the East and West Africa program, the LAC continued to keep the West Africa and East Africa cohorts separate in their program; taking separate, but consecutive study tours to the U.S. The overall goal of the Les Aspin program under the current, centrally funded grant from USAID is to promote accountable governance systems and reduce corruption in Africa. The program selects participants from multiple sectors, including media, civil society, and government. LAC provides criteria to the USAID Missions for the identification and selection of participants, which includes gender equity—defined as equal numbers of men and women selected—and a national balance of participants, not just from capital cities but from secondary cities and towns throughout the country. Participants are also selected based on age and must be between twenty-five to fifty-five years of age. The selection criteria also ask that the participants be selected from government and civil society organizations that work in the administration of justice. The criteria suggest that participants can also come from media, or other sectors working on accountability issues. According to the Les Aspin Center, when faced with the need for cost-savings, the program was reduced from six weeks to five in 2006. Currently, the first week of each program (East and West Africa) begins with one week in Africa, providing an orientation and general discussion around the definition of corruption and anti-corruption efforts within the host country. For the West Africa program, this orientation week is held in Ghana, the East Africa program, in Kenya. Participants then travel to Washington, D.C. for a week of site visits and presentations on the U.S. political process. They then travel to Milwaukee, Wisconsin, where they are hosted by Marquette University and provided with a variety of presentations on the campus, as well as meetings with local government officials (city council members, staff of the Mayor’s office and judges). The participants then return to Washington, D.C. for two additional weeks, which include site visits and presentations by various USG organizations and think tanks. A number of these, including the Office of Government Ethics and the U.S. Government Accountability Office (GAO), focus specifically on ethics and anti-corruption. The group also tours the White House. Upon completion of the program at the end of the five weeks, the participants return to their respective countries. The training framework developed by the LAC states that the training will be “in an experiential and multi-dimensional format” and will “provide participants with skill-enhancement training on how corruption can be managed.” Finally, it states that, at the end of the training, “participants will also learn how to utilize the training resources and experiences to plan and develop results-based strategies for the implementation of accountability procedures.” 2 Table 1: Exposure Hours LOCATION LECTURE SITE VISITS DISCUSSION/WORKSHOPS7 TOTAL Accra 17 hours 2 hours 6 hours 25 hours Wisconsin 12 hours 13 hours8 11.5 hours 36.5 hours Washington, D.C. 34 hours 11 hours 30 hours 75 hours TOTALS 63 HOURS 26 HOURS 47.5 HOURS 136.5 HOURS The program presenters vary widely, from government officials working in the area of anti-corruption to academicians, civil society organizations, and think tanks. There are a few portions of the program that follow traditional “training” approaches, in terms of formal training modules, exercises and clear lesson objectives. These include the strategic planning workshop conducted in Ghana and the conflict-resolution training in Milwaukee. The majority of the presentations, in Africa and the U.S., are standard, lecture-style presentations—a speaker presents for twenty to thirty minutes, with a question-and-answer session at the end. Throughout the program, very few of the presenters used, in either Africa or the U.S., are trained facilitators or have training in contemporary adult learning methods. Site visits are included in the program, and are either visits to relevant organizations or institutions, or cultural visits. These visits allow participants to observe activities in context, in addition to the presentation; however, interaction with key officials at the sites is inconsistent. Some visits were just ‘look and see,’ in contrast to the tours of the U.S. Congress and the White House, where the participants actually interact with officials who provide information about the working system of the entity. Table 1 above is an estimated breakdown of the number of hours participants in 2011 spent in lecture, discussion groups and site visits. These hours were determined from the sessions listed on the schedules provided to us by the Les Aspin Center. Certain lecture sessions do create discussions or other group work, but this varies widely and is not listed on the workshop schedules. PURPOSE, SCOPE AND METHODOLOGY PURPOSE In September 2011, the Les Aspin center will have completed its tenth year of programming under the grant provided by United States Agency for International Development Africa Bureau (USAID/AFR) in Washington, D.C.9 7 It was not always clear from the Les Aspin schedule if a session was a lecture or a workshop. In those cases, the evaluation team assumed (based on their observation of numerous sessions throughout the program), that any session lasting an entire day was a workshop. Any partial day session was assumed to be a lecture unless otherwise specified. At this point, USAID will have given nearly $5 million in support to the program. Given that this sizeable, sustained support has not been evaluated to date, and that USAID has an increased interest in evaluation and measurement of impact, USAID/AFR commissioned this evaluation to assess the 8 Eight of these thirteen hours of “site visits” would more appropriately fall under a “cultural experience” category. The schedule describes the day’s activities as “Cultural Experience: Milwaukee. Visit to sites of cultural and historical importance. 9 A no-cost extension through June 2012 has recently been provided to the LAC to complete final reports and close out the project. 3 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report effectiveness of the Les Aspin program in achieving its objectives over the last ten years. These objectives include increasing individual and institutional capacity of participants and strengthening the accountability systems in target countries. METHODOLOGY Key Evaluation Questions The evaluation team used the following Scope of Work (SOW)10 questions to guide their analysis: 1. How effective is the partner’s approach to achieving results? 2. Do the program participants have the ability to utilize skills or information learned during the program to affect corruption and good governance in their home countries? 3. Do participants’ experiences in the program help to strengthen accountability systems and the U.S. government’s effort to enhance the lives of the people of Africa? 4. What is the (cost-benefit) value of this program to the objectives of the U.S. government and do the anti-corruption and good governance programs directly contribute to the goals of USAID and to the U.S. government’s interests in promoting accountable governance systems and the capacity to sustain the accountability systems in Africa as the Les Aspin center claims? 5. How has the partner built sustainability into the program for long-term impact? Could or should these sustainability efforts have been strengthened? Design and Methods Used The team used a mixed-methodology approach, combining quantitative survey data with qualitative in-depth interviews, focus groups, and program observation to strengthen the validity of the findings. This approach allowed the evaluators to obtain and document training results beyond the initial reports by participants about a training experience. The evaluation took place from October 2010 through November 2011. This allowed the team to examine the full program implementation cycle, including site visits to Marquette University in Milwaukee, Wisconsin, as well as to the Les Aspin Center in Washington D.C. and Accra, Ghana for the 2011 classes. Additional site visits to Accra and Nairobi, Kenya for the alumni refresher courses followed. The alumni refresher courses were conducted for the first time this year, at the suggestion of USAID/Washington. USAID suggested combining the East and West Africa programs into one program and then adding an “alumni refresher” course as the second program in 2011. According to USAID staff, the idea was that this would provide insight into what has been accomplished by the program (through its alumni) and also allow Les Aspin to reconnect with its participants.11 Specifically, the evaluators conducted the following data collection activities: 1. Desk Research/Background Review. This review includes the original grant agreement, activity progress reports, and annual reports; updates on Les Aspin alumni; training program itineraries; and lesson modules. Information pulled from these documents supports the background narrative as well 10 Full SOW found in Annex A 11 See Annex B for program schedules and agendas. 4 as the analysis of findings section of this report. A comprehensive list of the documents reviewed to date is attached in Annex C. 2. Electronic survey. The team developed two surveys, one for the Les Aspin alumni and the other for USAID Democracy and Governance (DG) staff that currently work, or have worked, on the program. The surveys were released on March 11, 2011. Of the twenty-nine USAID staff that received the survey, nine completed the entire survey, for a 31% response rate. Of the 229 Les Aspin alumni that received the electronic survey, seventy-two completed the entire survey for a 31.4% response rate (112 alumni completed at least part of the survey). Table 2: Alumni Survey Respondents by Country It is important to note that the number of participants figured in the table above represents the total number of participants. Unfortunately, we were not able to retrieve contact information for all participants over the last ten years. There is no standard database with participants’ current information, so we relied on the Les Aspin Center’s current contact lists; some participant information is not known to Les Aspin Center staff and/or some alumni live in areas with limited Internet availability. Therefore, a low response rate, such as the case with Uganda, may indicate that insufficient Ugandans received the survey. Survey questions can be found in Annex D. 3. Semi-structured personal interviews. Thirty-six individual interviews were conducted with Les Aspin Center and Marquette University staff, relevant USAID/Washington and Mission staff, program lecturers and anti-corruption experts.12 4. Focus-Group Discussions. Fourteen FGDs were conducted with both alumni from all six countries (refresher course participants and non-refresher course participants) in Ghana and Kenya, as well as 2011 program participants while in Milwaukee. The results of the FGDs provided qualitative data to supplement data on motivations for participation in the program and the results of the training collected via the survey. FGD guides can be found in Annex D. Interview guides can be found in Annex D. 5. Direct Observation. Observation of training sessions was carried out over five intervals in 2011: the first week session in Ghana, as part of the initial orientation for the East and West Africa participants in Accra in May 30–June 5; the one-week program in Milwaukee, Wisconsin, June 11–15; the two￾week program in Washington, D.C. June 20–August 1; the West Africa Alumni Refresher Program August 17–26; and the East Africa Alumni Refresher Program August 24–31.13 12 Seven with lecturers, thirteen with Les Aspin/Marquette staff, twelve with USAID staff, four with anti-corruption experts Structured 13 The refresher program was added in 2011 and is discussed further throughout this report. 0 10 20 30 40 50 60 70 Ghana Kenya Mali Nigeria Tanzania Uganda Number of participants Number of respondents 5 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report observation of the training, especially as it varies from one location to another and from regular program to refresher, allowed the evaluation team to gather data on adherence to curriculum structure, logistical implementation, and communication effectiveness of instructors. 6. Cost-effectiveness analysis. Through a detailed review of Les Aspin expenditures and annual outlays for fiscal years (FY) 2007, 2009, and 2010, we analyzed the programs cost-share structure, share of expenditure of relevant activities cost per beneficiary, and cost per beneficiary, per day. Threats to Validity While our team was able to observe the entire 2011 Les Aspin program, it should be noted that this year followed a format different than years past, described in the “design and methods used” section, above. We learned that the format of presentations and site visits did not change significantly, but it is possible that the combining of East and West African participants led to a slightly different experience than the previous nine years of single-region programs. Another methodological limitation is found in our survey. The survey respondents skew towards most recent graduates, with 58.9% of respondents from 2008, 2009, and 2010 classes. In addition, the survey over￾represents some countries, as shown in Table 2, above. The low response rate is a factor to be considered in assessing bias, and we try to point out when this may be the case. It is important to note, however, that we were able to overcome this bias in many of our findings by conducting focus group discussions with a more diverse group of alumni (from 2001–2011) during our field visits. During the refresher courses in Ghana and Kenya, and during the regular program in Ghana, the evaluation team spoke with alumni representatives from each year of the program. Much of this was due to the refresher-course-participant selection, which included representatives from almost every class from the last ten years of the program. These FGDs allowed us to explore some of the findings from the survey in more depth and gain additional perspectives on the issues. As explained in the discussion of cost benefit, we could not perform a rigorous cost-benefit analysis due to the lack of specifically targeted benefits, and the absence of any follow-up-evaluation metrics with regard to post-training outcomes and impact. Therefore, we were unable to assess the Les Aspin program’s quantitative value to the USG's good governance and anti-corruption program objectives. Our discussion of value is, by necessity, largely qualitative in nature. We have presented cost-effectiveness findings based on a comparison between the costs of the Les Aspin program and a similar, USAID-funded training program, the data for which was supplied to the team by USAID. A final limitation to note is the lack of a comparison group against which to measure the post-Les Aspin experience achievements by trainees. Individuals selected for the program are either already leaders or have high potential to become leaders, so their subsequent achievements may or may not be attributed to the Les Aspin program. Therefore, it is not possible to attribute impact beyond the self-reported attitudinal changes and reported personal skills. FINDINGS AND CONCLUSIONS This section is divided into the five main evaluation questions posed in the Scope of Work (SOW). Here, we describe the findings and conclusions associated with each question. General conclusions are found at the end of this document. 6 EFFECTIVENESS OF PARTNER’S APPROACH TO ACHIEVING RESULTS (SOW QUESTION 1) Findings The evaluation team defined the measurement of “effectiveness” as a higher-level impact that would result in a measured reduction in corruption and increase in good governance in the targeted African countries. This is based on the program’s overall objective to promote accountable governance systems and reduce corruption in Africa. For this higher-level definition, the evidence is too weak to conclude any link between the Les Aspin program activities and this result. Participant Selection Process In terms of program approach, interviews with USAID staff and anti-corruption specialists found that the selection criteria and composition of the participants was a strength. Participants are young (twenty-five to fifty-five years old); many are just beginning their careers and have the energy and idealism to apply the knowledge, practices and skills taught in the program. Gender equity, also a positive element, is applied consistently in the program. Another strong element in the program’s approach is the selection of participants from secondary cities and towns across the country, as opposed to focusing just on participants from the capital cities, with the exception of Mali. While the ratio varies by country, the selection of participants from outside of the capital helps to diversify the pool of participants away from the typical site of the capital city and supports individual capacity building at a decentralized level. Table 3: Participant Data 2001- 2011* KENYA TANZANIA UGANDA GHANA MALI NIGERIA TOTALS GENDER Male 29 22 21 33 23 31 159 Female 33 22 17 29 25 23 149 DISTRICT Capital 41 31 20 23 46 23 184 Other 21 13 18 39 2 31 124 SECTOR Civil Society 46 23 13 30 25 36 173 Government 10 16 21 24 15 12 98 Media/Private 6 5 4 8 8 6 37 *These figures are drawn from the list of participants given to us by the Les Aspin Center, and the sectors were deduced from participants’ organizations and job titles. The approach of mixing sectors between media, government and civil society, was seen as a unique element of the program. This was noted by the program’s anti-corruption experts and lecturers. Rather than working exclusively with civil society, the Les Aspin program takes a multi-sector approach, perceived to be an important element to addressing anti-corruption. Alumni themselves—particularly government representatives, who take strength from the civil society and media participants—noted that they found support in the diversity of the participants. Of note, in our survey of alumni, 21.3% said that they had participated in similar fellowships to study accountability and good governance in the U.S. or elsewhere. 7 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report In addition to the participants themselves, USAID/Washington interviewees involved with the programs agreed that one of the most effective aspects of the Les Aspin approach is the Center’s ability to secure Mission-level involvement in the participant-selection process. With the exception of Kenya, the Mission advertises the program and, along with program alumni, nominates participants based on selection criteria provided by the LAC. Overall, the impression at USAID/Washington is that the selection process is, as one key informant (KI) stated, “a great approach which ensures Mission support to the program.” Survey results illustrate a similar sentiment from USAID DG officers and Les Aspin alumni. Five out of nine USAID respondents said that they were involved in the participant recruitment or selection process. Again, with the exception of Kenya, this was confirmed in our individual interviews with DG officers at participating Missions. One hundred percent of those who reported being involved with the selection believe that the recruitment process targets appropriate candidates. They explained that the USAID Missions go to great lengths to select good candidates, and that these candidates come from a wide range of backgrounds, representing “both the supply and demand side of accountability,” as one respondent wrote. This perspective on effective selection of appropriate participants seems fairly consistent with the alumni responses. Individual interviews with USAID Mission staff reiterated this sentiment and confirmed that USAID Mission staff is in control of the selection process for participants, with the exception of Kenya. The Kenya Mission noted that they are not involved in participant selection. Instead, the Les Aspin Center staff is directly in control of participant identification and recruitment. Advertisements are placed in local Kenyan papers and Les Aspin staff travel to Nairobi to personally conduct the interviews. Interviews with LAC staff in Washington confirmed that they play a more direct role in the selection of the Kenya participants, due to potential concern over favoritism amongst USAID Mission staff towards the staff involved in their own projects and initiatives. During interviews with the Kenyan Mission staff, a former DG officer involved with the original Les Aspin program beginning in 1996 and the current officer connected with the Les Aspin program, confirmed that the Mission is only involved in logistics (i.e.: helping participants get visas). They originally were involved in the selection process and said that the Mission was informed by the LAC, possibly as early as 2003, that the Center would be conducting the participant selection on their own.14 Our interviewees reported that the Mission’s DG officers found this acceptable because the process took up a lot of time and “the program wasn’t in line with [their] DG work” in Kenya. The former DG officer we interviewed admitted that many of the DG officers had their own agenda, behind which participants were selected (i.e., candidates who worked on projects of interest to the DG officers), and this may be why the Center elected to cease working with the Mission in the selection process. However, our interviewee said that, as far as he knew, the DG officers were always selecting participants across sectors, age and the “likelihood to contribute.” Both USAID/Kenya interviewees said that they would like to be involved in the process and they feel their involvement would also lead to more involvement on the part of the Mission with participant follow-up. Program content and pedagogy Focus group discussions noted that the program did not allow for the “contextualization” of the systems and approaches they learned in the U.S. They felt that the program provides a plethora of new information and ideas, but some felt far from their reality. The participants would have appreciated time to study a comparative analysis between the U.S. and Africa in order to strategize about effective approaches to anti￾corruption interventions. The survey results reinforce this: 31.3% of alumni respondents think that the Africa-based segment could be improved. Suggestions included incorporating more site visits to observe new 14 The Kenya Mission interviewees could not remember the exact year. The Les Aspin interviewees said that it was earlier (circa 2003) that Kenya stopped being involved in the selection process. 8 approaches to transparency and anti-corruption and revising program content to include case studies, comparative exercises, and current events specifically tailored to the African context. It is important to note that the Les Aspin program eliminated a post-U.S., one-week period in Africa in 2006 due to funding constraints. Direct observation of the program and input from focus groups noted the program design’s limited flexibility, which restricted the Center’s ability to incorporate contemporary political events into the program. The team directly observed this during the Wisconsin program, which made no reference to, or example of, the public backlash to the State’s controversy over limiting labor union’s ability to conduct collective bargaining. In fact, the Madison portion of the program, which typically looks at State-level governance systems, was cancelled due to the crisis. This was a lost opportunity to present real-life instruction in U.S. political process, advocacy, and lobbying activities. Alumni in West Africa also noted that they would like the program to address contemporary issues, such as corruption and trafficking of humans and narcotics across borders. Alumni from Mali, in particular, noted that this is a serious contemporary issue they face and that they seek guidance on how to combat corruption in the customs and immigration sector. While some time was spent in Ghana on presentations by various anti-corruption groups, there was limited time in Washington for participants to meet with international and African organizations active in good governance and anti-corruption issues. The majority of USG meetings were focused on U.S. systems of good governance and anti-corruption (i.e., U.S. Chamber of Commerce, SEC, the Federal Elections Commission [FEC], the U.S. State Department, the Office of General Counsel of the U.S. Department of Justice, the Federal Elections Administration, and the Office of Government Ethics). Meetings, or setting up affiliations, with international and African-based good governance and/or anti-corruption groups like the National Endowment for Democracy (NED), National Democratic Institute (NDI), International Republican Institute (IRI), Transparency International (TI), and Freedom House would provide direct working links with these organizations, which can help support initiatives once participants return to their home countries. Direct observation of the program also found limited links with other USG activities focused on anti-corruption and good governance issues internationally or in Africa. The data in Table 1 in the background section above, along with direct observation and feedback from program participants, shows that many of the presentations were not participatory in nature and not entirely relevant to the good governance objectives of the grant. For example, the majority of the program in the U.S. focuses on U.S. political systems. There is limited exposure to international comparative analysis to allow the participants to contextualize what they are seeing and learning. While many of the presentations, particularly in the Africa segment the program and the alumni refresher, were described as “participatory,” they typically followed a standard presentation format, with questions and answers at the end. This approach does not follow standard training best practices, which include individual and group exercises to support the learning of concepts. Across all focus group discussions, participants found site visits in the U.S. to be the strongest element of the program. The opportunity to directly observe and discuss systems and approaches with real-world professionals was the highlight of the program. Direct observation and discussion with practitioners provided participants motivation for the ways things can be and the relationship developed with the practitioner, at “For me, I think generally the visits to the U.S. institutions are key, because it gives us. . . a better understanding of how they talk about these institutions over there, how they work and it makes you understand that they are not things that are too abstract and cannot be done here for me that was one of the things that I realized. And there are simple, simple things that can be done or you can use for advocacy. For me that’s what helped me.” —Ghana Les Aspin alumni focus group participant 9 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report times—based on input during the focus group discussions—resulted in on-going communication and support to implement new approaches upon their return home. The Washington, D.C. and Milwaukee site visits, meetings with U.S. accountability and governance institutions, and the conflict-resolution workshop in Milwaukee are rated as “more effective” by participants. Participants from the 2011 program, which combined East and West Africa participants, noted as well that they enjoyed learning from each other about each participating country’s initiatives in the area of anti-corruption and good governance. Post Training Evaluation and Follow-up A recurring area of weakness in approach raised by USAID staff in Washington and in Missions is the limited follow-up once participants return home. Alumni themselves noted that they have scant knowledge of subsequent and previous alumni and would appreciate a systematic way to connect with new participants and alumni. Limited follow-up not only reduces the level of accountability for alumni once they return to their home countries, but also weakens the synergy that can be developed among alumni and other USAID anti￾corruption initiatives. The lack of adequate, structured follow-up with alumni was cited by USAID interviewees and survey respondents as the major shortcoming of the Les Aspin program design. Les Aspin staff keeps in touch with some alumni, but on an informal basis. Les Aspin staff acknowledged this, and added that instituting more structured follow-up would require additional funds.15 The lack of follow-up, particularly in an age of increased social networking, also raised questions about the ability, or willingness, of the program to innovate or change over time. No USAID interviewee could describe specific innovations or significant changes to the program design since 2001. One of the largest changes in the program—the incorporation of East and West Africa participants into a single program experience and the development of an alumni refresher program—was requested by USAID for the 2011 program year. The Les Aspin staff did explain that one of the most popular modules in the program, led by the Dispute Resolution Center at Marquette, has now become two full￾day sessions during the weeklong session at Marquette University because of positive feedback from participants. However, the Les Aspin staff did question the need for significant change over time as they feel, in comparison to other programs, the Les Aspin program is a strong model of experiential learning. Built-in capacity or tools to evaluate the program is also limited, thereby making evidenced-based changes to the program over time difficult. There is no official USAID Performance Management Plan (PMP) for the Les Aspin grant, and anecdotal information is collected on anti￾corruption programs and activities led by Les Aspin alumni and reported to USAID. As the objectives of the program have grown and changed over the course of its development, so has the level of performance measurement. In 2009, following the development of the common “F Indicators,” Les Aspin was asked by USAID to begin tracking F indicators for their program results. The 15 Please see the sustainability section for further discussion on participant follow-up. “If we have an e-discussion this is something we can share with people so that people know that we are here, we are also “Les Aspinites,” they are not the only people that are there. It’s only when we met at USAID before we came here that I saw those people; I was always looking for their addresses when we came back, we wanted to have all the Les Aspinites together and have a meeting but I’ve never had the opportunity, I’ve never had their contacts so I cannot call them. . . for me this is something we really need from them (Les Aspin Center)” —Les Aspin Alumni Mali 10 required indicators that USAID must report to Congress on this activity are output indicators on the number of government officials trained, broken down by gender, and number of civil society groups trained (See Annex E for indicator table provided by the Les Aspin staff). In addition to this required information, Les Aspin, from the inception of its program, has maintained informal contact with some of its fellows and attempts to track their progress. In addition to this anecdotal information, Les Aspin staff conduct a feedback session at the end of the U.S. program, using the appreciative inquiry (AI) method to assess participant impression of the program. No systematic, written evaluation gathering input from alumni, USAID or lecturers is conducted of the program. AI is a useful tool for evaluating programs such as Les Aspin’s, but we recommend this as a tool to use in conjunction with other qualitative and quantitative data collection methods. It is difficult for people to speak openly in a critical manner about the program during group feedback sessions. In order to secure objective and honest responses on the program, facilitators, facilities, schedule, etc., anonymous, written feedback questionnaires would be very helpful. In addition, questionnaires do help to quantify feedback and results, which is important to USAID, particularly in light of USAID’s new Evaluation Policy that requires higher-level evaluation and M&E rigor. It would also assist the Les Aspin staff in making more evidence-based decisions with regards to programming.16 Program Cost Effectiveness Measuring Costs and Benefits: A Caveat To measure costs and benefits, the benefits should be quantified using some metric or standard indicator. For many reasons, there are no metrics or quantifiable indicators for outputs or outcomes (in the methodological approach familiar to evaluators) of the Les Aspin program. As discussed, end-of-training evaluations were not undertaken after seminars ended, nor has there been any ex-post-evaluative effort to collect outcome indicators to benchmark against earlier, established targets. More broadly, measuring impact of educational or training programs, even in highly targeted and specific work environments, has proved elusive, short of using sophisticated experimental methods, such as randomized controlled trials (RCT). Furthermore, dissecting the budget cost-items in terms of program objectives is not possible, given the structure of the program. For example, Les Aspin’s overall program structure does not deliver a module on corruption in one country, nor is there a specific week or set of days in the program dedicated to discussing democracy promotion, as opposed to democratic theory and U.S. experience. Therefore, very few distinct activities can be identified, in terms of intended output, to which a cost figure can be applied. This section of the report provides an analysis of Les Aspin’s 2007, 2009, and 2010 fiscal budgets and of cost per participant and cost per day, per participant. Les Aspin’s Budget Expenditure Analysis The USAID grant to support the Les Aspin training program has remained relatively constant at $500,000 each year from 2001 to 2011, the period for this evaluation. Based on FYs 2007, 2009, and 2010 expenditure 16 Les Aspin staff responded to this by saying, “Due to financial resource constraints, we have as yet been unable to do a more comprehensive post-program review of the trainings, where this will require travel to the respective program countries to do on the ground assessments. Thus, we have relied on participant self-reports, which they send to the Les Aspin Center, and which we also capture through contact with participants. Through these report narratives, graduates explain how they have used or are using the skills gained at the Les Aspin Center back in their respective countries. These narratives outline work in progress, work achieved, and work planned but awaiting execution with the acquisition of further financial and other support. In line with the training framework, program deliverables, and structured results matrix for determining program outcome/impact indicators, we consider the self-reports of graduate accomplishments to meet the requirements of expected indicators of program success (in terms of whether immediate change -impact- is resulting now or in the long term at a future time).” 11 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report data, Marquette University has contributed an average of $66,422.30 each year (an average institutional contribution of 11.73% of the program’s cost). Table 4, below, illustrates the major expenditure line items and cost-shares of the Les Aspin revised fiscal year expenditures for FY2007, FY2009, and FY2010 submitted to the evaluation team in September 2011 which displayed figures different from those reviewed during the interim evaluation period.17 Table 4: Les Aspin’s expenditure analysis (FY 2007, 2009, 2010) AGGREGATED EXPENDITURE LINE ITEM FY 2007 FY 2009 FY 2010 Salary and Benefits $ 104,645.40 $ 115,579.62 $ 92,375.11 Program Supplies / Administrative Expenses $ 5,972.57 $ 10,841.41 $ 7,534.98 Staff Travel $ 27,494.65 $ 29,849.75 $ 12,146.47 Participant Travel $ 75,324.29 $ 83,156.76 $ 78,594.98 Hotel and Per Diem $ 173,565.77 $ 188,461.30 $ 196,725.77 Other Costs $ 57,117.54 $ 66,487.28 $ 33,533.51 Marquette Fringe Cost $ 91,041.86 $ 96,474.49 $ 88,404.20 TOTAL PROJECT COST $ 535,162.08 $ 590,850.61 $ 509,315.02 Accounted Marquette Cost-Share + $ 54,307.84 + $ 92,399.06 + $ 52,559.99 USAID Award Expenditure18 + $ 480,854.24 + $ 498,451.55 + $ 456,755.03 The cost breakouts show incremental changes over the three-year fiscal expenditures submitted. Most of these changes are understandable, given changing prices and currency exchange in host countries. Hotel and per diem expenditures show a steady increase, while other line items reflect the Les Aspin Center’s efforts to control costs. Staff travel, for example, was cut by more than half in 2010. In addition, in 2006, Les Aspin switched the program from six weeks long to five weeks. Although travel, lodging, and salary and benefits remain the biggest expenditures, the expenditure report disaggregates travel between staff and students. Student travel, per diem of all students, as well as student travel and hotel accommodation, are listed as separate budget items. Re-aggregating these numbers, we find that the travel and lodging costs (for both students and staff) accounts for 52% of the total budget ($265,000)—a number that continues to be quite significant, as further cost savings would likely come from this line item. This would require either reducing the number of students, or additional cuts in time spent in one of the U.S. locations. Although the exact cost figures differ between FY 2007–2010, the majority share of the budget expenditure was still held by these big￾ticket items. 17 The SI team made several requests to the LAC for actual program expenditure data over the eleven-month course of this evaluation. Actual expenditure data was not received until September 2011, although a 2010 Budget Estimate was received in March 2011(See Annex F). 18 Although it is outside the scope of this evaluation to perform an accounting audit of Les Aspin’s finances, it should be noted that the latest iteration of budgets submitted in September, 2011 does not account for the entire USAID award granted ($500,000). This was not the case in the previous iteration of the budget submitted, where the entire amount was listed as an accrued cost. Furthermore, third-party, cost-share (e.g. various USAID mission support is not appropriately labeled. 12 Charts 1, 2 and 3, below, illustrate the information in Table 4 as percent proportions of each fiscal year’s total costs. Chart 1: Proportion of Major Cost Items in FY2007 Chart 2: Proportion of Major Cost Items in FY2009 Salary and Benefits 20% Program Supplies /Administrative Expenses 1% Staff Travel 5% Student Travel 14% Hotel and Per Diem 32% Other Costs 11% Marquette Fringe Cost 17% FY 2007 Total Budget Expenditure: $ 535,162 Salary and Benefits 20% Program Supplies / Administrative Expenses 2% Staff Travel 5% Student Travel 14% Hotel and Per Diem 32% Other Costs 11% Marquette Fringe Cost 16% FY 2009 Total Budget Expenditure: $ 590,850.61 13 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report Chart 3: Proportion of Major Cost Items in FY2010 As shown by the three charts above, the proportion of expenditure costs for administrative and programmatic expenses has remained largely constant across all years. However, the total costs per each year did change. In other words, the slices of the pie proportionally grew and shrunk relative to the size of the pie overall. The three fiscal years studied do not show any significant shift in costs toward a certain activity or an expenditure line item. However, the Les Aspin overall program expenditure in FY2009 shows a marked increase from the total FY2007 levels. This is followed by a significant decline in 2010 (at levels well below those in 2007). These changes occurred while maintaining (1) the same percentage share of student travel and hotel costs in East and West Africa and the U.S.; and (2) the same number of participants. The ability to sustain a relatively constant share of expenses on travel and lodging of students is due in part to reduction in staff travel (or at least in total cost of travel as evidenced in Charts 2 and 3). In 2010, staff travel dropped from 5% to 2% of the expenditure. Therefore, some reallocation of resources between expenditure items may explain part of the story. However, interviews with African participants suggest that a major cost-savings decision was to require trainees to share rooms during the off-site training periods, a measure which did not sit well with many of the participants. Cost Effectiveness: Comparison with Another USAID-funded Foreign Exchange Educational Program19 To determine cost effectiveness, we have taken a comparative approach. USAID provided the evaluation team with a detailed budget of a comparable program without identifying the responsible implementing partner grantee. Upon review of the comparison program’s budget (which is much larger in budget and scope of work than that of Les Aspin’s), one component of the program—a training seminar—was selected for cost-comparison. 19 This comparison budget is to remain anonymous; therefore we have not provided a copy of this budget in the report. We have extracted relevant expenditure data from the comparison cost figures for our analysis. Salary and Benefits 18% Program Supplies / Administrative Expenses 2% Staff Travel 2% Student Travel 15% Hotel and Per Diem 39% Other Costs 7% Marquette Fringe Cost 17% FY 2010 Total Budget Expenditure: $ 509,315.02 14 Both training programs (Les Aspin and the training seminar) were carried out by institutes within their respective universities in the United States. Both universities are located in geographies where the costs of lodging and per diem are similar. Furthermore, both training programs were normalized in the analysis for number of participants and number of training days. Les Aspin’s has 976 participant program days (30.5 days x 32 participants) while the comparison program has 126 participant program days (21 days x 6 participants). Both programs budgets’ line items were also aggregated to allow for a more uniform comparison of their respective costs. However, there are fundamental differences between the two programs; these play a part in the structural differences in costs, which are explained below in a line-by-line budget comparison. Broadly, the vast majority of the USAID comparison program’s work was conducted in the home country. Furthermore, the isolated component of the comparison budget (i.e., the training seminar) is integrated into a larger set of deliverables expected of the program. Therefore, many of the activities aim to develop curricula with individuals who represented partner organizations in the field and play other roles in the implementation of the work down the line. Therefore, it is expected that they would require significant consultant and staff support in their work while training in the United States. Les Aspin, on the other hand, focuses more on the individual development of participants and does not require a specific set of deliverables at the conclusion of the training in the United States. Furthermore, Les Aspin incurs extra costs by having participants stay in Washington, D.C. before continuing their travels to Wisconsin, whereas the comparison program does not accrue the costs associated with this intermediary step. Finally, it is important to note that the USAID comparison program is a budget for Fiscal Year 2007 spending, while Les Aspin’s has presented an expenditure report for FY 2007. Therefore, it may be the case that some of the estimated costs for the comparison program shifted in implementation. Table 5, below, highlights and analyses the major cost-items of each expenditure and budget sheets received from the two programs. 15 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report Table 5: Amount, Percent of Expenditure, and Cost Per Participant for Les Aspin and USAID Comparison Program FY 2007 LES ASPIN EXPENDITURE COMPARISON PROGRAM BUDGET USAID COMPARISON PROGRAM20 LES ASPIN Amount ($ U.S.) Share of Total Cost Cost per participant per day ($ U.S.) Amount ($ U.S.) Share of Total Cost Cost per participant per day ($ U.S.) (1) Salaries & Benefits 12,306.46 20.53% 97.67 104,645.40 19.55% 107.22 (2) General Administrative Costs 13,250 22.10% 105.16 5,972.57 1.12% 6.12 (3) Consulting 3,000 5.00% 23.81 9,600.00 1.79% 9.84 (4) Ground Transportation 600 1.00% 4.76 18,401.98 3.44% 18.85 (5) Insurance 900 1.50% 7.14 2,232.00 0.42% 2.29 Total Admin. Costs 17,750 29.61% 140.87 36,206.55 6.77% 37.10 (7) Student Travel to U.S. 7,200 12.01% 57.14 75,324.29 14.08% 77.18 (8) Student Lodging in Africa N/A N/A 46,264.64 8.64% 47.40 (9) Participant Lodging in U.S. (not Washington D.C.) 12,600 21.02% 100.00 72,356.13 13.52% – (10) Participant Lodging in Washington D.C. N/A N/A 59,538.00 11.13% – (11) Per Diem (includes Washington D.C. if applicable) included in U.S. lodging cost N/A 54,945.00 10.27% 56.30 (12) University Fringe Cost 144,251.97 21.80% 1,144.86 91,041.86 17.01% 93.28 (13) TOTAL COSTS $59,480.77 – $ 475.83 $535,162.08 – $ 548.32 20 The comparison program includes multiple components. Therefore, budget line items were adjusted to reflect the costs associated with the training seminar component of the comparison program. Line items #1 and #13 are adjusted accordingly. Refer to Footnote 20 for further explanation of pro-rated adjustment of costs. Line item #12 is reflected of both programs’ true total costs. 16 As Table 5 shows, both programs are comparable in cost. The cost per participant, per day for the USAID comparison program is $475.83. The Les Aspin’s program, on the other hand, costs $548.32 per participant, per day. As discussed above, the comparison program activities go beyond the training provided. They include curriculum development and training in the home country following the training seminar in the U.S. To that extent, the total costs (line item #13) in Table 5 do not represent the total cost of the comparison program, but rather the relevant portions of the budget that deal with the comparison budget’s training module. This includes line items #2 through #11, as well as the proportion of the salaries, benefits, and fringe costs associated with the training seminar component.21 As for each line item, “Salaries and Benefits” (line item #1) shows that the share of the budget on salaries and benefit for both programs is very comparable, with negligible difference. This is further re-enforced by the small difference of salary and benefits costs per participant, per day. Although it should be noted that economies of scale suggest that cost per participant, per day should decrease for a program of longer duration. This is not the case, however, as Les Aspin’s costs about $10 per participant-day in salaries and benefit. Line item #2 (“General Administrative Costs”) bundles expenses such as telecommunication costs, office supplies, administrative support, etc. The general administrative costs here are significantly higher for the USAID comparison program, which may be due in large part to the fact that a local NGO was hired and has contributed to supporting the training program, as well as other program components. The local NGO also provided logistical support for development of curriculums that may have required communication beyond the training period. This also appears to be the case for line item #3 (“Consulting”). Here, the USAID comparison program is also higher in cost than Les Aspin. This may be attributed to the fact that specialized consultants were contracted to assist in the development of curriculum program for the host country. On the other hand, ground transportation costs (line item #4) for Les Aspin are higher than those of the comparison program. Again, a possible explanation lies is the fact that many Les Aspin participants live outside the main training locations of Nairobi and Accra, while the USAID-presented comparison program took place in the same country. Turning to travel to the U.S. (item #7): the cost per participant in the comparison program is $1200 per person, while Les Aspin’s travel cost fares are much higher ($2353.88).22 Lodging in Africa (item #8) does not appear as a cost in the comparison expenditure, mainly because the training in the United States is the only component found to be most similar to Les Aspin. On the ground for the comparison program, the implementing partner's sub-contract with a local NGO has deliverables and costs that differ significantly from those of Les Aspin and are therefore excluded from this analysis. Les Aspin’s program participants, on the other hand, come to one central location, such as Nairobi (where travel and lodging expenses are accrued), for the first phase of the training and then are flown to two locations in the United States for subsequent segments of the training. The average lodging cost per participant in Africa is $47.40 per day, for a total of $46,264.64 in FY 2007. This constitutes 8.64% percent of the Fiscal Year outlay.23 21 The “Salaries & Benefits” line item for the comparison program is calculated by dividing the total cost of salaries and benefits in FY2007 by the estimated number of federal working days (250 days), then multiplying them by days spent on the training program. As a conservative estimate, the level of effort for the training program of the comparison budget is twenty-one days. Number of working days was retrieved from the Office of Personnel Management 2007 Calendar. Source: http://www.opm.gov/operating_status_schedules/fedhol/2007.asp There are also fringe costs added to the total costs in line item #13. For the comparison program, the fringe costs are adjusted to reflect the proportion that deals with the training program. 22 We lack enough information to adequately explain the high costs of travel for the Les Aspin program. However, it may be a consequence of airline pricing structures or other factors, such as travel agencies utilized. For the comparison program, the lower price can be explained by the departure location, which has a heavier flight volume than the departure locations of the Les Aspin program. 23 The cost comparison above also presents an isolated financial account of lodging in Washington D.C. Again, an activity that does not have a specific output link to it remains a costly part of the budget. At $59,538, hotel lodging in Washington, D.C. is 11.13% of the Les Aspin Budget expenditure (line item #10). The comparison budget, as stated earlier, is limited to two geographies and does not have this intermediary component. 17 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report In terms of Les Aspin’s final training location, the two mid-western locations of Les Aspin’s and the comparison program do in fact differ in lodging and per diem costs as sanctioned by the U.S. government.24 The comparison program offers lodging and per diem for its students at $100 per participant per day. Therefore, the comparison program is $23 below the total amount listed by the U.S. Government Services Agency (GSA) per person for lodging, meals and incidental expenses.25 Meanwhile, Les Aspin’s lodging costs are roughly $10 less per person than the amount listed by the GSA ($89). However, the per diem in the U.S. is $56.30 per participant per day (which is roughly comparable to the GSA meals and incidentals expenses amount listed by GSA). Although Les Aspin travel, lodging, and per diem costs are evidently above those of the comparison program, the trade-off (at least when compared with the USAID sample comparison budget) seems to be the result of three expenditure line item differences. First, the management costs of Les Aspin’s program are less than those of the comparison expenditure (Table 5, line item #6). This could be due to the different management responsibilities for each program. Furthermore, the biggest cost-difference between the two programs appears outside the direct cost items. Les Aspin has a lower fringe cost. As line item #12 shows in Table 5, the comparison expenditure’s fringe cost is 21.8% of the total costs. This is compared to Les Aspin's fringe costs of 17.01%. Finally, Les Aspin offers a greater cost-share than the comparison program. Unit Cost Accounting Isolating travel costs (which includes airfare and lodging), we find that each participant on average incurs $8,087.50 for travel costs alone. This is similar across the other years examined. In other words, each participant’s travel and lodging accounts for about 1.5% of the entire outlay. When normalizing the Les Aspin program’s cost by day in FY 2010, Les Aspin spent $561.49 per participant per day.26 This participant per-day cost is quite low. Although not a completely valid comparison, the unit cost for the USAID comparison program is $5,260 per day.27 USAID pays for 91% of the Les Aspin expenditure. Measuring the total cost to USAID for each participant for thirty and one-half days of training, the USAID grant supported costs of $15,625.00 per participant. Effectiveness Conclusions 1. Inadequate monitoring and evaluation data makes it difficult to impossible to assess adequately the effectiveness and impact of the program and its participants. 2. The program’s standard presentation approach, with Q&A at the end and limiting the use of current training approaches (i.e., individual and group exercises) reduces the potential for learning. 24 A survey of the Government Services Administration (GSA) Per Diem schedule for shows lodging in Madison, Wisconsin at a maximum of $89, with maximum $56 additional dollars for meals and incidental expenses. The comparison program’s geography offers $77 for lodging and a maximum of $56 for meals and incidental expenses. Source: U.S. Government Services Administration CONUS Per Diem rates. Accessed from: http://www.gsa.gov/portal/category/21287 25 The evaluation team was able to locate the comparison program’s geography and use the GSA accordingly. However, geographical information may identify the program and remains undisclosed in this report. In addition, the comparison program took place in 2008, which may impact differences in the costs. 26 Cost per participant, per day is calculated as a ratio of total budget divided by number of participants, and then divided again by program days. Designated free days on the schedule are counted as program days—all partial days are accounted for as 0.5 days. 27 It may be argued that higher participant per-day costs are justified in a "train the trainers" program because of the assumed multiplier effect when trainers return to the work places. However, the difference in unit costs is remarkable between the two programs. 18 3. The program has limited flexibility in its ability to incorporate contemporary political realities and current international trends in good governance and anti-corruption programming. 4. Inconsistent development of participant action plans reduces the potential result of the program as well as the level of accountability for participants. 5. While USAID Missions are heavily involved in the recruitment of participants, there is no follow-up and limited involvement in participant action plans or initiatives. 6. The program has weak links to international comparative analysts and international NGOs involved in USG initiatives for good governance and anti-corruption, which reduces potential synergy. 7. The limited sharing of alumni contact information reduces the ability of alumni to work together and mobilize as a group. Cost Specific 1. Calculating the benefit-cost ratio for training programs is always problematic. In the Les Aspin Center program, the problem is made more difficult by the lack of specific output metrics of learning objectives for students. This is complicated further by the fact that activities (sessions, site-visits, seminars, and geographies chosen to conduct the seminar) are not associated to specific outputs. Moreover, the lack of any systematic end-of-training or ex-post-evaluation evidence regarding alumni achievements severely limits the ability to measure evaluation results to unit costs. 2. The Les Aspin program has been cost effective, especially with regard to keeping unit costs and administrative costs at a reasonably low level. This was found to be true when compared to the USAID anonymous budget submitted to the evaluation team. 3. Shuffling the expenditure outlays does not tell the whole story. The USAID grant has remained at a constant level over ten years, forcing Marquette University to adjust the program structurally to meet rising costs, especially with regard to travel, per diem and time spent in Africa. These represent a significant share of the overall program outlays and efforts to reduce costs have led to participants having to share rooms. This room-sharing has resulted in the major complaint by trainees, many of whom are mature professionals. Other than the room-sharing scheme, a triangulation of interviews and budget analysis shows no evidence that Les Aspin’s leadership has undertaken review and internal evaluation of the effectiveness of various phases of the program, with the possibility of consolidating or eliminating less productive and/or more expensive parts of the program that may yield little results. This is due partially to the fact that desired outputs are not linked to specific activities undertaken, nor are they properly defined. Little evidence was found to suggest if outputs for the Washington, D.C. portion of the program can be isolated for evaluation. Furthermore, if those identified activities in Washington D.C. or Madison, Wisconsin do contribute specifically to helping Les Aspin arrive at desired goals, are they doing so in the most cost-effective manner? 4. More holistically, however, the issue of significant budget expenditure on travel, per diem, and lodging contradicts the limited funding dilemma of the program. Emphasis on maintaining the three phases of the program is directly linked to a sizable portion of the budget being spent on travel and limiting spending on other program activities. 19 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report PARTICIPANTS’ ABILITY TO UTILIZE SKILLS OR INFORMATION LEARNED DURING THE PROGRAM (SOW QUESTION 2) Findings USAID views None of the USAID interviewees knew whether the program participants are able to utilize the skills or information learned during the program to affect corruption and good governance in their home countries or organizations. This was due in large part to the lack of follow-up in the program. Interviews with Les Aspin staff showed they are confident that the participants are able to use the information effectively, and described participant profiles, which illustrate these results. The LAC has anecdotal updates on what participants are doing, but they do not clearly link the skills and knowledge taught in the program with what the participants have accomplished in their home countries. The participants are chosen based on their current or future ability to affect change in their home country, so there seems to be an assumption that they will be able to utilize their skills. However, the utilization of knowledge and skills is discussed anecdotally, but not measured or evaluated formally. Participant Survey Results The electronic survey of alumni showed that ninety-four of the ninety-five respondents think that the program is applicable to their work environment.28 As shown in the chart below, in response to the question, “Do any of the following skills and information that you obtained through the Les Aspin program apply to your daily work?” 90.8% marked ethics-based decision making and 85.1% marked leadership skills. Coalition building seemed to be the least relevant skill, with only 64.4% saying that it applied to their daily work. Two-thirds of alumni respondents said that additional skills would have been helpful to learn, including fundraising and resource mobilization; project, financial, and personnel management; communication; lobbying; networking; and policy development. Meeting these skill needs would require a very different kind of program, focused more on institutional-capacity development and management. Many of these themes, 28 Since this was a question that came relatively early in the survey, the number of respondents is greater than the number of alumni who completed the entire survey. 84.30% 65.20% 91% 85.40% 71.90% 0 22% Negiotiation and Dispute Mgmt Coalition Building Ethical based decision Making Leadership Skills Effective Public Mgmt None of the Above Other Chart 4: Applying learned skills Do any of the following skills and info that you obtained through the Les Aspin Program apply to your daily work? (Select all that apply) N=89 20 particularly lobbying, networking, and communication, were touched upon during the program, but alumni regretted that they were not able to pursue them in more depth, due to the short duration of the program. Focus Group Findings Focus group discussions proved helpful in further clarifying the most relevant skills learned during the program and in providing specific examples of how they have been applied upon participants’ return to their home districts. It was difficult for Les Aspin alumni to discuss immediately the skills and knowledge gained through the program and how they have applied it to their work. It is clear that the Les Aspin alumni are a dynamic group of individuals already engaged in inspirational, ethical work in their own sectors.29 The key element in assessing the effect of the Les Aspin program on the participants is to define clearly the skills and knowledge they gained from the program to implement new activities and approaches in their home countries. Many offered stories of personal motivation following the program to run for elected office or to develop civic education programs in schools. The program clearly motivates and inspires participants. Focus group participants noted that the actual visits—to offices such as the GAO and Government Ethics to watch their work and hear first-hand about how the offices function and interact with the USG—provided participants with information about how these systems work. It also inspired them to think of ways to adapt these systems to their own national structures. Despite this positive response, many Les Aspin alumni could not pinpoint specific skills or knowledge gained from the Les Aspin program that supported their ability to conduct follow-up activities in their own countries. However, when probed for more information, they were able to highlight specific knowledge and skills they learned that have had an impact on their efforts in good governance and anti-corruption. The skills and knowledge noted by alumni in focus groups that were most useful included effective communication; negotiation skills; ethics￾based decision-making; strategic planning; and lobbying. Participants felt that the portion of the program focused on negotiation and dispute management provided at Marquette University was one of the strongest elements of the program. It taught them how to enter into a dialogue with potential adversaries to resolve contentious issues. This approach linked to their ability to be effective communicators and to see communication as a two-way discussion, allowing others to voice an adversarial opinion without appearing to be weak. Many of the Les Aspin alumni have subsequently gone on to be appointed in leadership positions, such as district chief executive, promotions which alumni credited to their ability to provide objective facilitation of disputes. The success stories, developed by the evaluation team and included in this report (Annex G), provide more insight into the ways 29 Please refer to success stories documented in Annex G. “You can only brighten the corner where you are and hope that someone would also appreciate it and take it from there and continue.” —Les Aspin alumni and Judge Ghana The day we had the session about moral courage, it totally changed my life. In my upbringing as a girl, I’m not supposed to talk too much. . . . Sometimes I was confronted with issues of moral courage and I want to denounce these things, but can I? And after this training I now tell you the moral courage book is now my pillow at home. And I’m implementing it. Anytime I need to talk I say sorry but I have to talk, to tell it out-loud, I will tell.” —Les Aspin alumni Mali 21 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report in which the knowledge gained from the Les Aspin program have been adapted and used to effect change in alumni home countries. Ethics-based decision-making was a major element of the Les Aspin program that alumni considered to be of critical support to them and that they use in their work on a daily basis. This was particularly true for alumni coming from government positions who face corruption on a daily basis, in the form of bribes and personal favors. Issues The greatest weaknesses related to skills learned through the program and their application lies in the development of each presentation, which, across the program, is uneven in quality and does not always link directly to the issue of good governance and anti-corruption. There are no lesson plans with stated objectives for what each participant should know at the end of each presentation, and the presentations are more knowledge-based, informing participants of systems and concepts, as opposed to skills-based. The program sessions, in addition to being primarily lecture-style, also covered many topics, which allow the participants to study issues only superficially, as opposed to drilling deeper into their understanding of priority topics. The certificates provided to participants at the end of the program, cite over fifteen topics that the participants have studied. There was a sense from alumni that the program covered too many topics, with not enough focus. The evaluation team also observed this. From discussions with presenters of the program, the team determined that the Les Aspin program offers only limited guidance on the structure and topic of presentations. The focus group participants noted that it did not seem as though the presenters had any terms of reference for their presentations. The majority of presenters has been working with the program multiple years, and there is a heavy dependency in Africa upon using alumni as presenters, even though they are not formal trainers. The amount of time provided for presentations also does not allow for skills-based training, outside the strategic planning workshop in Ghana and the negotiation training in Milwaukee. In addition, the fact that no systematic evaluation is conducted for each portion of the program to assess learners’ perceptions and knowledge gained—nor is any formal follow-up tracking of alumni performed to monitor how the skills and knowledge of the program have been used in their home environments—is a major weakness of the program. Participants’ Ability to Utilize Skills or Information Learned During the Program Conclusions 1. There are clear links between some of the knowledge and skills learned in the program and the in-country, post-program activities of alumni. The skills and knowledge cited as the most useful are negotiation skills; effective communication; ethics-based decision making (moral courage); strategic planning; and lobbying/advocacy. It is likely that the participants apply these skills within the work environment, with the possible exception of lobbying/advocacy. 2. There is an intrinsic weakness in the Les Aspin Center’s management of the program, due to the lack of feedback from, and formal evaluation by, participants on the program. Anecdotal stories do not focus on the skills and knowledge learned from the program and therefore show weak causal links to overall program impact. “The problem that I notice from here in Ghana they have discuss about eleven thematic topics, Washington we had eighteen thematic, at Wisconsin, Milwaukie and Madison we had eight thematic. . . that was too much. Sometime people who are coming to address this subject use, you know that they are just trying to be here to say something.” —Les Aspin Alumni Mali 22 3. The program’s presentations are inconsistent in quality and too many themes introduced reduce the overall ability of participants to learn skills and concepts in a meaningful way. IMPACTS ON GOVERNANCE ACCOUNTABILITY SYSTEMS IN AFRICA (SOW QUESTION 3 AND PARTIAL QUESTION 4) This section combines SOW Question 3 and part of SOW Question 4, as both relate to the causal relationship between participation in the Les Aspin program and the USG’s effort to strengthen accountability systems in Africa. Findings Program Theory of Change By reviewing the 2001 grant document for this program, we developed the theory of change, or the program’s view on the causal relationships between USAID’s general objectives, those of the program, and the various activities that make up the bulk of the Les Aspin program. Program theory hypothesizes how change or impact is to occur. The overall goals of the program are to support USAID strategic objectives of strengthening good governance and fighting corruption in West and East Africa. We find the initial 2001 objectives of the Les Aspin Program are to: 1. Reduce election related conflict 2. Reduce public sector corruption These objectives would be attained by training activists (as selected by the LAC and USAID) in: Objective 1 1. Sensitization of causes of violence and strengthen ability to identify emergent conflict situations 2. Conflict management skills such as negotiation, peace brokering, consensus building 3. Election monitoring skills Objective 2 1. Learning about USG processes, systems, etc., in controlling corruption 2. Follow-on activities—collaboration with NDI/Ghana Anti-corruption Coalition, etc. The stated pedagogical approach is “interactive learning,” a learning method featuring experiential learning techniques, site visits, etc. As stated in the 2001 proposal/grant, the steps in the causal logic sequence are: 1. Select the right people (as determined by position and reputation for integrity, leadership, and activism). 23 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report 2. Use interactive-learning techniques and exposure to U.S. good-governance practices to build awareness and skills relevant to objectives in the West and East Africa context. 3. Promote follow-up activities designed to apply new attitudes, knowledge and skills to addressing problems of election violence and corruption. Later, Les Aspin revised their objectives to focus more generally on leadership attributes and skills. Specific skills linked to monitoring elections and resolving election-centered disputes became less important. The key components of the underlying theory are (1) that skills and knowledge learned from the U.S. experience will be applicable to dealing with corruption and bad governance in Africa and (2) the reliance on interactive, adult-learning techniques will successfully transfer skills, knowledge, and promote values and motivations for taking action on return from the United States. We have presented findings that suggest that, in practice, the Les Aspin use of “interactive learning” techniques falls rather short of the theory’s expectations. When actually used, as in the mediation workshop, this technique has had the desired effect. In the following section, we examine the extent to which skills learned and attitudes gained resulted in follow￾up activities to reduce corruption and strengthen good governance. Does Participant's experience in the program help to strengthen accountability systems in Africa? (SOW Question 3) The Les Aspin Center asserts that its programs directly contribute to the goals of USAID and to the USG’s interests in promoting accountable-governance systems and the capacity to sustain accountability systems in Africa. It is incredibly difficult, if not impossible, to determine a direct, causal link between participation in a Les Aspin program and a direct, sustained impact on corruption and good governance in a country. However, we can examine the impact participants have on their governments, at a national or regional level, and on their home organizations, which in turn can have an impact on larger anti-corruption efforts. In their 2007 report, the LAC claims that alumni “improve institutional capacity and resources in support of the implementation of accountability initiatives.” We did find, both through the survey and through focus group discussions and success stories, that alumni were relatively active in their organization when they returned home. In fact, 89.7% said that they had tried to implement good governance and accountability improvements in their own organizations. Of that number, 32.2% claimed to be very successful in their efforts, and another 49.4% said they were somewhat successful. Some did cite specific skills that they gained from Les Aspin, or at least a general awareness of the issues, but it is hard to say to what extent these efforts affected the large-scale, anti-corruption efforts of the USG. The LAC does document the activities of some alumni following completion of the program. These updates are found in annual reports to USAID. According to the Les Aspin staff, alumni are working on three levels: • Government anti-corruption commissions • Institutions that have an impact on corruption control: i.e., media, parliament, auditors • Institutions that affect policy reform (i.e., religious institutions, grassroots communities, NGOs) The success stories documented by the LAC described multiple instances in which alumni created and/or implemented a new program, trained others on anti-corruption measures, received career advancement based on their participation in the program, or implemented changes in their home organization. These stories, 24 while they do show that some Les Aspin alumni have been active after their participation in the program, cannot be directly attributed to the Les Aspin program. More detailed success stories were recorded by the evaluation team during their visits to the Ghana and Kenya refresher courses. These accounts better display how the skills and knowledge participants gained during the course were applied directly to action. In one instance, a headmaster and presiding member of a district assembly in northern Ghana was elected district chief executive after his participation in Les Aspin, and subsequently implemented substantial reforms that included making information on government projects accessible to the public. He attributed his new approach to what he learned about effective communication and transparency through the Les Aspin program, and specifically through observing the transparency of the U.S. political system. The Les Aspin staff was also able to give many accounts of successful alumni in high positions of power, whether in government, media or NGOs. However, being in a position of power does not automatically lead to impact, and there is still no formal mechanism by which Les Aspin measures whether the program directly contributes to USAID and USG goals. None of the USAID staff interviewed could describe specific impacts made by Les Aspin alumni, nor did they believe it was possible for direct impacts to be measured. All interviewees did acknowledge that, while this is a difficult concept to measure, individuals in positions of power with increased knowledge of accountability practices can be effective at creating change and reducing corruption. It was generally agreed that building a critical mass of informed leaders will help lead to positive impacts. The USAID survey respondents were somewhat split on the question of impact as well. Five of nine USAID DG officer respondents believe that Les Aspin alumni have been able to impact larger-level accountability issues in the country’s governance, such as policy reform. One person gave a very positive response: “Les Aspin participants are spearheading advocacy and reform of key anti-corruption laws in Nigeria. Some participants are also in strategic positions of both governmental and non-governmental agencies—Economic and Financial Crimes Commission, key legislative committees, heads of coalitions, electoral body, et cetera— and are influencing reforms of policies and processes.” Other DG officers seemed less sure, saying that they had not seen any evidence, that it was hard to tell given the prevalence of corruption, and that they were not familiar enough with the daily work of participants to be sure. Despite uncertainty about the ability of participants to impact their countries’ accountability and governance systems, eight of the nine DG officers surveyed said that, overall, the Les Aspin program supports their DG strategy and work. The one person who said “no” explained that the linkages to their Mission’s strategy of local service delivery and community participation were only tangential. One of the ways in which Les Aspin claims that its program supports the work of the USG is by increasing the skills and capacity of local personnel, with whom missions could cooperate when implementing democracy and governance programs. Although USAID survey respondents did indicate that the Les Aspin program supports their work, observations and interviews suggest that Les Aspin alumni have limited interaction with USAID. Only one of the success stories reviewed indicated collaboration with the USAID Mission in that country. In addition, it appears that the Les Aspin program maintains limited links with USG partners working in good governance and anti-corruption, such as National Democratic Institute, Freedom House, National Endowment for Democracy, International Republican Institute, etc. 25 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report As seen in Chart 5, above, 66.3% of alumni respondents believe that they have been able to make a greater impact on larger-level accountability and governance in their country since their participation in the Les Aspin program. Some examples of larger-level impact include: • As part of an organization that regulates NGOs in Kenya, one participant developed a code of conduct that ensures professionalism and ethics in the NGO sector nation-wide. • One participant reported assisting various government departments in setting up service charters and codes of conduct. • Another participant was involved in revealing mismanagement of the Pension Fund by the Inspector General of Government, which led to management changes and eventually the prosecution and conviction of the former manager. • A Malian participant’s ministry called upon her to help them build a strategic plan to fight corruption as an impediment to good governance. Those who responded that they had not been able to make a greater impact (33.7%) cited financial restraints, lack of time (for more recent graduates), and above all, rigidity and opposition to change by the government and politicians. Within their own organization, 88.8% said that they had tried to implement good governance and accountability improvements in their own organizations, with 31.5% of those people saying they were very successful in the efforts, and another 49.4% were somewhat successful. From the DG officers’ perspective, eight of the nine USAID respondents think that Les Aspin alumni have been able to affect larger-level accountability systems within their own organization. However, at least three of those people explained that they could not be sure, adding comments like, “Most probably,” or, “I am guessing yes,” or even, “Not possible to make a fair evaluation, since we do not track this.” Other respondents were able to cite concrete examples, particularly instances of alumni impact in the government. Unanticipated Consequences In addition to evaluating whether or not the Les Aspin program contributes to African countries’ anti￾corruption efforts and to the goals of the USG, it is important to consider other, unanticipated areas of impact that the program may have had over the past ten years. An unexpected outcome in terms of skills and knowledge was the level of importance placed on moral courage—a book provided to participants and discussed in the program as part of ethical decision-making 66% 34% Chart 5: Impact on larger-level accountability Do you believe that you have been able to make a greater impact on larger-level accountability and governance, such as policy reform, in country since your participation in the Les Aspin program? Yes No 26 practices. In particular, alumni working in government found this to be an extremely important element of the program. It provided them with motivation to face their jobs on a daily basis, to make ethical decisions, and to refuse to participate in bribery. It was clear from the focus group discussions that the level of corruption within government institutions is stifling and profound for the majority of the Les Aspin countries. Les Aspin alumni in government positions also noted that they need support from civil society alumni to take the lead on anti-corruption issues, and they will then follow and support. It was clear that alumni from the government sector feel a great deal of risk involved in being seen as launching anti￾corruption efforts. Some of them described personal risk that they have faced in opening up inquiries. The program offers very little discussion on the risk surrounding anti-corruption efforts, and provides no tools or strategies for risk reduction. In addition, the lack of a sustained effort to connect alumni with anti-corruption organizations and each other reduces their ability to support each other and share tactics for reducing risk. Impacts on Governance Accountability Systems in Africa Conclusions 1. Although USAID survey respondents indicated that the Les Aspin program supports their work, observations and interviews suggest that the Les Aspin program maintains limited links with USG partners working in good governance and anti-corruption, such as National Democratic Institute, Freedom House, National Endowment for Democracy, the International Republican Institute etc. This lost opportunity to network with other practitioners working in the field and to share best practices and challenges undermines the program’s ability to reduce corruption on a larger scale and makes it harder to mobilize a “critical mass” to achieve USAID/USG goals. SUSTAINABILITY AND THE EXTENT TO WHICH IT HAS BEEN BUILT INTO THE PROGRAM (SOW QUESTION 5) Findings We have defined sustainability as the continuation of networks, learning, sharing, reinforcing, and positive support for advocacy and action beyond the completion of the formal Les Aspin program. Efforts to build sustainability into the program for long-term impact remain a key concern for all stakeholders. The sustainability concerns cited by our interviewees and survey respondents, and confirmed through our team’s first-hand observation of the program, can be grouped into three major issues: 1. No stated objectives or concrete expectations set out for participants 2. No accountability or formal action planning or goal setting for participants 3. A lack of systemic follow-up or mechanism for interaction with program alumni We can see that these issues occur at three distinct intervals: at the outset of the program, during program implementation, and after completion of the program. In speaking with the Les Aspin staff, it is clear that they have great pride and enthusiasm for this program. They feel it is a strong leadership-development program that gives participants the skills to combat corruption and governance issues in their countries. The program director has developed a “Program Results Matrix” identifying “Experience or Knowledge Characteristics” they hope are gained by participants, how said characteristics will be demonstrated in participant’s “behavior,” and the impact this behavior ideally creates. However, alumni interviewed during both the East and West Africa refresher courses explained that they never felt that the program’s objectives or intended results were ever clearly articulated to them. No alumni 27 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report interviewed could recall receiving a formal agenda with objectives and expectations prior to arrival at the program. The agendas given upon arrival at the workshops were simplistic, typically providing only the session’s title and lecturer's name. The lecturers interviewed also noted that they received little guidance in terms of the objectives for their sessions or how their lectures were intended to relate to the rest of the program. In fact, no lecturer interviewed could explain exactly what the participants learned during their time in the Les Aspin program. All lecturers interviewed said a more detailed orientation to the program would be helpful.30 This lack of an articulated vision leads to the second issue, accountability. With no specific objectives communicated to the participants, there are no results by which to measure the impact of the session and, therefore, no accountability for the retention of information or mastery of skills. The most tangible effect of the lack of accountability is found in the action-planning session. At the end of the program, the participants attend a session on creating action plans for future work, or “next steps,” post-Les Aspin participation. Our team observed this most recent session in Washington D.C.: the participants were given resources on creating action plans and were put into groups to discuss their goals for the next five years. No formal action planning was performed. Our focus group discussions with alumni in Ghana and Kenya confirmed this finding. According to the majority of alumni contacted, no action plans were created during the original program they attended. Participants in the current year’s program added that the action planning session that did occur was very vague and overwhelming, as they did not know how to translate what they had learned during the Les Aspin program to an actionable future plan. We should note here that action plans were created during the refresher course, and this is discussed at the end of this section. Once program participation ends—which is when sustainability efforts can be observed best—we find that no formal mechanisms for outreach to alumni, or amongst them. The Les Aspin program director keeps in touch with alumni informally, and tries to update an Excel file with these alumni’s most recent email addresses and job titles.31 In addition, during program orientations in Ghana and Kenya, Les Aspin staff brings former alumni from Ghana and Kenya for the opening ceremony at the U.S. Embassy. However, no time is allocated after the ceremony for alumni to meet new participants and get to know each other. Any sustained contact amongst alumni is left to the more outgoing participants. In our survey, we found that USAID officers also had little communication with program alumni: 40% interacted with them two to five times a year, 20% once a year, and 40% less than once a year or never. One officer from Uganda reported that they ask the participants to brief the Mission on the training and to provide a report. Another respondent from Mali said that they had conducted a follow-up evaluation and case study in previous years. A few respondents cited meetings with alumni at USAID, and others said that the lack of a coordination body made contact difficult. Perhaps because of this lack of communication, USAID staff was also unsure of which follow-up activities were coordinated by Les Aspin alumni. Three people said that they do organize alumni get-togethers and networking opportunities, but that was the extent of their familiarity with alumni activities. One hundred percent of USAID respondents think that follow-up activities would increase the impact of the Les Aspin program. They explained that follow-up activities offer important opportunities to utilize 30 Les Aspin staff commented on this finding: “We would want to note that presenters who have normally been invited have always been given instructions on why they have been invited, the topic to discuss, types of participants and backgrounds, expected learning outcomes, among other details (samples of presenter invitation letters can be made available if necessary). Details of letters of invitation vary with respect to whether we have a regular presenter or someone unfamiliar with the programs. It is also not uncommon, that in spite of a presenter receiving an invitation with terms of reference, when they arrive and meet the participants, they suddenly decide to refocus the discussion. This is an ongoing challenge that we are not able to predict and control ahead of time. Therefore, it is understandable here when participants note their reservations that they are not sure whether or not in one situation or the other presenters are given clear terms of reference for a presentation.” 31 We used this file to reach out to alumni during for the survey and, as noted earlier in this report, found that many of the contacts for alumni prior to 2006 were out of date or not available. 28 knowledge and skills, strategize and harness available resources, and maintain the momentum gained from the training. During our field visits, one officer at a participating East African mission said, “We have to foster leadership with integrity. There needs to be some form of sustained support for these individuals.” In our survey of DG officers, all respondents suggested various ways of increasing follow-up, including providing funding to the missions to track the progress of alumni and to organize events that bring together the country’s alumni. However, making this support a reality is more difficult in practice. The Les Aspin Center says they can only do so much with the funds they receive. They have received the same level of funding since 2001 ($500,000/grant year) and other program costs, like airfare, lodging, and meals, have continued to escalate.32 In addition, they have three staff members who work on the Les Aspin program. Adding follow-up activities to their current responsibilities is not possible right now, according to staff. For USAID, a grant such as this does not have the same rigorous monitoring requirements found in other types of mechanisms, like contracts and cooperative agreements. This means that there is less oversight overall and no official responsibility to keep records on the participants. In addition, the grant is funded out of USAID/Washington and operates outside Missions’ development strategies, so there is no consistent method by which the Missions follow up on this program. For now, this means that much of the impetus for sustainability is left to the alumni, with mixed results. Les Aspin staff said that the West Africa alumni created the West Africa Organization for Democracy and Economic Growth (WAODEG), with board members in all countries and chapters in Mali and Nigeria. Unfortunately, it never took off as an active organization. Too many of the members were transferred into new positions and they never had the time to make it function effectively. Individual alumni reported some success stories from their own work, however. One Kenya alumnus said that she worked with Action Aid and the Kenya Human Rights Commission to lobby against skewed trade agreements. Another raised funds to hold a regional conference on the civil society perspective of anti￾corruption mechanisms, and another trained women members of parliament (MPs) in South Sudan on lobbying in parliament. A journalist from Tanzania said that the Les Aspin program “ignited” him to publish more “hard hitting” stories, while an MP in Uganda introduced an anti-money laundering bill after her participation in the program. In our survey, Les Aspin alumni reported having fairly regular contact with other alumni both in their own country and in other target countries.33 32 Please refer to cost effectiveness section for more details. 33 In the survey, the alumni were not asked if this communication is directly related to their Les Aspin connection or if they knew each other beforehand. However, during our focus group discussions while on site visits, we found that most alumni met during the program. 29 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report OBSTACLES TO IMPLEMENTATION OF GOVERNANCE ACTIVITIES Despite the fact that majority of alumni were at least somewhat successful in implementing governance and accountability improvements in their organizations, they still reported facing numerous obstacles to change, including: • Financial resources; • Fear of change; • Resistance by management and a belief that change comes only from the top; • A general culture of impunity and gender inequality; • Bureaucratic structures and red tape; • Suspicion of alumni’s motives and fear of being investigated or fired; • Low levels of education and understanding As seen in the chart above, 86.9% of the respondents reported that they communicate with other alumni in their own country at least two to three times a year, with 56.6% saying that they communicate once or more a month. Contact with alumni from other countries is slightly less regular, with only 37.5% saying they communicate once or more a month, and 59.7% communicating at least two times per year. Contact with Les Aspin staff and program faculty was even less common, with 44% of alumni reporting that they communicate with staff once a year or less. It should be noted, however, that 58.9% of respondents were alumni from 2008, 2009, and 2010. We do not have survey data to tell whether this level of interaction is sustained several years after participation in the program. In our discussions with alumni during the Ghana focus groups and at the refresher courses in Ghana and Kenya, most said that it was the first time they had met other alumni in their country. The focus of communication is primarily with participants in the same year. The Les Aspin program shares very little information on previous years’ alumni with new alumni. Some participants had kept in touch with alumni from their class, but there was no indication of alumni from different years (i.e.: a 2008 alum in touch with a 2004 alum) interacting in a sustained manner. Alumni attributed the lack of communication to several factors, including lack of contact information, lack of time, and lack of a structured forum for interacting with other alumni. Those who do maintain regular contact emphasized the benefits of collaborating with other like-minded alumni: “For my own country we have regular contact and help each another in implementing programs in our organizations.” “[Interactions with other alumni are] great networking opportunities to share and learn from each other on what techniques or programs are working in the field of governance and accountability.” Some alumni expressed regret that they were not able to stay in touch with Les Aspin staff and faculty, though several mentioned having regular contact with the program coordinator, Dr. Cephas Lerewonu. Unfortunately, as the Commission on Human Rights and Administrative Justice director in Ghana and Les Aspin staff noted, a collective effort to combat corruption has not been 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% Other program alumni in your country? Other program alumni in other countries? Les Aspin staff or faculty? Chart 6: Alumni Communication Do you communicate with: (N=76) Once a month or more 2-10 times a year Once a year or less 30 created. While individual alumni implement their own efforts to work on good governance and anti￾corruption efforts, their interest wanes over the years and they do not usually work together in the long-term. Aspiration for more sustainability-related support To address this issue in the current Les Aspin grant program, USAID/Washington suggested combining the East and West Africa programs and adding an alumni refresher course as the second program in 2011. According to the Les Aspin program director, the objectives of the refresher course were to explore ways of ensuring that alumni collaborate and work together to create collective actions within their countries, and to ensure that the main objectives and the outcomes of the Les Aspin Program are sustained in the long term. He said the Center believes that this desired collaboration can be achieved through knowledge, skills, and resource sharing by participants and through ongoing mentoring offered by Les Aspin Center staff. The participants in the refresher course were sent letters from the LAC or called by DG officers at their respective Missions to announce they had been selected for participation in the refresher course. LAC staff said that they, along with the Mission staff, chose alumni who they know are active in the anticorruption/governance field. The participants felt that the selection process was not transparent and they were unclear as to why or how they were selected over other alumni. Non-refresher course participants seemed especially upset about the selection process and their inability to get a clear description of the guidelines for selection. There were approximately ten participants from each grant recipient country, and the participants did seem engaged and energetic and all discussed at length their passion for good governance and anti-corruption work. All participants in the refresher courses (East and West Africa) said that they enjoyed the experience overall. They liked meeting with alumni from their regions and hearing about their experiences since participating in the original programs. They also supported the course’s main objective of creating platforms for joint action (i.e., action plans) with alumni from their country. This was a more formalized process than in the regular course, and much time was spent on this work, as described below. Participants in the refresher course indicated it “has energized me;” “given me new connections;” “was enriching;” all of these quotes were repeated by a majority of the refresher course participants. The East and West Africa refresher course agenda’s differed slightly, but in general, the seven-day course began with two-and-half days of introductory sessions, in which outside lecturers (some former alumni) gave presentations on emerging trends in corruption control and the outlook on governance and accountability trends. According to Les Aspin staff, this was intended to set the tone for the week; the following three-and￾a-half days involved learning from each other through lectures from participants, sharing of participant experiences, and country-action-plan development. There was also one field visit during the West Africa refresher course, but the visit in East Africa was cancelled the day before due to a pending government vote that the host organization needed to work on. The site visit in Ghana was to the National Museum of Ghana and included going past the old Parliament House and Independence Square; according to our evaluator present, it seemed more about sight-seeing than part of the learning process. When asked about the visit, alumni thought they could have visited Parliament while in session or the Attorney General or Accountant General’s department to see how they deal with corrupt practices in Ghana. All sessions in both Kenya and Ghana were facilitated by either Les Aspin staff or alumni (both those participating in the refresher course and those that were not), with the exception of two lectures. As the idea of the course was to connect alumni and have them share their experiences, the Les Aspin staff felt that having alumni use their skills and experience to facilitate sessions would be valuable and preferable to outside lecturers. 31 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report ACTION PLAN FROM KENYAN ALUMNI Goal: To form an integrity network that empowers Kenyans to engage in good governance and make informed choices. Specific Actions: • Conduct research on issues of integrity in Kenya • Act as a resource base for members • Think tank- reference point on governance issues • Articulate issues on governance issues • Awareness creation to general public on the network and related issues • Resource Mobilization • Duty & Responsibility Allocation (Sept – Dec 2011) • Survey Tools • Communication Strategy • Coordination • Policy and Legal Formulation Although the participants described the training methodologies as participatory, the only participatory tools used were group work and open discussions. It lacked activities such as case studies, role plays, and reflective exercises. There was substantial time allocated to each topic, but overall the time did not seem properly employed. Many of the alumni presenters talked about their own work, but it was not certain if these were “case studies,” as they were not followed up by group work to discuss the good practices and how others could apply them in their own settings. One participant thought that some of the presentations could be described as a presentation of “CVs”. The Les Aspin Center confirmed that they do not review all the presentations before the course, which many alumni felt could have helped correct this issue. In East Africa, there were sessions on social networking and information technology that were practical and useful for the teams in developing their action plans. The sessions were given by refresher course participants. Each presented an overview of the topic and tools, then offered specific examples on how to utilize these tools in the participant’s day-to-day work. In our focus group discussions, most alumni said these were the best presentations of the course, as they focused on an emerging topic in the field and taught new skills that alumni could use in their action plans and their own work. Overall, the alumni said they had expected to learn and discuss new trends or issues in corruption around the world and in their regions, in particular. They had also expected to learn about the impacts of alumni work in each country, to learn what had led to achievements or failures, what strategies/tools learned during the Les Aspin training have been implemented, and what they have learned. During one day of the program, each alumnus took a turn in describing what they have done in their job, what they learned from the Les Aspin program, and challenges they have faced. While some of the alumni’s presentations did create a good group discussion, overall, the presentations seemed like a publicity message and did not delve into specific issues or create more substantial topics for discussion. Many alumni pointed out that if they had been more involved in setting the agenda for the refresher course, they could have created interactive presentations based on reoccurring issues or topics that alumni face in their day-to-day work. In addition, the alumni did not receive any preparatory work to complete or reading requirements prior to arrival. All said that they would have been willing to do this work, especially if it would provide a greater focus to the discussions. Action plans were created in the refresher course, and we found that a substantial amount of time was given to the alumni to work in their country teams and come up with a way for the alumni to network, work together, and develop ways to make a collective impact in their countries. Most of the alumni agreed that the time devoted to action planning was substantial (almost three full days of the seven-day program) and helped them to develop detailed plans and next steps. They also said they appreciated being able to spend extended time developing their plans instead of having additional lectures. The major critique of the refresher course’s action-planning activity, and arguably of the sustainability issue more generally, is that it was still not clear how support will be sustained, either financially or with technical assistance. “If we are developing plans then we should have resources available to us,” said one participant, 32 and many agreed. During the Ghana refresher course, the Les Aspin Center program director mentioned that the alumni should keep in touch with the center for technical support for the work of the coalitions they have created, but that the center cannot supply materials or fund the coalitions. Most alumni understood that funds are limited, and mentioned that they would be willing to collect dues from each other for an alumni program or network, but most felt some form of resources or assistance in accessing resources is warranted. Sustainability Conclusions 1. The refresher course is a positive addition to the program, especially the creation of country action plans. It, along with the regular program, could be more effective with increased organization and transparent objectives, agendas, and goals. 2. Additional engagement with participants is also needed for increased effectiveness. Currently, there is no pre-work or research done by participants, nor are the majority contacted for input into the agenda or presentations. 3. Sustainability of the program is directly linked to the issue of lack of follow-up. At present, sustainability is questionable, notwithstanding positive appraisals by respondents. Without follow-up, morale, interest and activity wanes amongst participants. GENERAL CONCLUSIONS The Selection Process 1. In spite of weaknesses in program design and implementation identified in the Findings sections, the Les Aspin program has identified and motivated a potential critical mass of African leaders who could be utilized in USAID and other donor efforts to strengthen good democratic governance. This is a valuable asset for African governments, civil society, and USAID. 2. On balance, the selection process has fulfilled the purpose intended, including mix of public and private participants, gender balance, and geographic distribution among the six participating countries. However, there is considerable variation in the process, and transparency and clarity with regard to the initial selection process needs to be strengthened. Participants were often surprised they had been selected and, initially, were not fully informed about the program objectives and logistical arrangements. Program Effectiveness 1. Participants clearly value the program and the opportunity to go to the USA and learn about U.S. systems and processes for promoting good governance and controlling corruption. However, recall of specific skills learned or of their application in African settings was much weaker. General research into adult skill learning supports the conclusion that unless skills are used in the work place on return, they will eventually wither. The same applies to motivational and attitudinal changes promoted by Les Aspin program. Skills used were those taught through interactive workshops and which were applicable to the trainees’ immediate workplace. 2. Although the original proposal and the program theory emphasized interactive learning as the key distinctive feature, the actual program is heavily reliant on lectures for some segments of the thirty- 33 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report and-one-half days of exposure. This reliance, combined with the absence of specific program￾module-training objectives, lesson plans, and follow-up learning assessments serve to undermine the overall effectiveness of the program. 3. The LAC invites feedback from participants on an ad hoc basis, but has no systematic program evaluation process in place by which outcomes and impact can be periodically measured, and through which informed program improvements can be made. 4. The program’s standard presentation approach reduces the potential for learning, and does not capitalize on the depth of experience and knowledge brought by the participants. This approach signifies treatment of participants as “students” rather than key stakeholders in an adult learning program. 5. Based on the evidence available, the program, on the whole, is cost effective compared to one other USAID-funded program with some similar features. There has been no increase in the grant level over the ten-year period. Les Aspin has found ways to absorb rising transportation and lodging costs, largely by cutting the program’s duration, reducing staff travel, and cutting lodging costs by requiring participants to share hotel rooms. 6. The program has no systematic monitoring and evaluation process in place to guide Les Aspin Center leadership in coping with rising costs and changing circumstances in Africa. Without such a system in place, decisions about cost cutting or program re-orientation appear based more on anecdotal evidence, leading to a focus on administrative and logistical cost reduction measures, rather than a broader examination of the effectiveness of various program components. Follow-up and Sustainability 1. The program fails to meet the challenge of fostering and supporting sustainable results though follow-up activities or other mechanisms, either within countries or through networking across the regions. Individual trainees have taken initiatives, such as those mentioned in the success story annex,34 2. Other sections of this report have emphasized the lack of follow-up support for LAC alumni to pursue good governance-anti-corruption efforts on their return to their home countries. Any continuation or expansion of the program will require both a serious review of the effectiveness of, and necessity for, all phases of the existing program, as well as a realistic cost projection steps necessary to strengthen longer-term asset utilization and effective contributions to USAID and host country goals and objectives regarding strengthening good governance and controlling corruption. but in the main, no active follow-up program supports networking, knowledge management, or specific initiatives through, for example, a small grant program. While additional funding does not guarantee sustainability, the absence of a follow-up program backed by some funding would provide the kind of support needed to activate and mobilize the substantial human assets created by the Les Aspin Center Program. USAID Involvement 1. With the exception of USAID/Kenya, USAID mission participation in the participant selection process has been useful and well regarded. 2. USAID's aggregate involvement is not based on much continuous interaction or participation beyond the initial recruitment process. As a long-running grant program, USAID has seen the 34 Annex G 34 program as generally useful, but the program operates outside of the Missions’ development strategies; therefore, the program does not benefit from much support beyond the administration of the grant. 3. USAID has no policy or approach for capitalizing on the momentum for action that the program creates amongst its participants. This is a lost opportunity for USAID to demonstrate a sustained commitment to supporting outstanding leaders in these countries and gain a network of supporters for its DG work. OVERALL RECOMMENDATIONS 1. Continue to support a program of leadership and good governance/anti-corruption training and exposure in the United States for African participants who have already established a reputation for integrity, commitment and activism, taking into account gender, positional and regional balance. USAID Mission participation in the selection process should be continued in collaboration with the implementing partner retaining considerable control over the selection process. Criteria for participant selection needs to be clearly stated and the process made as transparent as possible to ensure objectivity in the selection process. 2. Separation of participants by region is unnecessary and does not allow for cross-regional sharing of anti-corruption efforts and governance systems. A strengthened emphasis on commonly shared problems/issues of good governance and anti-corruption would enrich the array of experience on which leaders can draw, and would facilitate cross border networking determined by shared interests. 3. The program should reduce the quantity of themes and focus more on the quality and preparation for the presentations. Time should be allocated to each module in order to support training with clear lesson plans and exercise activities to support standard adult learning approaches. Trained facilitators or those that have training in contemporary adult learning methods should be incorporated into the design of the program. 4. Invest in a range of follow-up activities (see specific recommendations below) that would substantially strengthen alumni connectivity, solidarity, program impact and sustainability. Follow-up activities should be at both the national and the regional level. Appropriate follow up activities may have cost implications; therefore, prioritization of activities is critical. 5. The program’s locations should include well structured sessions in the United States at the Federal and State level. State level exposure can be more to the ‘scale’ of African nations, therefore less intimidating. However, focused encounters with key national level governmental and Civil Society organizations reinforce specific learning objectives and strengthen the credibility of the participant’s experience and status on their return home. Actual training locations in Africa could vary from program to program, depending on training themes and least cost logistical arrangements. 6. A workable system for monitoring and evaluation metrics needs to be embedded in any follow on program in order to assess costs related to specific benefits, as well as ex-post outcome measures. Some elements of this would include pre-training needs assessment for baseline and program targeting purposes, systematic and simple qualitative evaluation surveys asking participants to rank their preference for presentations at the end of each week – soliciting specific, anonymous written feedback on all activities and lecturers. Simple follow-up interviews should be conducted with alumni to ask how they have used the knowledge and skills gained to change the way they perform 35 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report their work. The M&E system should engage program alumni as well as USAID and implementing partners to ensure stakeholder ownership and utilization of findings to improve program content, delivery, and follow-up outcomes and impacts. 7. Strengthen USAID Mission and Africa Bureau engagement in the program, possibly by issuing a follow-on program RFA for a Cooperative Agreement-type of relationship, which would include substantial involvement with an implementing partner. Substantial USAID involvement, beyond participant selection, would encourage USAID to make more effective use of the highly valuable program alumni group and would lead to more effective causal links between the training program results, other USAID good governance programs and higher level USG good governance and anti￾corruption objectives. SPECIFIC RECOMMNEDATIONS FOR EACH SOW QUESTION Effectiveness of Partner’s Approach in Achieving Results • Combine classes between East and West Africa and use additional funds to support communication between alumni in terms of networking. • Include an international comparative political analysis component between the US and African anti￾corruption and good governance systems. • Link the DC portion of the program more closely with other USG and international initiatives working in Africa to fight corruption and promote good governance by including meetings with groups such as National Endowment for Democracy (NED), National Democratic institute (NDI), International Republican Institute (IRI) and Freedom House. Participants Ability to Utilize Skills or Information Learned During the Program • Time in Washington, DC should be reduced from three weeks to two weeks, maintaining the most important elements of the program, GAO, Office of Government Ethics. • The week in Africa should take place AFTER the US portion with a focus on providing the participants with professionally-facilitated workshops to support contextualization, strategic planning and individual action plans. • Consider more interactive visits including shadowing or brief internship experience for participants to increase hands-on knowledge. Impacts on Governance Accountability Systems in Africa • Promote networking, knowledge sharing, and support structures for alumni leaders undertaking activities related to strengthening ethical governance, accountability, and transparency. • Develop a competitive small grants support program for program alumni action plans that focus on good governance and accountability.35 35 According to Les Aspin Center staff, USAID/Nigeria will begin this next year by adding money to their program budget. 36 Sustainability and the Extent to which it has been built into the program • A shared electronic site where resources, news and on-going communication can be maintained by alumni should be created (i.e.: SharePoint). Unlike the country sites that some alumni are now developing, this would be an independent common site to support on-going learning among alumni, keeping them abreast of new developments in anti-corruption work (including USAID’s programs), best practices and new tools and will also support alumni communication. • After each program cycle, there should be a collation of presentations and hard copies or electronic copies should be provided for all participants. This could be done via a SharePoint site or the distribution of flash drives. A simple flash drive would be especially useful for participants with limited access to internet. • In the regular program, it is important to state at the outset that participants will be expected to come-up with their own action plans and adequate time needs to be provided to participants with professional strategic planners to work on developing these plans. The implementer could assign a staff member to follow up with participants on these plans six months after completion. This could best be done during a final week in Africa, supported by a professional strategic planning expert to guide them. • There is a need for a sustainable link between USAID and alumni. There are varying levels of involvement to choose from, all with the ability to impact sustainability. o Provide a forum for alumni to meet once a year (i.e.: annual conference for alumni). This could be done by each country or regionally and cost-shared by the Missions. o Create long term relationships with the alumni and leverage their experience to help achieve the Mission’s DG goals through: • Financial support of short term mentoring from program facilitators for the most promising participants; • Financial support for the development of long-term sustainability networks in country and/or provide technical assistance to the networks that the alumni create. 37 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report ANNEXES 38 ANNEX A: SCOPE OF WORK C.1 BACKGROUND By the close of the grant on September 30, 2011, USAID will have provided nearly $5 million in support to the Les Aspin Center for Government of Marquette University for their Anti-corruption and Good Governance for Eastern and Western Africa program. USAID/AFR/SD has funded and USAID/DCHA/DG has managed the Les Aspin Center's grants, awarded in response to unsolicited proposals, for ten years. The Les Aspin Center for Government is a Washington, DC-based leadership training institute of Marquette University in Milwaukee, Wisconsin. With USAID support since 2001, the Les Aspin Center has run six-week international visitors' programs to the United States for 270 young African leaders in government, civil society, and the media on anti-corruption and good governance themes. Corruption continues to be identified as one of the major problems that besets and undermines governance and leadership practices, performance and service delivery systems, and the practices of private sector businesses. One of the main goals of the training programs has been be to provide government and civil society personnel with skill-enhancement training on how corruption can be managed, and with what tools, in order to allow private and public sector entities to function more effectively in answer to and in consonance with the requirements of the public trust. Another goal of the capacity development training program is to shape and strengthen the knowledge and skills of the participants to increase their competency in using and implementing standards of integrity and accountability in management, performance and service delivery and public policy reform contexts. In 2009, USAID Africa Regional resources via Les Aspin trained 32 young Africans from East (Kenya, Tanzania, Uganda) and West Africa (Ghana, Mali, Nigeria) in anti-corruption and good governance practices. The program creates an opportunity for talented young African government officials, representatives from the media, and other non-governmental actors to travel, debate, and interact with one another as they learn about anti-corruption best practices in the United States. The Les Aspin program's usual process has featured two program cycles per fiscal year, one focusing on West Africa and the other East Africa. At USAID's request, Les Aspin adjusted their final fiscal year of programming to conduct one training cycle with participants from both East and West Africa and to use the second training cycle for a "Refresher for Program Alumni" from both sub regions. The evaluation should take this deviation into account in preparing program implementation observation and in drafting the report. For additional information on the Les Aspin Center for Government, including summarized activity schedules for training cycles one and two of their program, please refer to Attachment 1: Appendix I. C.2 PURPOSE The purposes of this task order are to: 1. Evaluate the effectiveness of the partner's approach to achieving results. 39 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report 2. Evaluate program participants' ability to utilize skills or information learned during the program to affect corruption and good governance in their home countries. The grantee claims that the Les Aspin program graduates belong to organizations that partner with USAID Missions in East and West Africa. 3. Determine whether participants' experiences in the program help to strengthen accountability systems and the U.S. government's effort to enhance the lives of the people of Africa. 4. Evaluate the (cost-benefit) value of this program to the objectives of the U.S. Government and should assess the Les Aspin Center claims that its anti-corruption and good governance programs directly contribute to the goals of USAID and to the U.S. government's interests in promoting accountable governance systems and the capacity to sustain the accountability systems in Africa. 5. Evaluate the partner's efforts to build sustainability into the program for long-term impact. Describe ways in which sustainability could or should have been strengthened. C.3 ACTIVITIES In conducting this evaluation, the contractor shall: 1. Review USAID and grantee documentation regarding the program history, design, and implementation; 2. Collect necessary and appropriate data for this evaluation regarding the Les Aspin program, anti￾corruption needs and efforts in Africa, and USAID programming, for example, by distributing questionnaires to all past program participants; 3. Interview representatives of the grantee, USAID, participants, and others relevant to the evaluation; 4. Conduct a site visit to at least one participating country in each programming region —East and West Africa; 5. Observe select program activities during calendar year 2011; and 6. Employ both quantitative and qualitative evaluation techniques. Within 45 days of the award, the contractor shall provide to USAID for approval a detailed workplan for the life of the project that clearly states their evaluation approach and methodology. The contractor shall produce a written final report not to exceed thirty (30) pages containing at least the following sections: 1. Preface or Introduction 2. Executive Summary 3. Background 40 4. Evaluation of Program Design 5. Evaluation of Program Implementation 6. Evaluation of Program Sustainability 7. Conclusion / Summary of Findings and Recommendations Annexes may be appended to the report as needed and will not count against the page limit. The contractor shall produce the report in two stages. The first, interim stage should be as complete as possible, thoroughly evaluating program design and implementation, and addressing sustainability questions as addressed through Les Aspin's "regular" program cycle. The final report will take stock of Les Aspin's "Refresher for Program Alumni" activity and provide conclusions, among others, about this activity, the conceptual approach, and its effectiveness for assuring sustainability. C.5 KEY PERSONNEL & CORE STAFF A two to three-member Evaluation Team is envisioned for the conduct of this evaluation. The following is illustrative of the types of skills required for the completion of the evaluation: 1. One Level 1 Democracy and Governance Analyst: This individual will be responsible for managing and coordinating the overall evaluation process. They will also be responsible for the overall compilation of the final Evaluation Report and be the principal point of contact between the Evaluation Team and USAID. In addition to demonstrated experience in Africa and with anti￾corruption programming, he/she should have a broad range of subject matter knowledge and experience to complement the other members of the Team. 2. No more than two additional Democracy and Governance Analysts at Levels 2 or 3 to complete the evaluation team. This/These individuals should have demonstrated background(s) and skill sets necessary to address the purposes of this task order and conduct required activities (described above in Sections C.2 and C.3, respectively). 41 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report ANNEX B: PROGRAM SCHEDULES AND AGENDAS USAID ASSISTED LES ASPIN CENTER DEMOCRACY AND GOVERNANCE TRAINING PROGRAM (WEST AND EAST AFRICA: ANTI-CORRUPTION AND GOOD GOVERNANCE) WASHINGTON, DC USA JUNE – JULY 2011 ABSTRACT: The training program will use seminars, lectures, site visits and skill-building workshops to explore, discuss, analyze and develop strategies to make private and public sector entities and institutions more accountable in performance, management, and public service delivery settings. Thus, the training will provide participants with skill-enhancement training on how corruption can be managed, and with what tools, in order to allow private and public sector entities to function more effectively in answer to and in consonance with the demands of the public trust. The training will be conducted from MAY 29, 2011– JULY 2, 2011. The program involves a total of 16 government and civil society personnel from Ghana, Uganda, Nigeria, Kenya, Mali and Tanzania. WEEK 1: MAY 29, 2011 – JUNE 5, 2011 ACCRA-GHANA MONDAY – MAY 30, 2011 - 9.00 AM – 11.30 AM: DR. EMMANUEL GYIMAH BOADI (EXECUTIVE DIRECTOR, CDD GHANA): “LEADERSHIP CHARACTERISTICS THAT CONTRIBUTE TO THE EFFECTIVENESS OF ANTI-CORRUPTION POLICIES AND STRATEGIES” - 2.00 PM – 4.30 PM: PROF. SAMUEL N. WOOD (CIVIL SERVICE COMMISSION, ACCRA): “MANAGING CORRUPTION EFFECTIVELY IN THE PUBLIC SECTOR: BEST LESSONS IN THE ENFORCEMENT OF PROFESSIONAL ETHICS STANDARDS” TUESDAY – MAY 31, 2011 - 9.30 AM – 11.00 AM: OFFICIAL OPENING OF PROGRAM – US EMBASSY, ACCRA - 1.30 PM – 3.30 PM: DR. AWEDOBA KANLISI (UNIVERSITY OF GHANA INSTITUTE OF AFRICAN STUDIES, ACCRA): “CHALLENGES TO ACCOUNTABILITY IN GOVERNANCE PRACTICES: A CLOSER LOOK AT ETHNOCENTRICISM AND ITS IMPACT” - 4.00 PM – 6.00 PM: MICHAEL KWAME BOADI (PUBLIC AGENDA, ACCRA): “HOW TO DEVELOP AND USE COMMUNICATIONS STRATEGIES FOR EFFECTIVE ANTI￾CORRUPTION CAMPAIGNS” 42 WEDNESDAY – JUNE 1, 2011 - 10.00 AM-12.00 PM: SESSION AT THE GHANA PARLIAMENT (ACCOUNTABILITY FUNCTIONS OF PARLIAMENT: CASE - ROLE OF COMMITTEES) - 1.30 PM – 3.30 PM: ROSALINE B. OBENG-OFORI (CONSULTANT, GENDER AND HUMAN RESOURCE DEVELOPMENT, ACCRA): “CONTRIBUTION OF BEST LESSONS IN GENDER PROGRAMMING TO ANTI-CORRUPTION ADVOCACY” - 4.00 PM – 6.00 PM: DR. DOMINIC AGYEMAN (PROFESSOR, DEPARTMENT OF SOCIOLOGY, UNIVERSITY OF CAPE COAST): “HOW TO USE EDUCATION AWARENESS STRATEGIES TO IMPROVE CIVIC CAPACITY IN THE MANAGEMENT OF CORRUPTION” THURSDAY – JUNE 2, 2011 - 9.00 AM – 12.30 PM: MARIAN KUBABOM (DEVELOPMENT INFORMATION CONSULT, ACCRA): “ANTI-CORRUPTION IMPLEMENTATION IMPACT: THE USE OF STRATEGIC PLANNING TECHNIQUES” - 2.00 PM – 4.30 PM: MARAIAN CONTD: “ANTI-CORRUPTION IMPLEMENTATION IMPACT: THE USE OF STRATEGIC PLANNING TECHNIQUES” FRIDAY – JUNE 3, 2011 - 9.00 AM – 11.00 AM: ATO KWAMENA DADZIE (JOURNALIST, JOY FM RADIO, ACCRA): “HOW TO UNDERTAKE ETHICS-BASED MEDIA REPORTING: EXPERIENCES FROM THE FIELD” - 1.30 PM – 3.30 PM: ABDULLAH MUHAMMAD (SERIOUS FRAUD OFFICE, ACCRA): “HOW TO BUILD AND IMPLEMENT EFFECTIVE ANTI-CORRUPTION COALITIONS” WEEK 2: WASHINGTON, DC MONDAY – JUNE 6, 2011 - 3.30 pm: ARRIVAL: DULLES AIRPORT (KLM) TUESDAY – JUNE 7, 2011 - 10.00 am – 3.00 pm: SITE VISIT: HISTORICAL SITES AND MONUMENTS: FOUNDATIONS OF DEMOCRATIC DEVELOPMENT-A VIEW THROUGH REAL SYMBOLS - (MODERATED BY CHRISTOPHER MURRAY-LES ASPIN CENTER - 3.30 pm – 6.30 pm: WELCOME BRIEFING – LES ASPIN CENTER WEDNESDAY – JUNE 8, 2011 43 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report - 9.30 am – 11.30 am: CHRISTOPHER MURRAY (LES ASPIN CENTER): POLITICAL CULTURE AND ACCOUNTABILITY SYSTEMS: THE US DEMOCRATIC EXPERIMENT’ - 11.30 am – 12.30 pm: DISCUSSION - 12.30 pm– 2.00 pm: LUNCH - 2.00 pm – 4.30 pm: REV. TIMOTHY J. O’BRIEN, Ph.D. (LES ASPIN CENTER): “INTEREST GROUP POLITICS AND CORRUPTION CONTROL: THE US EXPERIENCE” THURSDAY – JUNE 9, 2011 - 9.30 am – 12.30 pm: SESSION AT THE US CONGRESS (TOUR AND DISCUSSION ON CONGRESSIONAL LEADERSHIP AND GOVERNMENT OVERSIGHT- COORDINATED BY MATT JORGENSEN, OFFICE OF SENATOR BARBARA BOXER, D-CA) - 12.30 pm – 2.00 pm: LUNCH - 2.00 pm – 4.00 pm: RIELLE MILLER-GABRIEL (LOCKHEED MARTIN): “HOW TO IMPLEMENT ORGANIZATIONAL INTEGRITY PROGRAMS” - 4.00 pm – 5.00 pm: DISCUSSION FRIDAY – JUNE 10, 2011 - 9.30 am – 11.30 am: WILLIAM NYARKO (TECRON TECHNOLOGIES, ALEXANDRIA, VIRGINIA): “GOVERNMENT—MEDIA-CIVIL SOCIETY COLLABORATION IN CORRUPTION CONTROL” - 11.30 am – 12.30 pm: DISCUSSION - 12.00 pm – 1.30 pm: LUNCH - 1.30 pm – 4.00 pm: DISCUSSION AND WORKSHOP: “HOW TO FRAME ACCOUNTABILITY SYSTEMS”/ ”OVERVIEW OF PARTICIPANT ORGANIZATIONAL INTEGRITY SYSTEMS” SATURDAY – JUNE 11, 2011 - DEPARTURE TO WISCONSIN WEEK 3: WISCONSIN (MARQUETTE UNIVERSITY-MILWAUKEE) SUNDAY – JUNE 12, 2011 - CULTURAL EXPERIENCE: MILWAUKEE (VISIT TO SITES OF HISTORICAL AND CULTURAL IMPORTANCE IN MILWAUKEE) MONDAY – JUNE 13, 2011 - 9.00 am – 12.30 pm: RANA ALTENBURG (VICE PRESIDENT, PUBLIC AFFAIRS, MARQUETTE UNIVERSITY): “LOBBYING IN A UNIVESITY CONTEXT”/KATHRYN 44 HEIN (ASST. DIRECTOR, LES ASPIN CENTER AT MARQUETTE UNIVERSITY CAMPUS)”HOW TO CONDUCT ELECTION CAMPAIGNS AND MANAGE FRAUD IN ELECTIONS PROCESSES” (DAVID STRAZ JR HALL RM 250) - 12.30 pm - 2.00 pm: LUNCH - 2.00 pm-4.30 pm: SESSION AT COURT ROOM WITH JUDGE PAUL VAN GRUNSVEN “JUDICIAL INTEGRITY AND CORRUPTION CONTROL” - (MILWAUKEE COUNTY COURT HOUSE) TUESDAY – JUNE 14, 2011 - 9.00 am – 12.00 pm: EVA SOEKA (DIRECTOR, MARQUETTE UNIVERSITY CENTER FOR DISPUTE RESOLUTION, MILWAUKEE): “DISPUTE MANAGEMENT AND NEGOTIATION TECHNIQUES” )(DAVID STRAZ JR HALL RM 250) - 12.00 pm – 1.30 pm: LUNCH - 1.30 pm – 4.30 pm: EVA SOEKA AND CHRISTINE HARRIS TAYLOR: FAMILY DISPUTE AND MEDIATION PROCESSES (STATE OF WISCONSIN)(DAVID STRAZ JR HALL RM 250) WEDNESDAY – JUNE 15, 2011 - 9.00 am – 12.30 pm: EVA SOEKA AND HOWARD BELLMAN: “MANAGING DISPUTES AND NEGOTIATION IN POLICY SETTINGS” - 12.30 pm – 2.00 pm: Lunch - 2.30 pm – 4.30 pm: EVA AND BELLMAN CONTD. - (DAVID STRAZ JR HALL RM 250) THURSDAY – JUNE 16, 2011 - 9.00 am – 12.00 pm: DR. TOBY PETERS (SNR. VICE PRESIDENT FOR ADMINISTRATION, MARQUETTE UNIVERSITY, MILWAUKEE, WI) “LEADERSHIP MODELS IN SUPPORT OF ACCOUNTABILITY SYSTEMS: TRANSFORMATIONAL AND SERVANT LEADERSHIP” - 12.00 pm – 1.30 pm: LUNCH - 2.00 pm – 4.30 pm: DR. MADELINE WAKE (UNIVERSITY PROFESSOR, MARQUETTE UNIVERSITY COLLEGE OF NURSING): “CHANGE MANAGEMENT TECHNIQUES FOR SOCIAL CHANGE” - (DAVID STRAZ JR HALL RM 250) 45 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report FRIDAY – JUNE 17, 2011 - 9.00 am – 12.00 pm: SESSION WITH BILL DREW ON MILWAUKEE AREA COMMUNITY ECONOMIC REVITALIZATION INITIATIVES - 12.00 pm – 1.30 pm: Lunch - 2.00 pm – 4.30 pm: SESSION AT MILWAUKEE MEDIUM SECURITY DETENTION CENTER: “ACCOUNTABILITY SYSTEMS AND THE ADMINSTRATION OF JUSTICE￾THE WISCONSIN PENAL PROCESS” SATURDAY – JUNE 18, 2011 - Departure – Washington, DC SUNDAY – JUNE 19, 2011 - FREE WEEK 4: WASHINGTON, D.C. MONDAY – JUNE 20, 2011 - 9.30 am – 12.00 pm: CHARLES LEWIS (PROFESSOR, INVESTIGATIVE JOURNALISM WORKSHOP, AMERICAN UNIVERSITY, WASHINGTON, D.C.): “INVESTIGATIVE JOURNALISM TECHNIQUES” - 12.00 pm – 1.30 pm: LUNCH - 1.30 pm – 4.30 pm: DISCUSSION AND WORKSHOP: “HOW TO SUSTAIN MORAL COURAGE IN SUPPORT OF ANTI-CORRUPTION INITIATIVES”. TUESDAY – JUNE 21, 2011 - 9.30 am - 12.00 pm: WENDY POND (OFFICE OF GOVERNMENT ETHICS-OGE, WAHSINGTON, D.C.): “THE OFFICE OF GOVERNMENT ETHICS AND THE ENFORCEMENT OF CODES OF PROFESSIONAL CONDUCT FOR GOVERNMENT EMPLOYEES” - 12.00 pm – 1.30 pm: LUNCH - 1.30 pm – 4.30 pm: WORKSHOP: “HOW TO INCREASE COOPERATION AND REDUCE NON-COOPERATION IN CORRUPTION CONTROL INITIATIVES” WEDNESDAY – JUNE 22, 2011 - 9.00 am – 12.00 pm: ROBERT LEVENTHAL (US STATE DEPARTMENT ANTI￾CORRUPTION, INTERNATIONAL CRIME AND NARCOTICS PROGRAMS): “THE USE OF ANTI-CORRUPTION STRATEGIES TO CONTROL TRAFFICKING IN PERSONS, ECONOMIC FRAUD, AND INTERNATIONAL CYBER CRIME” 46 - 12.00 pm – 1.30 pm: LUNCH - pm – 4.30 pm: DR. LOUIS WRIGHT (PROFESSOR, DEPARTMENT OF POLITICAL SCIENCE, HOWARD UNIVERSITY, WASHINGTON, DC): “ETHICS-BASED LEADERSHIP: THE CIVL RIGHTS EXPERIENCE” THURSDAY – JUNE 23, 2011 - 9.30 am – 11.30 am: AARON PETERS (ACCOUNTANT, PRICEWATER COOPERS, WASHINGTON, DC): “HOW TO USE AUDIT TECHNIQUES TO DEVELOP AND IMPLEMENT SOCIAL AUDIT AND MANAGEMENT SYSTEMS” - 11.30 am – 12.30 pm: Discussion - 12.30 pm – 1.30 pm: Lunch - 1.30 pm – 4.30 pm: TAINA ALEXANDER (INTERACTION, WASHINGTON, DC): “HOW TO IMPROVE ACCOUNTABILITY SYSTEMS IN NGO ACTIVITIES”: THE MODEL OF INTERACTION” FRIDAY – JUNE 24, 2011 - 9.00 am – 12.00 pm: SESSION AT THE WHITEHOUSE: TOUR AND DISCUSSION WITH WHITEHOUSE AFRICA AFFAIRS STAFF: US-AFRICA DEVELOPMENT POLICY-HOW THE US ENGAGES GOVERNMENT AND CIVIL SOCIETY ENTITIES TO SUPPORT EFFECTIVE ACCOUNTABILITY PRACTICES IN AFRICA” - 12.00 pm – 1.30 pm: LUNCH - 2.00 pm – 4.30 pm: ALEKSANDR SHKOLNIKOV (CENTER FOR INTERNATIONAL PRIVATE ENTERPRISE-CIPE, WASHINGTON, DC): “HOW TO USE ANTI-CORRUPTION AND CORPORATE GOVERNANCE STRATEGIES TO IMPROVE BUSINESS PRACTICES” SATURDAY & SUNDAY– JUNE 25 & 26, 2011 - FREE WEEK 5: WASHINGTON, DC MONDAY – JUNE 27, 2011 - 9.00 am – 12.00 pm: : PROGRAM DEVELOPMENT WORKSHOP: ‘STRATEGIES IN PROGRAM DEVELOPMENT: ENHANCING SKILLS IN GRANTCRAFT, GRANTERS￾GRANTEES COLLABORATION (CASE STUDY: THE OPEN SOCIETY-WASHINGTON, DC AND PACT-USA) - 12.00 pm – 1.30 pm: DISCUSSION AND WORKSHOP: WIDENING THE FIELD OF NETWORKS: HOW TO USE NETWORK MAPS 47 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report TUESDAY – JUNE 28, 2011 - 9.0 am -12.00 pm: DR. KWAME ACQUAAH (ATTORNEY AND INTERNATIONAL DEVELOPMENT CONSULTANT, BAILEY LAW GROUP, WASHINGTON, DC): “PUBLIC SECTOR MANAGEMENT STRATEGIES AND CORRUPTION CONTROL” - 12.00 pm – 1.30 pm: Lunch - 1.30 pm – 4.30 pm: ACQUAAH CONTD WEDNESDAY – JUNE 29, 2011 - 9.00 am – 4.30 pm: PROGRAM DEVELOPMENT AND PLANNING CONTD - 6.00 pm – 83.0 pm: PROGRAM DEVELOPMENT AND PLANNING CONTD THURSDAY – JUNE 30, 2011 - 9.00 am – 4.30 pm: DEPARTURE PREPARATIONS - 6.00 pm – 8.00 pm: CLOSING – LES ASPIN CENTER FRIDAY – JULY 1, 2011 - FREE SATURDAY – JULY 2, 2011 - DEPARTURE-DULLES AIRPORT CONTACT: DR. CEPHAS LEREWONU COORDINATOR OF AFRICA PROGRAMS LES ASPIN CENTER FOR GOVERNMENT 502 EAST CAPITOL STREET, N.E., WASHINGTON, DC 20003 TEL. 202-544-6140 (OFFICE) & 202-320-7327 (CELLPH) Email: cephas.lerewonu@marquette.edu 48 LES ASPIN CENTER CAPACITY DEVELOPMENT TRAINING ON ANTI-CORRUPTION AND GOOD GOVERNANCE REFRESHER COURSE FOR WEST AFRICA PROGRAM GRADUATES ACCRA – GHANA: JULY 19-30, 2011 DRAFT PROGRAM: TUESDAY – JULY 19, 2011 - 4.00 pm: Arrival of participants: Ghana, Mali and Nigeria - 6.00 pm: Introductory Briefing WEDNESDAY – JULY 20, 2011 - 9.00 am – 11.00 am: Dr. Emmanuel Gyimah Boadi (Executive Director, CDD-Ghana): Key Note Presentation on “Outlook on Governance and Accountability Trends in Africa: New Opportunities for Improved Anti-Corruption Advocacy” - 11.00 am-12.30 pm: Discussion on key lessons for improved anti-corruption advocacy - 12.30 pm-2.00 pm: Lunch - 2.00 pm – 4.30 pm: WORKGROUPS: Discussion of organizational and country contexts that provide opportunities/challenges to the effective implementation of anti-corruption initiatives (Case: Video presentation: “Enemies of the Nation”) THURSDAY – JULY 21, 2011 - 9.00 am – 12.30 am: COUNTRY TEAMS: Best Practices in the design and implementation of anti-corruption initiatives: Experiences from the field - 12.30 pm – 2.00 pm: Lunch - 2.00 pm – 4.30 pm: COUNTRY TEAMS: Best Practices in the design and implementation of anti￾corruption initiatives: Experiences from the field FRIDAY – JULY 22, 2011 - 9.00 am – 12.00 pm: WORKGROUP: How to build effective civil society coalitions: The use of power analysis and drivers of change models - 12.00 pm – 1.30 pm: Lunch - 1.30 pm – 2.00 pm: WORKGROUP: The design and implementation of sustainable integrity networks SATURDAY – JULY 23. 2011 - Cultural visits SUNDAY – JULY 24, 2011 49 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report - FREE MONDAY – JULY 25, 2011 - 9.00 am – 11.00 am: How to exercise collaborative leadership for development impact - 11.00 am – 12.30 pm: Discussion on key leadership lessons (Revisit: Moral Courage in Leadership Practices) - 12.30 pm - 2.00 pm: Lunch - 2.00 pm – 4.30pm: WORKGROUPS: Planning for Effective Networks: The Use of the Appreciative Inquiry Model TUESDAY – JULY 26, 2011 - 9.00 am – 12.00 pm: WORKGROUPS: Discussion on tools and platforms for network and partnership sustainability - 12.00 pm – 1.30 pm: Lunch - 1.30 pm – 4.30 pm: WORKGROUP S - CASE: How to use new information and communications technology systems to improve sustainability and impact of advocacy networks WEDNESDAY – JULY 27, 2011 - 9.00 am – 12.30 am: WORKGROUPS: Design of frameworks for sustained and collaborative partnerships and alliances: Use of network analysis and network maps - 12.30 am – 2.00 pm: Lunch - 2.00 pm – 4.30 pm: WORKGROUPS: Implementation of frameworks for sustained partnerships and alliances for performance impact THURSDAY – JULY 28, 2011 - 9.00 am – 12.30 pm: WORKGROUPS: Future directions for post-program sustainability: Overview of action plans - 12.30 pm – 2.00 pm: Lunch - 2.00 pm – 4.30 pm: WORKGROUPS: Future directions and action plans contd. FRIDAY – JULY 29, 2011 - Departure 50 LES ASPIN CENTER CAPACITY DEVELOPMENT TRAINING ON ANTI-CORRUPTION AND GOOD GOVERNANCE REFRESHER COURSE FOR EAST AFRICA PROGRAM GRADUATES NAIROBI - KENYA: AUGUST 22-31, 2011 PROGRAM SCHEDULE VENUE: SAFARI PARK (THIKA ROAD – NAIROBI-KENYA) MONDAY – AUGUST 22, 2011 - 4.00 pm: Arrival of participants: Kenya, Uganda, Tanzania - pm: Introductory Briefing: Overview of Refresher Course - (Video presentation for comparative analysis: “Fighting Corruption in Yemen”) TUESDAY- AUGUST 23, 2011 - 9.00 am – 11.00 am: Key Note Presentation: PROF. PATRICK L.O. LUMUMBA (DIRECTOR, KENYA ANTI-CORRUPTION COMMISSION, NAIROBI): “Outlook on Governance and Accountability Trends in Africa: New Opportunities for Improved Implementation of Anti￾Corruption Initiatives and Advocacy” - 11.00 am-12.30 pm: Discussion on key lessons for improved anti-corruption advocacy - 12.30 pm-2.00 pm: Lunch - 2.00 pm – 4.30 pm: WORKGROUPS: Understanding emerging trends in the nature of corruption and corruption control strategies: An overview of anti-corruption protocols within organizational and country contexts (Facilitated by RONALD WANYAMA, KENYA ANTI-CORRUPTION COMMISSION) WEDNESDAY – AUGUST 24, 2011 - 9.00 am – 12.00 pm: WORKGROUP: ANNE GATHUMBI (PROGRAM OFFICER, HEALTH AND HUMAN RIGHTS, OPEN SOCIEITY INITIATIVE FOR EAST AFRICA-OSIEA, NAIROBI - KENYA 2003): “How to build effective civil society coalitions: The use of power analysis and drivers of change models” - 12.00 pm – 1.30 pm: Lunch - 1.30 pm – 4.30 pm: WORKGROUP (COUNTRY TEAMS): The design and implementation of sustainable integrity networks THURSDAY – AUGUST 25, 2011 - 9.00 am – 12.30 am: COUNTRY TEAMS: Best Practices in the design and implementation of anti￾corruption initiatives: Experiences from the field - 12.30 pm – 2.00 pm: Lunch - 2.00 pm – 4.30 pm: COUNTRY TEAMS: Best Practices in the design and implementation of anti￾corruption initiatives: Experiences from the field 51 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report FRIDAY- AUGUST 26, 2011 - 9.00 am – 12.30 pm: PETER KIAMA (EXECUTIVE DIRECTOR, INTERNATIONAL MEDICO-LEGAL UNIT, NAIROBI): “How to exercise collaborative leadership for development impact” - 12.30 pm – 2.00 pm: Lunch - 2.00 pm – 4.30 pm: Discussion on key leadership lessons (Revisit: The practice of “Moral Courage” in the exercise of leadership)(Case: Alumni Leadership Stories Panel: HON. BEATRICE ATIM - UGANDA-2008/ BLAK ODANYIRO – KENYA 2000/ OLE KAUNGA – KENYA 2000/ HASSAN OMAR – KENYA 1999). SATURDAY –AUGUST 27, 2011 - 9.00 am – 12.00 pm: WORKGROUPS: Discussion on tools and platforms for network and partnership sustainability (Facilitated: THEODUL MACHA, PACT-TANZANIA – TZ 2008: USE OF NETWORK ANALYSIS AND NETWORK MAPS) - 12.00 pm – 1.30 pm: Lunch - 1.30 pm – 4.30 pm: WORKGROUP – SALUM AWADH (EXECUTIVE DIRECTOR, SUSTAINABLE SOLUTIONS CONSULT, DAR ES SALAAM – TZ 2005): “How to use new information and communications SUNDAY - AUGUST 28, 2011 - Individual activity MONDAY – AUGUST 29, 2011 - 9.00 am – 1.00 pm: WORKGROUPS (COUNTRY TEAMS): How to implement frameworks for sustained partnerships and alliances for performance impact - 1.00 pm – 2.30 pm: Lunch - 2.30 pm – 4.30 pm: WORKGROUPS: CONTD (COUNTRY TEAMS): How to implement frameworks for sustained partnerships and alliances for performance impact TUESDAY – AUGUST 30, 2011: - 9.00 am – 12.30 pm: WORKGROUPS: Future directions for post-program sustainability: Overview of action plans - 12.30 pm – 1.30 pm: Lunch - 1.30 pm – 4.30 pm: WORKGROUPS: Future directions for post-program sustainability: Presentation of action plans WEDNESDAY – AUGUST 31, 2011 - DEPARTURE 52 CONTACT: DR. CEPHAS LEREWONU COORDINATOR OF AFRICA PROGRAMS LES ASPIN CENTER FOR GOVERNMENT 502 EAST CAPITOL STREET, N.E. WASHINGTON, DC 20003 USA TEL. 202-544-6140 & 202-320-71327 EMAIL: cephas.lerewonu@marquette.edu 53 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report ANNEX C: DOCUMENTS REVIEWED • Performance indicators for the Les Aspin program • Names of the 2010 program participants • East and West Africa program schedule for 2010 • Participant Data from 2001-2010 • Contact info for 2010 program participants • Timeline for 2011 programs • Activity Progress Report: East and West Africa 2010 • Les Aspin Center Progress Report 2007 • Program Activity Brief: West and East Africa Programs 2007 (including appendices on program schedule and profile of students) • Les Aspin history 1996-2003 • Activity Brief 2008 • List of participants 2008 • West and East Africa Program Schedules 2008 • Program Activity Brief 2008 • DG Officers Contact List (email only) • Modification of Les Aspin Center’s contract • Action Memorandum from Wade Warren in 2002 re: Les Aspin program (hard copy scanned into pdf) • 2001 Les Aspin Proposal (hard copy scanned into pdf) • FY 2010 Les Aspin Program Budget Hard Copy Documents received from Les Aspin: - DESCRIPTION OF STRATEGIC OBJECTIVES - DESCRIPTION OF TRAINING FRAMEWORK - RESULTS FRAMEWORK - CONCEPTUAL FRAMEWORK (ACCOUNTABILITY IN RESPECT OF ANTI- CORRUPTION PROGRAMS) - GUIDE FOR SELECTION OF CANDIDATES (ALL 6 PROGRAM COUNTRIES) - GRADUATE ACTIVITIES - SAMPLE BUDGET - COPIES OF REPORTS - CONTACT LISTS - LIST OF ALUMNI WHO HAVE ATTENDED AND COMPLETED PROGRAMS - PROGRAM RESOURCES - SAMPLE LESSONS 54 ANNEX D: DATA COLLECTION INSTRUMENTS Survey Questions for Les Aspin Alumni Introduction: Social Impact and Management Systems International have been commissioned by USAID to conduct a final evaluation of the Les Aspin Governance Program in East and West Africa. Our evaluation team is trying to understand the impact this project has had on good governance and anti-corruption issues in West and East Africa, the sustainability of the program design, the cost effectiveness of the program and ultimately whether or not this is the appropriate mechanism for obtaining the desired results. We are asking all Les Aspin alumni from 2001 through the present to complete this survey. The survey should take no more than 15 minutes. Most questions offer the respondent an opportunity to provide clarification and more in depth answers. Please answer these open ended questions as they will provide more meaningful input to this process. All comments will remain anonymous. Thank you! Demographics 1. Name: (optional) 2. Gender: Male/Female 3. Age: (From Selection of ranges) 4. Sector involved in during program participation: (Drop down menu of Media, NGO, Government, and Other) 5. Sector involved in currently: (Drop down menu of Media, NGO, Government, and Other 6. Current job/position 7. Current organization 8. Have you worked in government service? a. Currently b. Formerly c. Never 9. What year did you participate in the Les Aspin Program? a. Select from Drop Down menu of 2001-2010 10. Current Country of residence 11. Current city of residence 12. Country of residence during Les Aspin participation if different from current residence 13. City of residence during Les Aspin participation if different from current city 55 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report Effectiveness of approach Select your level of agreement with the following statements: Strongly Agree Agree Neutral Disagree Strongly Disagree 1. The structure of the Les Aspin program is effective in teaching about accountability and good governance systems. 2. The content of the Les Aspin program is effective in teaching about accountability and good governance systems. 3. The program segments (Africa, Washington, DC and Milwaukee, WI) have clearly stated objectives and learning outcomes. 4. There is a good balance between academic presentations and field visits? 5. The program is the right length of time. 6. The program was applicable to my work environment. Comment Box provided: “Please comment on any of your answers” 7. Did the Les Aspin program teach you new skills or information regarding good governance? Yes or No If yes, what types of skills and information have you learned from Les Aspin? 8. Did the Les Aspin program teach you new skills or information regarding accountability? Yes or No If yes, what types of skills and information have you learned from Les Aspin? 9. Do you find the Les Aspin program to be an effective tool in achieving more accountable governance systems in your country? Yes /No Please explain 10. Are there any elements of the Les Aspin program that you would have improved? a. Overall Schedule b. Faculty c. Content d. Africa based segment e. Washington based segment f. Wisconsin based segment g. Other: h. Please explain : Alumni – Impact 1. Do any of the following skills and information that you obtained through the Les Aspin program apply to your daily work? (select all that apply) Negotiation and dispute management Coalition building Ethics based decision making 56 Leadership skills Effective public mgmt None of the above Other Comments: 2. Are there additional skills that would have been helpful to learn for your work? Yes/No Please explain which would have been helpful to learn 3. Did you create a post-program implementation plan while a Les Aspin participant? Yes or no If yes, were you able to implement your plan? Options: (Mainly, Some, a Little, not much, Still working on it) 4. Since participating in Les Aspin, have you tried to implement improvements in your organization to strengthen good governance and accountability? Yes, No, Not Yet, Give specific examples 5. How successful were you in implementing good governance and accountability improvements into your organization? a. Very successful b. Somewhat successful c. Not very successful d. Not successful 6. What were/are the major obstacles to change? Comment__________ 7. Did the Les Aspin program assist in building your capacity for improving your home organization’s efforts to address corruption issues? a. Yes b. Somewhat c. Not a all d. N/A Please explain 8. Did the Les Aspin program assist in building your capacity for improving your home organization’s efforts to address governance issues? a. Yes b. Somewhat c. Not at all d. N/A Please explain 57 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report Alumni impact on larger-level accountability in governance systems 1. Do you believe that you have been able to make a greater impact on larger-level accountability and governance, such as policy reform, in your country since your participation in the Les Aspin program? Yes /No Please explain Sustainability 1. Do you communicate with other Program alumni in your country? 2. Do you communicate with other program alumni in other countries? 3. Do you communicate with Les Aspin staff or faculty? a) Once a week or more b) 2-3 times a month c) Once a month d) 5-10 times a year e) 2-5 times a year f) Once a year g) Less than once a year h) Never. Comments welcome Note: In Survey Monkey this question will be all in one organized field 4. Have you collaborated on any governance related projects or work with any other program alumni since your participation? Yes/No Please explain. 5. Do other Les Aspin alumni coordinate any follow-up activities: Yes/No /Don’t Know If yes, please describe 6. What changes would you recommend that might improve the long-term utility and effectiveness of the program? Comment box provided 7. Would you recommend participation in the Les Aspin program to your colleagues? Yes/No Comments Welcome 8. Have you participated in any similar fellowships to study accountability and good governance in the US or elsewhere? 58 Yes/No If yes, please name and describe how they compare with the Les Aspin program 9. What are the strongest elements of the Les Aspin program? (Please rank 1-8) a. Fostering linkages and understanding b/t Africans and the US in accountability and good governance b. Negotiation and dispute mgmt c. Coalition building d. Ethics based decision making e. Leadership skills f. Effective public mgmt g. Developing civil society institutions to support anti-corruption efforts h. Developing media institutions/organizations to support anti-corruption efforts 10. What do you see as the weaknesses of the Les Aspin Program? 11. Are there any changes you would make to improve it, or do you think the program is no longer needed? i. No longer needed_______ j. Continue if changes are made __________(be specific) k. Continue as it is. l. Overall general comments 59 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report Survey Questions for USAID DG Officers and FSNs Introduction: Social Impact and Management Systems International have been commissioned by USAID to conduct a final evaluation of the Les Aspin Governance Program in East and West Africa. Our evaluation team is trying to understand the impact this project has had on good governance and anti-corruption issues in West and East Africa, the sustainability of the program design, the cost effectiveness of the program and ultimately whether or not this is the appropriate mechanism for obtaining the desired results. We are asking all USAID DG Officers and Foreign Service Nationals who have been involved with the Les Aspin program anytime from 2001 through the present to complete this survey. The survey should take no more than 15 minutes. Most questions offer the respondent an opportunity to provide clarification and more in depth answers. Please answer these open ended questions as they will provide more meaningful input to this process. All comments will remain anonymous. Thank you! Demographics 1. Check the USAID Mission you worked in while involved with the Les Aspin Program: Drop down menu with the 6 target countries and then a write-in option for “other” 2. What year or years did you work on the Les Aspin program? (select all that apply) 2001 – 2011 will be listed as options 3. How well do you remember the Les Aspin program? a. Vaguely b. Somewhat c. Well d. Very well e. N/A. Effectiveness of approach 4. Do you find the Les Aspin program to be an effective tool in achieving more accountable governance systems in the target countries? Options: Yes, Somewhat, Not Effective, Don’t Know AND If Yes, please give specific evidence or example of “more accountable governance”. 5. Were you involved in the recruitment or the selection process? Options: Yes or No 60 7. Does the Les Aspin recruitment process target appropriate candidates for an accountability and good governance fellowship? Options: Yes, No, or Don’t Know AND Comment Box provided 8. How was the program advertised (select all that apply) Newspaper, Television, radio, Civic associations or NGO networks, other 9. How effective was the program for the following participants: Highly Effective Effective Minimally Effective Not Effective Don’t Know Government officers Media representatives NGO representatives Other: (write in option) Alumni - Impact 10. Do you communicate with Les Aspin Program alumni? (A) Once a week or more (B) 2-3 times a month (C) Once a month (D) 5-10 times a year (E) 2-5 times a year (F) Once a year (G) Less than once a year (H) Never. Comments welcome 11. Have you seen positive improvement in any of the following skills or information used by the alumni in their work since their return? (select all that apply) Negotiation and dispute mgmt Coalition building Ethics based decision making Leadership skills Effective public mgmt Other: Comments: 12. Are there other skills or information that might have been equally or more useful for supporting the participant’s efforts to improve accountability and good governance than those listed in question 6? Yes or No If yes, please give examples 13. Can you identify specific instances where Les Aspin alumni helped USAID in strengthening good governance and accountability efforts in your country? Yes or No If yes, please explain how; If no please explain why not? 61 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report 14. On average, did you observe an improvement in participants’ capacity for improving their home organization’s efforts to address corruption and governance issues after completing the program? (A) Yes (B) Somewhat (C) Not at all (D) N/A. Please explain. 15. Please add any general comments on differences you were able to see in the skills or knowledge of alumni upon their return. Alumni impact on larger-level accountability in governance systems 16. In your opinion, have Les Aspin alumni been able to impact larger-level accountability issues in the country’s governance such as policy reform? Yes or No If yes, please explain how 17. In your opinion, have Les Aspin alumni been able to impact larger-level accountability systems within their own organization (NGO, media, Govn’t entity)? Yes or No Comments: 18. On average, did you observe an improvement in participants’ capacity for positively affecting governance and accountability at a high level in your country? (A) Yes (B) Somewhat (C) Not at all (D) N/A. Please explain. Cost effectiveness 19. Please select your level of agreement with the following statement: 20. The Les Aspin program is a cost-effective means for building African leaders’ commitment to strengthened accountability and good governance. (A) Strongly Agree (B) Agree (C) Neutral (D) Disagree (E) Strongly Disagree 21. If Disagree, can you cite a program you believe to be more cost effective in achieving similar objectives with similar clientele? 62 Sustainability 22. Do Les Aspin alumni currently coordinate any of the following follow-up activities: Yes No Don’t Know Training sessions led by Les Aspin alumni on accountability Creation of units of integrity by Les Aspin alumni Newsletter Alumni Get Together Networking Opportunities Alumni Email List￾Serve/website Post-Experience Evaluation or Discussion Other: 23. Do you believe follow-up activities would increase the impact of the Les Aspin program? Yes or No Please explain 24. Do you believe that the Les Aspin program is designed to create a long-term impact? Yes or No Please explain 25. Overall, does the Les Aspin fellows program support your DG strategy and work? Yes or No Please explain 26. Based on the Les Aspin program’s accomplishments to date, if the program were to end, what would be the effect if any on strengthening leadership for anti-corruption, good governance and improved accountability? a. Significant effect b. Some effect c. Marginal effect d. Little or no effect e. Don’t know f. N/A Comment: 63 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report 27. Please provide any comments on additional ideas you have on sustainability of the Les Aspin program. 28. What are the strongest elements of the Les Aspin program? (Please rank 1-8) No particularly strong elements Fostering linkages and understanding b/t Africans and the US in accountability and good governance Negotiation and dispute mgmt Coalition building Ethics based decision making Leadership skills Effective public mgmt Developing civil society institutions to support anti-corruption efforts Developing media institutions/organizations to support anti-corruption efforts 29. From your experience and perspective, should USAID continue to fund the Les Aspin program? Yes, No, Only if major changes were made. 30. If Les Aspin were to continue, what changes would you like to see in the program content or management to make it more effective? 64 ANNEX E: TABLE OF PERFORMANCE INDICATORS MARQUETTE UNIVERSITY LES ASPIN CENTER FOR GOVERNMENT USAID ASSISTED DEMOCRACY AND GOVERNANCE PROGRAM (EAST & WEST AFRICA: ANTI-CORRUPTION AND GOOD GOVERNANCE) USAID PERFORMANCE INDICATORS: FY 2010 Indicator FY 10 Result FY 11 Target Civil Society NUMBER OF CIVIL SOCIETY ORGANIZATIONS USING USG ASSISTANCE TO PROMOTE POLITICAL PARTICIPATION - FY 2010: Total of 32 (15 Government and 17 Civil Society; Male 15 and Female 17; Government male 5, government female 8; civil society male 9 and civil society female 8) personnel from East and West Africa: Total of 32. For FY 2011, an additional number of 16 personnel expected from East and West Africa (Note: Expected total of personnel will be 16 because the first cycle of regular programming will involve a combination of 16 participants from East and West Africa. The second cycle of training in 2011 will be devoted to alumni refresher training) NUMBER OF CIVIL SOCIETY ORGANIZATIONS USING USG ASSISTANCE TO IMPROVE INTERNAL ORGANIZATIONAL CAPACITY – FY 2010: Total of 32 (15 government and 17 civil society; Male 15 and Female 17; Government male 5, government female 8; civil society male 9 and civil society female 9) personnel from East and West Africa: Total of 32. For FY 2011, an additional number of 16 personnel expected from East and West Africa (Note: Expected total of personnel will be 16 because the first cycle of regular programming will involve a combination of 16 participants from East and West Africa. The second cycle of training in 2011 will be devoted to alumni refresher training) NUMBER OF CSO ADVOCACY CAMPAIGNS SUPPORTED BY USG - Total of 17 (Civil society female 8, civil society male 9)(civil society personnel/organizations - since the trained members of civil society are also basically engaged in advocacy at the local community and national levels). For FY 11, the number cannot be predetermined until final pool of selected candidates is confirmed. NUMBER OF POSITIVE MODIFICATIONS TO ENABLING LEGISLATION/ REGULATION FOR CIVIL SOCIETY ACCOMPLISHED WITH USG ASSISTANCE NUMBER OF PEOPLE WHO HAVE COMPLETED USG ASSISTED CIVIC EDUCATION PROGRAMS*** NUMBER OF PARTICIPANTS IN USG-FUNDED PROGRAMS SUPPORTING PARTICIPATION AND INCLUSION OF TRADITIONALLY MARGINALIZED ETHNIC MINORITY AND/OR RELIGIOUS MINORITY GROUPS (FY 10: Total of 17 (8 female and 9 male) personnel from civil society from East and West Africa. These personnel and their organizations work mostly with grassroots, under￾served, local communities. Indicator here will reflect both the organizations and individual personnel. The individual trained represents their organization and hence adds to institutional capacity as well). For FY 11, the number cannot be predetermined until final pool of selected qualified candidates is confirmed. NUMBER OF INDEPENDENT AND DEMOCRATIC TRADE/LABOR UNIONS SUPPORTED BY USG TO PROMOTE INTERNATIONAL CORE LABOR STANDARDS 65 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report NUMBER OF USG ASSISTED CIVIL SOCIETY ORGANIZATIONS THAT ENGAGE IN ADVOCACY AND WATCHDOG FUNCTIONS - FY 10: Total of 17 (8 female and 9 male) civil society personnel from East and West Africa. For FY 11, the number cannot be predetermined until final pool of selected qualified candidates is confirmed. NUMBER OF POLICIES THAT HAVE BEEN INFLUENCED BY CSOS MEDIA NUMBER OF MEDIA OUTLETS THAT RECEIVED USG-SUPPORTED TRAINING TO PROMOTE FINANCIAL SUSTAINABILITY - FY 10: Total of 6 (3 female and 3 male) media personnel from private media. For FY 11, the number cannot be predetermined until final pool of selected qualified candidates is confirmed. NUMBER OF POSITIVE MODIFICATIONS TO ENABLING LEGISLATION/ REGULATIONS FOR MEDIA DRAFTED WITH USG ASSISTANCE NUMBER OF NON-STATE NEWS OUTLETS ASSISTED BY USG - FY 10: Total of 6 (3 female and 3 male) personnel from private media. For FY 11, the number cannot be predetermined until final pool of selected qualified candidates is confirmed. NUMBER OF JOURNALISTS TRAINED WITH USG ASSISTANCE - FY 09: Total of 6 (3 female and 3 male) personnel from private media. For FY 10, the number cannot be predetermined until final pool of selected qualified candidates is confirmed. NUMBER OF MEDIA CIVIL SOCIETY ORGANIZATIONS AND/OR SUPPORT INSTITUTIONS ASSISTED BY USG - FY 10: Total of 6 (3 female and 3 male) personnel from private/civil society media organizations. For FY 11, the number cannot be predetermined until final pool of selected candidates is confirmed. NUMBER OF GOVERNMENT MEDIA RELATIONS STAFF TRAINED WITH USG ASSISTANCE NUMBER OF PEOPLE WHO RECEIVED INTERNET ACCESS AS A RESULT OF USG FUNDED PROGRAMS 66 ANNEX F: FY 2010 LES ASPIN PROGRAM BUDGET FY 2010 ITEMIZED PROGRAM COSTS PROGRAM COSTS % OF BUDGET East Africa Project Coordination $ 15,750.00 2.87% East Africa Travel Costs for Trainees $ 128,500.00 23.45% West Africa Project Coordination $ 12,300.00 2.24% West Africa Travel Cost for Trainees $ 130,300.00 23.78% Total Programs $ 286,850.00 52.34% Non-Program Costs $ 261,163.00 47.66 % FY 2010 ITEMIZED NON-PROGRAM COSTS NON￾PROGRAM COSTS PERCENT OF BUDGET Facilities and Admin Costs $ 96,774.00 17.66% Project Administration- Marquette U. $ 85,640.00 15.63% Fringe Benefits $ 23,979.00 4.38% Staff Travel $ 30,466.00 5.56% Office Supplies $ 4,750.00 0.87% Speaker Fees $ 17,000.00 3.10% Conclusion and Follow-up $ 2,554.00 0.47% Total Non-Program Costs $ 261,163.00 47.66% Total Program Costs $ 286,850.00 52.34% 67 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report ANNEX G: SUCCESS STORIES AND ANALYSIS The following success story analysis is based on the “Graduate Post-Program Activities” document provided by the Les Aspin Center. The following success story was written by the evaluation team based on interviews with alumni in Ghana. SUCCESS STORIES ACTIVITY NAME GENDER LOCATION CLASS YEAR OF ACTIVITY Chosen to join new Public Procurement Authority Yero Diallo Male Mali 2002 2009 Kadiatou Konate Female Mali 2003 2009 Arandane Toure Male Mali 2006 2009 Developed Code of Ethics for home organization Richard Ellimah Male Ghana 2009 2009 Participated in radio discussion on ethics, accountability, and public service Invited to work as Elections Observer and was trained as an ECOWAS elections observer Joyce Amegah Female Ghana 2008 2009 Developed program to train legislators on how to implement anti-corruption programs, currently seeking funding Babatunde Oluajo Male Nigeria 2009 2010? Developed a program on ethics education for students Lawrencia Abakisi and other alumni in northern Ghana Female Ghana 2002 2010? Developed a program to create a state￾level (Lagos) government accountability office, modeled on the US GAO Judge Joseph Oyewole Male Nigeria 2008 2009? Designed and implemented an 18 month program for journalists on how to do good quality reporting of economic and budgetary issues Gabriel Aduda Male Nigeria 2006 Wrote a book entitled “Good Governance and Procurement: An Appraisal of Public Procurement Mechanisms in Nigeria” Produced a video on how to manage corruption in Nigeria, currently seeking funding for mass distribution Patricia Oloyede Female Nigeria 2005 Initiated program on political Dr. Mustapha Male Nigeria 2006 68 accountability and human rights abuses for some key institutions in Kano region Ismail ACTIVITY NAME GENDER LOCATION CLASS YEAR OF ACTIVITY “attributes her ability to function as Special Advisor on Social Orientation to skills developed during program Chinmwe Anowai Female Nigeria 2003 Developed “gender, good governance, and political corruption” program that she piloted in Bamako, then expanded to 3 other regions through WB funding Kadia Traore Female Mali 2005 Initiated consultancy program in Bamako on Ethics, Business, and Management; counsels and trains businesses on ethics issues Fatoumata Lagdaf Maiga Female Mali 2005 Designed a youth against corruption program for secondary schools that has been implemented in 2 schools Alasidongor Bawa Baluri Ghana 2006 2006 Developed and implemented program on gender, good governance, and leadership for young women in secondary schools, funded by ACTION￾AID, and later incorporated into regular programming of his home organization due to its success; Kwaku Antwi￾Boasiako (with support from fellow alum Florence Dennis Male Ghana 2005 2005- present Used advocacy skills to launch appeal to assist hunger victims in Niger, spurred state institutions to make donations Helped to reorganize the Ghana Anti￾Corruption Coalition into a more active body Florence Dennis Female Ghana 2005 2005, ongoing Designed and implemented a national competition in anti-corruption advocacy for school children ages 12-18 2007 Organized training seminars for youth leaders in tertiary educational institutions With other grads in northern Ghana, developed programs in accountability for Budget and Finance Officers Samuel Awugah Male Ghana 2004 Collaborates as a resource person with the Ghana Integrity Initiative to train community leaders Developed, launched, and now runs the “Enquirer” newspaper in Ghana Raymond Archer Male Ghana 2002 2004- present 69 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report ACTIVITY NAME GENDER LOCATION CLASS YEAR OF ACTIVITY Developed and is implementing radio programs on good governance, sponsored by ACTION-AID Margaret Akparibo Female Ghana 2002 ? – Present Works with the student council of her school to train students in accountability and good governance practices Serves as a resource person on training programs conducted by the Ghana Integrity Initiative Developed program on ethics and anti￾corruption for training civil servants who are studying at the Kenya Institute of Administration, currently refining with help of LA Koki Robi Ochieng Female Kenya 2008 Expanded his network of civil society and government practitioners and uses those networks in hi s work on the resolution of the Northern Uganda situation involving the Lord’s Resistance Army James Nyeko Male Uganda 2008 2008- present Completed a Code of Ethics for personnel at her home organization, as well as for civil servants and government personnel in Uganda Angela Bafokuzara Female Uganda 2005 Wrote and published articles in the Uganda “Monitor” newspaper on the current attempts to control corruption in Uganda Alex Atuhaire Male Uganda 2005 Selected to be Special Editor covering the meeting of the Heads of Government of Commonwealth Countries in Kampala, Uganda 2007 Working with home organization to develop and implement a Citizen’s Charter for the improvement of services delivery in the public sector Charlote Myesigye Female Uganda 2003 2009 Established a citizens forum called “Public Information and Public Accountability Forum”, currently in second phase of implementation ? - present 70 ACTIVITY NAME GENDER LOCATION CLASS YEAR OF ACTIVITY Worked with the research dept of Uganda parliament to identify and correct gaps in the implementation of a leadership code for public officials Hon. Geoffrey Ekanya Male Uganda 2003 Assists with development of codes of ethics for personnel in government ministries and client service charters in offices of ministries Assisted the Directorate of Ethics and Integrity to streamline the Ugandan Anti-Corruption Strategy for 2005-2007 2004-2005 Completed blood drive initiative Kenya graduates ---- Kenya ---- 2007 and 2008 Spearheaded grassroots civic education program on community oversight strategies for Community Development Funds Job Sirari Male Kenya 2005 Assisted Kenya Anti-Corruption Campaign Steering Committee to conduct civic education programs in Bungoma District 2007 Along with other members of Anti￾corruption Campaign Steering Committee, launched civilian oversight committees in 9 provinces in Kenya from 2006-2007 Rose Arungu Olende Female Kenya 2005 2006-07 Designed and organized a conference on “anti-corruption and good governance in East Africa” for members of the Law Society of Kenya (over 500 lawyers) Justus Munyithya Male Kenya 2003 2007 Involved in the design and implementation of “Citizen Score Cards” Sophie Ngugi Female Kenya 2005 2009 At university created student organization on governance called “Young Diplomats Forum” and organized conferences on the role of youth in the fight against corruption Dennis Kimani Male Kenya 2004 Was promoted to head the UNDP Somalia desk initiative due to her additional training from LA Charity Ndegwa Female Kenya 2004 71 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report ACTIVITY NAME GENDER LOCATION CLASS YEAR OF ACTIVITY Started and ran a community development organization called “Sustainable Solutions Consultancy” and conducts capacity development training programs for young students Salum Awadh Male Tanzania 2005 Implements leadership training programs for civic leaders from local community wards Currently (Oct 2009) developing online internet portal to facilitate networking and joint programs by East African LA alumni 2009 Expanded her work to include consultancy on “good governance and accountability” for community development institutions Magreth Henjewele Female Tanzania 2005 2005? Selected by the Ethics Resource Center of Washington, DC to represent Tanzania in a global integrity program in Istanbul Romuald Rwechungura Male Tanzania 2003 Used training experience and resources to develop programs to improve judicial integrity among judges and court personnel in Mali Mamadou Diarra Male Mali 2005 Collaborating with USAID/Mali and the US Embassy in Mali to undertake discussions on corruption through video-conferencing (with Moriba Diarra) Mohamed Thiam Male Mali 2005 Selected by Oxfam to spearhead seminars on transparency initiatives concerning the extractive industries in Mali Use resources of training to assist in development and launching of an anti￾corruption strategy for the Anti￾Corruption Commission of Uganda Geoffery Rwakabale Male Uganda 2005 Was hired as Executive Secretary of Transparency International-Tanzania, runs and manages its programs and focuses on assisting the new Tanzania Parliament with parliamentary oversight programs William Kitima Male Tanzania 2003 72 Out of the forty alumni featured in these success stories: • Ten received career advancement or were invited to participate in a program as a result of participating in the program • Twenty-five developed and/or implemented a program • Ten implemented changes in home organization • Six used media (radio, video, newspaper) to educate and address the public on anti-corruption • Ten trained others on corruption and good governance • Five engaged in or implemented a forum or network to discuss corruption and good governance • Four success stories included Les Aspin alumni interacting with each other to make their impact • Three mentioned receiving support from Les Aspin in the implementation or design of their post￾program activity 7 9 6 8 6 4 Success Stories by Country Mali Ghana Nigeria Kenya Uganda Tanzania 24 16 Success Stories by Gender Male Female 0 2 4 6 8 10 12 14 16 2002 2003 2004 2005 2006 2007 2008 2009 Success Stories by Year of Participation # of success stories 73 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report Les Aspin Fellows Program Success Story Prepared by the Evaluation Team Transparency and Good Governance on the Local Level: District Assembly of Walewale Perhaps one of the most interesting elements of the Les Aspin program is the diversity of participants who come not just from civil society but also from government. When these individuals work together, they can implement good governance reforms that are sustainable and effectively implemented. The District of Walewale is a case in point where three Les Aspin alumni have worked together to effect major reforms in public accountability and transparency in local government operations. Nabla Sulemana was a school headmaster and presiding member of the District Assembly of Walewale in northern Ghana. He participated in the Les Aspin program in 2004 and learned about effective communication strategies in the political process. He also learned about a different model of governance which was open to the public and transparent in its operations. He learned this through observation of the US political system and the transparency that is provided to the public on expenditure of government funding. He also learned that political leaders should not just communicate through a one-way channel but should have a two-way channel of communication with the public. Upon his return to Ghana, Nabla was appointed District Chief Executive (DCE) of the Walewale District Assembly in 2005. As DCE, he completely transformed the way in which the public interacted and communicated with the District Assembly. The public, for the first time in the history of this assembly, had access to information on government projects in the district. He credits the new approach he instituted to what he learned about effective communication and transparency on the Les Aspin program. He chaired all public fora and earned the respect of his DA members in the way he ran meetings so much so that people asked him to chair other public meetings and fora. A subsequent alumni on the program, Diana Ndego (2010) works with the Social Enterprise Development (SEND) Foundation to train citizens on how to review district-level public budgets and expenditures. She noted that Walewale is known as one of the most transparent and open DAs in the north, primarily due to Nabla’s leadership. She recounted a story of a citizen’s group that was tracking the Highly Indebted Poor Countries (HIPC) school feeding program funds in Walewale. They approached the DA staff to review the accounts for the feeding program and were told that because the person in charge was not in the office, they could not see the accounts. They then called the DCE, Nabla Sulemana, who subsequently called the person in charge back to the office to open all accounts on the feeding program immediately which was done. Another Les Aspin alumni, Issifu Sulemana (2010) with Zasilari Ecological Farms Project (ZEFP) a civic organization in Walewale, upon his return from the program, requested a review of the accounting systems in the DA. His request was agreed to by the DCE, Nabla Sulemana. The idea for Issifu’s request came directly from his meeting with the State of Wisconsin’s Accountability Board which opened his eyes to a truly open and accountable way to run government. The result of this review allowed for full public access to the tracking of funds and regular review of accounts by civil society. This confluence of individuals linked to the running of the Walewale District, both within the government and civil society, allowed for a unique set of leaders that has resulted in a more transparent and accountable way to run local government. As Ghana continues to strengthen the local government system, the District Assemblies become ever more important in their ability to budget and plan in a way that is open and accountable to the public they serve. Walewale can serve as a model for other districts certainly but the model itself of a two-way communication system between government and civil society was learned through the Les Aspin program. 74 ANNEX H: LES ASPIN CENTER’S RESPONSE TO DRAFT EVALUATION REPORT This final report takes into account the Les Aspin Center’s response by correcting any factual errors as well as clarifying our analysis and conclusions where we felt it was warranted. USAID has concurred with our changes in this final version of the report. We are providing the full text of the Les Aspin Center’s response to the draft evaluation report here for further reference. THE LES ASPIN CENTER FOR GOVERNMENT WASHINGTON, DC USA RESPONSE TO DRAFT EVALUATION REPORT ON THE LES ASPIN CENTER CAPACITY DEVLOPMENT TRAINING PROGRAMS ON ANTI-CORRUPTION AND GOOD GOVERNANCE FOR WEST AND EAST AFRICA 1. Introduction: We acknowledge the observations of the evaluation team concerning the Les Aspin Center’s capacity development training programs for East and West Africa. Upon a closer review of the conclusions of the evaluations, we appreciate comments that have highlighted the positive dimensions of the training programs, and indications of some challenges that will need improvement. Obviously there will always be opportunities for refinements and improvements as a result of an assessment effort. We have noted several positive observations. However, these have been presented in such a way as to be clouded by other overly negative statements and conclusions. We reject as unacceptable several generalizations, misrepresentations and contradictions in the conclusions of the evaluators regarding the effectiveness of the training programs and whether or not they have created any positive impact for the participants and in states of affairs in the respective program countries. While the program graduates and USAID personnel who participated in giving feedback in the evaluation have indicated positive results from the training programs, the conclusions of the evaluators are at odds with the findings. In what follows, we would like to clarify specific issues that the evaluation report has highlighted. 2. The Strategic Objective of USAID/Africa Bureau and Les Aspin Training Framework: The capacity development training programs have been designed around a key strategic objective of USAID (Africa Bureau) under which the training is envisioned: Over the years, we have noted an evolution in USAID approaches to the management of corruption in private and public sector entities, with more emphasis now being placed on accountability in governance practices as well as in foreign development assistance practices with respect to Africa. Further, there is the emphasis on “investing in people” who can acquire the necessary advocacy and other skills to work to make governments and holders of the public trust more responsive (accountable) to the requirements and expectations of this trust. The Les Aspin approach has therefore responded to address how this objective may be facilitated in order to achieve the governance assistance aims of USAID. Overall, the LAC has attempted to design and implement a training response that makes accountability a central concern in the efforts to manage corruption in private and public sector settings in the participating program countries. The training also considers that on account of the complex nature and many levels of the incidences of corruption, it will be more appropriate to adopt a holistic and multi-dimensional approach. The training framework outlines the different dimensions of the training response. In setting the context for evaluating the effectiveness and impact of the training, the evaluators have misrepresented the training framework and the change and results assumptions of the program. The evaluation assessment ignores the key elements of the framework, which guide training activities, training modules and training results. While the evaluators assume impact by considering an exaggerated eradication of corruption in the program countries, we consider impact rather within the context of utilizing capacities acquired from the training program to manage corruption by improving accountability systems in 75 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report management, performance, and service delivery settings in the program countries. Further, we have stated that one of the training objectives will be to assist participants to improve their on-the-job responsibilities. We will recapitulate the main elements of the framework as follows: 3. Focus of Training Program: One of the main goals of the training program will be to explore, discuss, analyze and develop ways to make private and public sector entities and institutions more accountable to the public trust in performance, management and public delivery settings. As one dimension of improving accountability systems, the training will provide participants with skill-enhancement training on how corruption can be managed, and with what tools, in order to allow private and public sector entities to function more effectively in answer to and in consonance with the requirements of the public trust. -Activities: In a period of six weeks, in an experiential and multi-dimensional format, participants and their trainers will use the following instruments to develop and increase knowledge and skills: a. Formal lectures will analyze, for example: - i. Diverse democratic cultures and political cultures – ideas, ideologies, values, norms, institutions, symbols, worldviews, practices, social interactions; - ii. The nature of governance arrangements, systems and practices; - iii. The nature of accountability and accountability systems in the context of democratic and political cultures, accountability lapses and incidences of corruption, mechanisms to manage accountability and accountability lapses, directions for policy reform and implementation to improve accountability in governance systems - iii. Border area themes concerning personal, organizational or institutional capacity development relevant to effective and accountable performance, management, and service delivery in consonance with the requirements of the public trust(e.g., leadership, professional ethics, advocacy, lobbying, dispute management and negotiation, risk analysis and management, change management, e-governance and e-admin, elections management, building and sustaining cooperation, coalition building and networking, etc) b. Group discussions and workshops will involve case studies, role plays, skill development exercises, strategy development exercises, program development and planning exercises, etc c. Cross-sectoral, cross-country, cross-cultural, interpersonal interactions and learning d. Site Visits: Institutions and sites relevant to thematic focus as in section “a” above. Sites will include the following: i. Federal level government agencies such as the US Congress, the General Accountability Office (GAO), Office of Government Ethics (OGE), the Commerce Department, USAID, etc.; ii. State and local level government institutions such as the Wisconsin State Assembly, State Accountability Board of Wisconsin, Milwaukee County Common Council and Board of Supervisors, County Ethics Board, County Judiciary, Milwaukee City administration and agencies, etc; Private sector organizations, such as lobbying and advocacy organizations, universities and other educational institutions, etc. e. Training materials (books, etc) Under this section, we want to clarify the following other structural elements: - Lectures are seminar-format presentations by a lead facilitator, including discussions between facilitator and participants (Duration of session is 2hrs – 2hrs 30 mins). - Site Visits: First, these include visits to organizations and institutions outside of classroom settings. Here, sessions integrate observation and seminar type lectures-discussions with duration of between 2.00 hrs – 3hrs. Although we normally request and encourage interaction between participants and staff of organizations visited, some organizations limit sessions to only designated staff from sections relevant to the theme of the visits. Other organizations invite colleagues to join sessions with the participants. In cases where site visits are to historical sites, monuments, museums, etc, there are facilitators who accompany participants to give real time briefings, commentary and explanations to enhance learning. We are aware of participant expectations that they normally would want to have more interaction with staff of organizations during the site visits. However, the organizations also accommodate the Les Aspin groups within the context of the organizational schedules, availability of 76 staff, and sector assignments. We note that the evaluators incorrectly lump all site visits as cultural visits. - Skill-building workshops include specific skill development sessions. Examples include strategic planning, strategy development concerning procurements, how to build cooperation and prevent non-cooperation, how to develop and implement organizational integrity systems, how to develop communication and branding strategies, negotiation strategies, public administration strategies, etc (Duration of sessions be 2.00 hrs – one whole day or 2 days). - Group discussions: These are sessions normally facilitated by Les Aspin staff or a selected participant to discuss further previously completed lectures, address issues of concern to participants, logistics issues or general program review (Session duration-1 hr 30 mins – 2.00 hrs) - Change Theory: The evaluators have neglected to recall the key change assumption behind the training, but rather, in a simplistic way attempted to read into the program “a change theory” as they explain in the report. Bearing in mind that the entire training is built around an experiential and collaborative learning format, we restate an extract of the main change assumptions that guide the training as follows: [Assumptions behind the cooperative learning model of the training program: The training program will integrate certain key assumptions in the attempts to develop and enhance learning and skills. An important requirement for fostering social change, through collaborative social engagement with others, is to do “productive bridging”. That is, there is the need for participants in this training program to build on the foundations of personal, institutional, national and other value frameworks. These will enable participants to “widen the circle” of interactions to embrace others, to recognize and respect people’s distinct identities and experiences, to enter into new worlds and respond to new stories, take on new worlds of meaning, acquire a sense of shared purpose, allow best energies, positive potential and strengths to come to the fore, feel and contribute to a larger purpose by pooling together individual energies into a common power (See, Paul Loeb, “Widening the Circle”, in Loeb, Soul of a Citizen: Living with Conviction in a Cynical Time, NY: St. Martins Griffin, 1999, Chapter 9). Thus, participants will envision new worlds of meaning together (Dream), examine more creative possibilities (Discover), develop appropriate responses and strategies (Design) and create social change through more effective accountability systems (Destiny) (See, Diana Whitney and Amanda Trosten Bloom, The Power of Appreciative Inquiry: A Practical Guide to Positive Change, 2002] (Extract taken from text of program framework) Indeed, as envisioned, it is the broad range of interactive processes of the entire training experiences that is capable of generating the learning outcomes that participants have acknowledged they acquired from the training program. Also, the program endeavors to include groups of participants who can bring together a multidimensional, multi-cultural, and multi-sectoral pool of resources, and experiences in order to increase collaborative learning. Therefore, to isolate learning outcomes and impact outside of this comprehensive interactional and transformational context will be to misplace training impact on a program that is rather different than the stated focus of the Les Aspin experience. The program has consistently emphasized outcomes of program deliverables to include “knowledge, experiences, and other skills”. The multi-dimensional approach to the management of corruption frames the implementation of the program activities cited above under the five sections of the training framework. Thus, the various activities and tools provided in the training schedules are planned to target and respond, in a holistic way, to the many dimensions of addressing corruption in its complex forms and settings. Therefore, we expect that any inquiry on impact should embrace these key areas, which should be seen to function together. In fact, there is a results framework that also explains the expected outcomes from these interactional processes. The framework also depicts linkages among program goals, program activities/participant activities and program outcomes/participant outcomes. As explained in program briefs, the results indicators will be the post-program activities of program graduates and the social benefit to the graduates, to their organizations, and to their communities, as a result of the implementation of the post￾program activities. 77 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report It is gratifying to note that the findings of the evaluation report of the observations of participants concerning results acquired bear out the consistency between the training framework, training deliverables and envisioned results. As such, it is unacceptable that the evaluators claim weak links between training activities (inputs) and results (achievement of program objectives). The Les Aspin model is not a “variable-oriented” model, but a “holistic” approach. Knowledge, experiences and other skills acquired must be assessed against the background of the entire training experience as conveyed by the training framework. Although, the training functions with the view to contributing to a long-term vision of reduced (not eradicated) corruption in program countries, the training provides capabilities and other tools for participants to be able to accomplish outcomes (in the immediate and foreseen long terms) in their post-program undertakings in management, performance and service delivery settings. As the current findings of the evaluation stands, participants note that they are using the training experiences to improve practices back in their communities. In regards to the evaluator’s conclusion, that considering the current state and levels of corruption in program countries, the Les Aspin graduates cannot be considered to have created any impact, we clarify as follows. Depending on context, the role of other stakeholders and the influence of the external environment where each participant lives and works, ability to apply learned skills and accomplish impact may not be immediately apparent. Further, in their totality, it may or may not be immediately apparent that the activities of the Les Aspin trained cohorts in the respective program countries are considered to be contributing to global/macro changes in the wider societies as a whole. Nonetheless, this does not invalidate the personal claims of the program graduates that they are making modest impact gains around them as a result of applying knowledge and skills obtained from the programs they attended. Thus, the evaluators ironically indicate that some participants come on the program who are already persons of integrity in their fields of work. Such personnel have precisely been admitted to the programs because of their standing and their ability to add on more learning in order to continue to excel. The evaluators also grossly exaggerate when they make comparisons between corruption control in other countries and the participating program countries. Due regard must be given to governance practices that tied are to or greatly influenced by complex socio-cultural contexts, unique local histories, and other factors that make these countries more amenable to corruption control measures than obtains in the Les Aspin program countries. In other countries, such as Rwanda and Bostwana, a whole range of accountability measures have been institutionalized with high levels of commitment to enforcement, public education, and heavy investments in institutional and human capacity development, from the political leadership down through public and private sector institutions. It is widespread knowledge, and there is consensus of conviction, that anti-corruption work cannot be seen separately from other social, economic and political reforms, as well as heavy investments in human and institutional capacity development. It is therefore an over expectation, and without consideration for the unique environments of the Les Aspin program countries, to conclude that since corruption is still high in these countries, the value of the Les Aspin graduates is negligible in respect of contributions to US government goals to reduce corruption and improve governance. The corruption vulnerabilities in these contexts still make capacity development of government and civil society personnel more compelling. In spite of a fleeting observation in the evaluation report of participants who have indicated they have used the program to transition into leadership positions, the report does not appreciate and reflect the depth of impact of the Les Aspin programs or appreciate the depth of expertise we have developed (as also exemplified by the expertise reflected by graduates during the recent alumni refresher programs in Ghana and in Kenya). We must emphasize that it takes the exercise of leadership to advance governance reform. The Les Aspin Center highlights the leadership potential of several of the graduates who have gone into leadership positions and credit their advancement to the participation in the training programs. Several examples include the following: Hon. Anne Makinda, Speaker of the National Parliament of the Republic of Tanzania; Hon. Anna 78 Kilango Malecella (Member of Parliament); Emelia Arthur, current Deputy Minister of Western Region (Ghana - She was appointed as a result of a performance grounded in integrity and accountable in the discharge of duties and for standing up to an entire District Assembly that attempted to indulge in bribery as a condition for civic appointments). Others include H.E. Kirimi Kaberia, who has gone on to be the current Ambassador of the Republic of Kenya to Brazil; High Court Judges Laurenda Owusu (Accra-Ghana) and Jennifer Tagoe (Kumasi-Ghana); Judge Pauline Nyamweya (recently appointed, Nairobi-Kenya), Comercial Court Judge Mariam Cisse Coulibaly (Bamako-Mali), 3 Les Aspin graduates who are members of the National Procurements Board of Mali, Gabriel Aduda in the administration of the Economic and Financial Crimes Commission of Nigeria, Florence Jaoko and 3 other Les Aspin graduates who are Commissioners in the Kenya National Human Rights Commission, Hon.Akua Dansuah who went from a District Chief Executive too become current Minister of Tourism in Ghana, among many others. It is also informative to mention one more example of a Les Aspin program graduate who exemplifies possibilities of progressive long-term impact. This gives a comparative perspective of alumni cohorts of other years not covered within the program years being evaluated. Apollos Machira, a Kenyan graduate of the Les Aspin programs (1997) started a Center for Dispute Management in Nakuru (Kenya) after completing the programs. He modeled the Center from sessions on dispute management presented at Marquette University during the training for his class. Through the activities of Apollo’s Center, he included seminars on conflict management for military personnel from Kenya and other countries in the East Africa sub-region. These seminars later resulted into the development of a full-blown course in Military Science for military personnel. It is currently being taught in Edgerton University (Nakuru-Kenya). The above examples are indicators of long-term opportunities of broader macro-level impact. As Les Aspin graduates continue to transition into leadership and other positions of influence, at higher levels of authority in the public and private sectors in program countries, we expect long-term results to emerge. 4. Training Activities: We would also clarify and elaborate on aspects of the training activities as follows i. Information for Prospective Training Candidates: It is baffling that participants will claim ignorance of program focus and objectives before they attend the program. Normally briefing materials are sent to participating USAID country missions to be used in orientation briefing sessions for participants before travel for the programs. The materials detail specific program objectives, training activities, as well as expectations of post-program actions (These bear presenting here in full as follows): - Introduction: The Les Aspin Center Government will conduct a capacity development training program on “anti￾corruption and good governance” for a total of 16 government and civil society personnel from West Africa (Ghana, Mali and Nigeria) and a total of 16 personnel from East Africa (Kenya, Tanzania and Uganda). The program will be completed at the Les Aspin Center, Washington, DC (USA). - Program Objectives: Generally, one of the principal goals of the training programs is to provide participants with knowledge and skills in preventive, proactive, multi-dimensional, institutional support strategies for improving accountability and other good governance practices in the program countries. Therefore, program objectives include the following: i. To train participants to understand accountability and good governance systems as well as the necessary strategies to promote accountability in the public and private sectors. Participants and their presenters will use comparative analysis to discuss and explain how governance systems and bureaucracies work and what the fundamental requirements are for effective governance approaches. Participants will also use diagnostic research instruments to understand the dynamics of governance, the institutional bases of corrupt transactions, the impact of corruption, as well as the intricate intersection between corruption environments, corruption networks and governance systems. ii. Participants will learn how ethics infrastructure functions to improve integrity and accountability systems in the private and public domains. Assisted by specialists from specific government and private sector 79 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report institutions in the US, participants will discuss the nature of ethics infrastructure and how they ensure effective oversight and implementation of accountability in governance. iii. Participants will develop and implement country-specific strategies for the building of integrity and accountability systems in their respective countries. Participants will use the training experiences, knowledge and resources to develop specific strategies for managing and implementing effective accountability in management, performance and service delivery systems. - Training Activities: Specifically, the training will utilize rigorous seminar discussions, interactive lectures, strategy and skill building workshops, case studies, as well as hands-on, practical, experience, to provide a deeper understanding of the linkages among governance processes and accountability systems and how corruption functions as a key governance risk. The activities will aim at building and improving the knowledge and skills of participants at the individual participant level, and, through the participants, to improve institutional capacity and resources in support of the implementation of accountability initiatives. On the whole participants will learn how accountability standards are applied in the US context to address the complexity of accountability issues. - Training Cycle: Generally, the program involves six-weeks of rigorous training. This will be implemented in 2 phases for West and East Africa respectively. - Phase 1: Introductory Seminars and Orientation in one of the program countries. The orientation and seminar sessions will provide baseline knowledge of country-specific issues. Particularly, participants and the program presenters will examine the nature of accountability and governance systems and how these are managed through specific accountability improvement strategies in the democratic and cultural environments of the participating West and East African countries. Participants will also learn how to utilize the training resources and experiences to plan and develop results-based strategies for the implementation of accountability procedures. - Phase 2: Main program completed at the Les Aspin Center (Washington, DC, USA): Here, participants will learn how traditions of accountability function effectively and how leadership and professional responsibility compliance and enforcement standards are implemented. - Post-Program Activities of Participants: Participants are expected to develop and implement initiatives back in their respective countries after completing the training program. Among others, the following outputs by program graduates are envisaged during the post-program activity implementation period: - Through acquired knowledge and skills, participants will implement accountability strategies to improve the integrity and effectiveness of government and public sector management, performance and service delivery - Participants will use accountability strategies, professional skills informed by integrity standards, and other good governance skills learned, to improve on-the-task performance and management of job responsibilities - Participants will implement programs to help improve citizen oversight systems - Participants will implement programs to help streamline accountability approaches in governance practices and in the operations of public sector institutions - Participants will conduct training sessions on accountability approaches for colleagues and staff in departmental units - Participants will assist in developing and establishing units of integrity in the various departments. These units will function as operational and coordinating bodies to oversee and implement and manage accountability practices and processes within the respective departments - Participants will be integrated into the network of Les Aspin trained graduates from Western and Eastern Africa. 80 5. Training program, presentations, lectures and workshops: We do not accept the fact that the evaluators have aggregated the whole range of the programs into a one￾time observation of activities. This has led to generalizations that have undermined the value of contributions to the programs as implemented over the years. First, the training modules are considered within the pedagogical model of experiential learning. The assumption is that through the collaborative, multi-sector, multi-cultural interactions, new or additional knowledge, experiences and skills will emerge to expand the learning of participants in the program areas and other life contexts. The entire program experiences become very relevant and are designed to contribute to such holistic learning, transformation and acquisition of capabilities (program schedules reflect this multi-dimensional approach). Thus, any attempt to see one level of learning, skill or experience, different from the entire holistic development of new or additional ideas, perspectives, attitudes, experiences and skills, rather approaches the assessment too narrowly. Second, we acknowledge that there are diverse models of adult teaching and learning which all have different levels of effectiveness depending on context, issues, and pedagogical preferences. In order to ensure collaborative learning, the program providers established the primary learning format to be best conveyed by seminar style discussions. In these sessions, whether in class or on site visits, the program providers ensure that each presenter uses models that enhance learning. However, some lecturers tend to follow formats that best convey the issues that they are tackling at any one point in time. Even though the lecture presentation format is followed by some presenters, these lecturers have always endeavored to create room for extensive interactive discussions with participants and always within a time frame not less than 2 hr or 2 hr and a half sessions. However, the evaluators give the impression that participants were just nonresponsive and mute during discussions with only the opportunity to ask just questions and receive answers in return from presenters. The evaluators have never mentioned the quality and depth of participation and exchange that the participants demonstrate during every session that they have had with presenters. Whether in Africa, Wisconsin or Washington, DC, presenters have openly acknowledged they appreciate and cherish the learning they take away from the participants. It is hard to accept the observation that participants were normally considered or treated as merely “students”. Various other presenters have used different presentation formats such as power point, and have utilized diverse media such as video, audio, as well as online Internet tools from the Worldwide Web. All the same, there is no denying that different models could be explored to expand the quality of the presentation formats. Third, the claim that the training providers do not use professional trainers and rather rely heavily on program alumi is inaccurate and another unacceptable generalization. As training schedules indicate, the program presenters are specialists in knowledge and skill transfer, and with expertise in governance and other thematic areas. The evaluators will be grossly belittling the caliber of persons who contribute to the programs by claiming the program does not use “professional trainers”. This claim gives us a sense of a bias and stereotypical view about university institutions and “academicians” and “think tanks” who, ostensibly, are incapable of linking theories with practice. The whole Marquette university system is involved and contributes professional expertise. Also, the multi-disciplinary approach of the Les Aspin programs necessitates engaging presenters who are professionals and practitioners from government institutions, educational institutions, as well as from public and private sector organizations. With respect to alumni, it has always been the goal of the trainings to invite one or two alumni, who are also specialists in their fields of work or in specific thematic areas, to do presentations and facilitate discussions with the new groups of participants. The graduates bring their vast experiences to the discussion of issues, share best lessons from the field on what works in development and change initiatives, as well as create opportunities for the new participants to establish networking relations with other graduates who are working in the field. That a graduate lectures on the program or facilitates a workshop session for new graduates does not make the graduate any less a professional. In fact, these graduates are also normally seasoned trainers within civil society or government settings. It is also a claim by the evaluators that presenters have noted they have been contacted to do discussions without clear terms of reference. We would want to note that presenters who have normally been invited have always been given instructions on why they have been invited, the topic to discuss, types of participants 81 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report and backgrounds, expected learning outcomes, among other details (samples of presenter invitation letters can be made available if necessary). Details of letters of invitation vary with respect to whether we have a regular presenter or someone unfamiliar with the programs. It is also not uncommon, that in spite of a presenter receiving an invitation with terms of reference, when they arrive and meet the participants, they suddenly decide to refocus the discussion. This is an ongoing challenge that we are not able to predict and control ahead of time. Therefore, it is understandable here when participants note their reservations that they are not sure whether or not in one situation or the other presenters are given clear terms of reference for a presentation. 6. Program Structure and Programming Flexibility: The main thematic focus of the training programs has been anti-corruption and good governance. There is no denying the fact that within the development community, the consensus of thinking is that corruption in governance practices in Africa, and in other parts of the world, continues to be a central risk. It is therefore a given that this theme will constantly frame the focus of the Les Aspin capacity development training programs. Also, the multi-dimensional approach to the training continues to be a useful tool for its productive potential in addressing corruption as a complex, multi￾faceted, phenomenon that requires a holistic, multi-perspective, multi-sector and multi-disciplinary approach (as the “USAID Anti-Corruption Strategy of 2005” explains well). Further, there is the realization that the incidence of corruption was more insidiously subtle, with “many faces”, in complex cultures, social situations, and governance arrangements, in African countries. Therefore, we consider that instruments through which corruption could be managed in more creative ways would be through holistic models rather than “variable oriented” models alone (e.g., use of a singular instrument to counter bribery). The structure of the Les Aspin training programs derives from this approach. This has been the constant pillar of the training programs. However, individual training modules, and the diversity of training tools, are selected to reflect this holistic, multi-dimensional understanding of the “many faces” of corruption. In these regards, there has always been flexibility in expanding and adding on more creative tools, themes, approaches, and knowledge resources. This has been the case as the programs have evolved from 2001 up until now. We are constantly reviewing trends and developments in approaches to the management of corruption control and the promotion of democracy and good governance with respect to not only Africa but globally. Participants also bring along unique and diverse knowledge and experiences. These have contributed to the expansion and formulation of innovative themes and tools used over the history of the training (e.g., modules on “e-administration” through Information Technology platforms, “moral courage”, “ethics-based decision modeling”, “servant and transformational leadership”, “change management”, “cooperation models”, “rational choice and social choice models”, “lobbying for social change”, etc). It must also be noted that the programming contexts of Washington, DC and Wisconsin have afforded the programs, and participants who attend the programs, the opportunities to draw constantly from a rich variety of resources produced by a convergence of US domestic and international institutions and other global experiences in general. It is a reality that during training programs, an event is always taking place in the public domain domestically in the USA or internationally. Program sessions regularly benefit from these occurrences as opportunities for learning whether inside or outside of formal training sites (as when participants directly witnessed and discussed the impeachment of President Clinton; experienced and discussed the 2002 Election stalemate between President George H.W. Bush and Vice President Al-Gore; directly witnessed the funeral of President Ronald Reagan in Washington, DC, with the later discussions in class on presidential leadership in the US and the treatment of African presidents; discussion s with County government officials on pension scandals in County Government in Milwaukee, Wisconsin; discussion with Budget Committee members of the Wisconsin State Assembly on budget stalemates in State Government, experiences of the corruption scandals of Jack Abrahamoff and comparative discussions on massive corruption in Kenya exposed by the Whistleblower John Githongo, etc). One must admit that for any educational and formative experience, there will always be structural and thematic constants. Yet room must be created to account for new developments and emerging best lessons that can contribute to expand learning and the ongoing development of capacity. Regrettably, the evaluation 82 report shows a lack of appreciation for the programs strengths and ability to integrate creative tools and solutions. 7. Specification of Learning Objectives: During participant feedback in the course of the training, we do receive some isolated concern that learning objectives have not been fully specified by one or the other presenter. Generally, since the trainings combine formal lectures in class, site visit presentations, and practical workshops, there are a wide variety of styles and formats of presentation that lecturers use. In the majority of cases, lecturers present notes to participants during sessions (samples of presentation outlines with learning objectives have been made available to evaluators). Other presenters give lecture outlines but follow up with full texts of lectures, which are made available to participants. Where participants visit organizations on site, the presenters give comprehensive learning materials to participants with additional references to further resources on the Web sites of the organizations. Thus, participants are generally afforded opportunities to understand the thematic focus of presentations and expected learning benefits. 8. Action Plans for Participants: From the very onset of the first week of training participants have sessions in strategic development strategies. These provide the tools for developing further plans for post-program activities. Although, participants are encouraged to identify actionable activities to do after the program, there has been one challenge that has tended to deemphasize the complete development of action plans during actual training cycles. From programming experience, it has often been difficult for participants to determine or identify with precision, beyond generalities, what they plan to do after the training. This is because of the necessity also to consult with other stakeholders back in their respective countries on any activity that participants may have in view. However, the training sessions still include days when participants have the opportunity to discuss extensively how to develop program ideas and plans to implement the initiatives. During the training, there is one training session that is normally devoted to strategies on how to control corruption in order to improve public sector administration systems. Here the facilitator, who has expertise in public administration systems, does an excellent job to assist participants to draw from and analyze issues against the background of the entire training experiences. At the end of the sessions each participants is tasked to develop a program idea and action plan upon return and communicate back to the Les Aspin Center. Further, during the last week of each program session, we hold extensive discussions with participants on how to develop and implement action plans. We also discuss and provide written guides on how to develop programs, and how to develop effective program proposals to secure funding for program activities. We must indicate that some participants have normally communicated with the Les Aspin to share their action plans and to ask for further guidance on how to refine their planned activities. Others have developed proposals and secured funding to support program activities (We have submitted to the evaluators some samples of in-training action planning guides, as well as an example of a post-training funded program for one of the graduates). 9. Collaboration with other organizations such as NDI, NED, IRI, etc: Although the Les Aspin Center has not engaged NDI, NED or IRI in recent times, we are very familiar with their work. We have invited personnel from these organizations in the past to have discussion sessions with program participants. It is noteworthy, that during one such panel discussion on governance issues, rather than discuss the topic at issue, staff from NDI and IRI, who were leading the session, instead attempted to gather information from the participants about their countries. This upset the participants a great deal, So the decision was made to rather have participants engage members of these organizations who they knew back in their respective countries. We have however consistently had programs in the past with IFES (International Foundation for Election Systems) on anti-corruption measures and the integrity of electoral processes. We must also note that we currently continue to collaborate with the following institutions to train program participants. These include private and US government affiliated institutions, such as the Center for International Private Enterprises (CIPE) of the US Chamber of Commerce and also an affiliate of the NED group, the Securities and Exchange Commission (SEC), the Federal Elections Commission (FEC), the US State Department, the Office of General Counsel of the US Department of Justice, the Federal Elections Administration (FEA), the 83 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report Office of Government Ethics, as well as the BALDRIDGE Leadership Program of the National Institutes of Science and Technology (NIST). The Les Aspin programs have also included training from personnel of other US-based organizations (with domestic as well as international outreach). These have included Risk-Control Group, the American Bar Association (through its Africa section and Central and East European Law Initiative), the Center for Global Integrity, the Center for Public Integrity, the Ethics Resource Center, Lockheed Martin, Deloitte and Touche Tamasu, Pricewater Coopers, the International Union of the Conservation of Nature (IUCN), to mention these examples. We therefore consider it another generalization for the evaluators to claim that we do not collaborate with organizations such as NDI, IRI, or other. In fact, though we admit a value in the exchange of resources, we do not consider that affiliation with these organizations should be the basis for legitimacy of the Les Aspin programs, as the evaluators suggest. 10. Tools for program evaluation: The claim of the evaluators that the program does not do an in-program or follow up evaluation of the training sessions is unacceptable. Though not through the formal questionnaire instruments, we employ different tools within program cycles to understand training progress. The methodology we follow is drawn from an insightful change instrument and model called “The Power of Appreciative Inquiry”. This is based on the work of change experts Amanda Trosten Bloom, David Cooperrider, Diana Whitney. The model focuses on how to uncover positive change in persons (or organizations) and facilitate performance impact. The process uses a diversity of group discussion formats where members use question and answer style to elicit knowledge from each other. The goal is to determine skill and knowledge potential of group members, and use that as basis to develop other strategies to improve progress, impact or performance effectiveness in the ongoing life-cycle of an undertaking. Also, we have often drawn insights from these questions to facilitate discussion with the participants and inquire from them, and elicit responses to know where participants are as the training unfolds, how the training impacts them during the course of the program and at the end of training sessions. Through the use of this model in program review sessions, participants understand, determine and explain much more profoundly the meaning and outcomes of the program experiences for them. Participants are also able to explain ways in which learning has contributed to bringing out their best potential, and how to use this potential to enhance further program activities. Examples of the work of the authors (cited above) that we have relied on for insights on an inquiry and discussion format, include the following: David L. Copperidder, Sorensen Peter, Diana Whitney & Theresa Yaeger, "Appreciative Inquiry: Rethinking Human Organizations Towards a Positive Theory of Change" (1999); Amanda Trosten Bloom, David Cooperrider and Diana Whitney, "The Power of Appreciative Inquiry: A Practical Guide to Positive Change" (2003. There are later editions too). Much more than ordinary survey questionnaires are able to do, from training experiences, and with close attention to the training interactions and emerging experiences, the appreciative enquiry model leads to a better understanding of change transformations as these result from complex interpersonal interactions, transformations from the exchange or transfer of knowledge and skills, and transformations from participation in ongoing experiences whose depth and value cannot be easily quantified. Useful as questionnaires may be, the evaluators nonetheless admit that these are not adequate enough to capture the essence of change impact as much as conversations and oral narratives do in open forum discussions. 11. On efforts to do post-program, follow-up, feedback: Due to financial resource constraints, we have as yet been unable to do a more comprehensive post-program review of the trainings, where this will require travel to the respective program countries to do on the ground assessments. Thus, we have relied on participant self-reports which they send to the Les Aspin Center, and which we also capture through contact with participants. Through these report narratives, graduates explain how they have used or are using the skills gained at the Les Aspin Center back in their respective countries. These narratives outline work in progress, work achieved, and work planned but awaiting execution with the acquisition of further financial and other support. In line with the training framework, program deliverables, and structured results matrix for determining program outcome/impact indicators, we consider the self-reports of graduate 84 accomplishments to meet the requirements of expected indicators of program success (in terms of whether immediate change -impact- is resulting now or in the long term at a future time). 12. Performance Plan: The evaluators observe that the program providers operate without a performance plan. We want to clarify that the timeline of program activities (as indicated in the program proposals) functions as the general guide for program performance. However, a strict adherence to these timelines has been impossible due to the fact that the actual implementation of the activities of the Africa programs has to account for the scheduled events and time frames of the wider Marquette University, other ongoing educational programs at the Les Aspin Center, as well as activity schedules of partners in the USA and in program countries. Thus, workflows have been greatly determined by an intricate coordination with program partners. 13. Follow up training programs for program graduates: The observation that the Les Aspin Center does not conduct follow up training for program graduates has to be clarified and set in proper context. Up until 2006, the duration of the programs (for West and East Africa respectively) was 6 weeks, with 5 weeks of regular training in the USA and a one whole week devoted to follow up enhancement training for recently returned graduates back in one of the program countries in East and West Africa. During this follow up week, graduates of other years were normally invited to join the new class of participants. Having returned home and worked for a while before the follow up sessions, the graduates brought into the discussions refreshed ideas and insights on how they have utilized and will continue to utilize the skills gained during the training in the USA to attain the objectives of the program back in the respective countries. Therefore, assisted by strategic development specialists and other program implementation experts, participants developed post-program activities from synthesis of the knowledge and skills learned from the entire training experience. It should be noted that after year 2006, this critical component of the training programs was lost due to USAID funding limitations. Hence, the Les Aspin Center was required to reduce the duration of the program to 5 weeks. Components such as the strategic planning sessions were reintegrated into the first week of training to allow participants to acquire and use the planning tools vital for post-program activity development. The only drawback was the loss of the opportunity for graduates to meet again in formal follow up sessions to re-engage each other, refresh ideas and experiences, share best lessons, and strengthen networks. The above notwithstanding, the Les Aspin Center has always endeavored in the past, and in present times, to create follow up opportunities for graduates to work together. From the onset of the programs, the Les Aspin graduates used the follow up sessions of the Week 6 to develop and establish alumni organizations. These were formal platforms through which alumni could collaborate and network on a sustained basis. In West Africa, the graduates created the “West Africa Organization for Democracy and Governance” (WAODEG), with country Chapters in Mali, Ghana and Nigeria. The East Africa alumni organization was called “HUSIKA” (“come join the effort”, in SWAHILI), with formally registered Chapters in Kenya, Uganda and Tanzania. However, the graduates of West and East Africa have been unable to sustain these organizations because of implementation and logistic challenges that the alumni faced. Among others, members found it impossible to leave their jobs to establish centralized coordinating centers and secretariats to run the alumni organizations. They had to rely on hosting by member institutions. Member work commitments also did not enable alumni to meet regularly and to work on common projects. Further, due to the fact that alumni lived and worked in different regions of the country, it was a challenge to develop and implement common alumni activities. The more serious challenge was sustaining leadership to coordinate activities of the organizations. Many of the leaders of WAODEG and HUSIKA either left the country, went for further studies, or changed jobs with more responsibilities and thus unable to devote time to coordinate alumni activities. Irrespective of the above challenges, alumni maintain contact with each other on account of relationships among program year groups, geographical proximity, collaboration on issues common to organizational commitments of members, membership in NGO and other work forums, as well as ability to engage each other in other collaborative engagements. Even though not generalized to all graduates, some year groups 85 Evaluation of the Les Aspin Center for Government Anti-Corruption and Good Governance in Africa Grant Program Final Report created platforms for common communication and networking with each other through Internet platforms such as “google groups”. The Les Aspin Center staff has endeavored to maintain contacts with alumni at least once a year. Some dedicated alumni also maintain follow-ups with Les Aspin Center. Unfortunately, due to financial resource constraints the Les Aspin Center is unable to do a more aggressive and formal follow ups with program graduates, as it used to be the case before 2006. Nonetheless, the Les Aspin Center regularly creates other opportunities for follow up with alumni especially during the WEEK 1 training sessions conducted in-country. A day is normally set aside where alumni are invited to join the new class for interaction, discussions or to give presentations and formal training. Although not all alumni are able to attend due to travel and other logistic constraints (since the Les Aspin center is unable to support such travel), other alumni do attend these sessions. Of late, with the view to strengthening a more coordinated and sustained engagement and networking among alumni of the programs, USAID made available funding for the Les Aspin Center to conduct 2 refresher courses for selected alumni from East and West Africa respectively. The refresher training enabled the participants to discuss, plan and develop platforms for all Les Aspin graduates in the 6 program countries to collaborate and network better on a more sustained basis. In order to develop and sustain better follow up activities for the Les Aspin graduates, it has always been the expectation of the Les Aspin Center that since these personnel have been trained by USAID, the various USAID missions in the participating program countries will find ways to integrate the Les Aspin alumni into the respective governance and other programs of these missions. Since 1996 to date, the programs have produced over 400 personnel from diverse institutional affiliations from the public and private sectors. The graduates constitute great networks of talent and bring with them a wide range of personal and organizational resources that can benefit the respective USAID missions. USAID missions in program countries have provided invaluable assistance to us to conduct the programs, for which we are immensely grateful. We are ready to continue to work with them in order to strengthen post-training follow-ups for program graduates. We are gratified that USAID missions have consistently demonstrated confidence in the Les Aspin Center’s ability to provide capabilities to government and civil society personnel in the respective program countries 14. Conclusion: We express our thanks for the opportunity to clarify key issues raised in the draft evaluation report. We look forward to further discussions on the above clarifications.