August 28, 2014 This publication was produced for review by the United States Agency for International Development. It was prepared by Development & Training Services, Inc. (dTS) under Contract No. AID-RAN-I-00-09-00015, Task Order AID-OAA-TO-12-00007. Authors: W. Kent Edwards & Warigia Bowman, PhD. FINAL REPORT UNIVERSAL SERVICE/UNIVERSAL ACCESS FUND (USAF) PERFORMANCE EVALUATION Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation ii Prepared for the United States Agency for International Development, USAID Contract Number AID￾RAN-I-00-09-00015, Task Order Number AID-OAA-TO-12-00007, Evaluation of Universal Service and Access Funds. Implemented by: Development & Training Services, Inc. (dTS) 4600 North Fairfax Drive, Suite 402 Arlington, VA 22203 Phone: +1 703-465-9388 Fax: +1 703-465-9344 www.onlinedts.com Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation iii FINAL REPORT UNIVERSAL SERVICE / UNIVERSAL ACCESS FUND (USAF) PERFORMANCE EVALUATION August 28, 2014 DISCLAIMER The authors' views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation iv CONTENTS Acronyms ...................................................................................................................................... vi Project Summary ........................................................................................................................... 1 Executive Summary ...................................................................................................................... 2 Major Findings and Conclusions ............................................................................................................................... 3 Recommendations ....................................................................................................................................................... 4 Lessons Learned........................................................................................................................................................... 5 I. Introduction ................................................................................................................................ 7 Objectives & Desired Results ................................................................................................................................... 7 Background to the Performance Evaluation .......................................................................................................... 8 Implementation of the Performance Evaluation ................................................................................................... 9 II. The Development Problem and USAID’s Response ...........................................................11 Problem Statement.................................................................................................................................................... 11 USAID’s Intervention in Response to the Problem .......................................................................................... 11 III. Purpose of the Evaluation .....................................................................................................12 IV. Research Design and Evaluation Methodology ...................................................................13 Limitations ................................................................................................................................................................... 14 V. Findings ....................................................................................................................................15 Overview ..................................................................................................................................................................... 15 Findings About USAF Project Performance Achievements ............................................................................. 16 Kenya ...................................................................................................................................................................... 16 Nigeria .................................................................................................................................................................... 18 Ghana ..................................................................................................................................................................... 20 Tanzania ................................................................................................................................................................. 22 Cost-Effectiveness Measurements - Findings ...................................................................................................... 24 General .................................................................................................................................................................. 24 Funds Leveraged .................................................................................................................................................. 28 Strategies and Policies Adopted ....................................................................................................................... 29 VI. Conclusions .............................................................................................................................30 General Conclusions................................................................................................................................................. 30 Regional Impact .................................................................................................................................................... 30 Unexpected Consequences .............................................................................................................................. 30 Primary Factor Contributing to Success ........................................................................................................ 30 Country Specific Conclusions ................................................................................................................................. 30 Kenya ...................................................................................................................................................................... 31 Nigeria .................................................................................................................................................................... 31 Ghana ..................................................................................................................................................................... 32 Tanzania ................................................................................................................................................................. 32 Other Capacity Building Activities .................................................................................................................. 32 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation v Cost-Effectiveness ..................................................................................................................................................... 33 VII. Recommendations ................................................................................................................35 VIII. Lessons Learned ..................................................................................................................42 Annexes ........................................................................................................................................44 Annex 1. Statement of Work/Terms of Reference ........................................................................................... 45 Annex 2. COR Designation Letters for the USAF Evaluation........................................................................ 52 Annex 3. GBI USAF TOR ........................................................................................................................................ 71 Annex 4. GBI Workplan .......................................................................................................................................... 89 Annex 5. Integra Selection Criteria for Countries to Receive Intensive Technical Assistance .............. 99 Annex 6. GBI TORs for Core Countries .......................................................................................................... 101 Nigeria .................................................................................................................................................................. 101 Ghana ................................................................................................................................................................... 106 Kenya .................................................................................................................................................................... 110 Tanzania ............................................................................................................................................................... 125 Annex 7. Evaluation Methodology – Details ..................................................................................................... 127 Annex 8. Key Informant Interview Guides ........................................................................................................ 171 Annex 9. Key Informant Interview Write-Ups................................................................................................. 180 Nigeria .................................................................................................................................................................. 180 Ghana ................................................................................................................................................................... 208 Tanzania ............................................................................................................................................................... 222 Kenya.................................................................................................................................................................... 242 United States – USAID ..................................................................................................................................... 259 United States – Integra ..................................................................................................................................... 271 United States – NetHope ................................................................................................................................ 286 Annex 10. Non-Core SSA Country Survey ...................................................................................................... 288 Online Survey Questionnaire ......................................................................................................................... 288 List of Online Survey Addresses .................................................................................................................... 293 Narrative Discussion of Results of Survey of Non-Core Countries .................................................... 295 Tabulation of Survey Results of Survey of Non-Core Countries .......................................................... 297 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation vi ACRONYMS A4AI Alliance for Affordable Internet AQL Acceptable Quality Level AUSAFA African Universal Service and Access Fund Association CAPEX Capital Expenditure CCK Communications Commission of Kenya CCN Cooperating Country National CEO Chief Executive Officer CO Contracting Officer COMESA Common Market for Eastern and Southern Africa COP Chief of Party COR Contracting Officer’s Representative COSTECH Tanzania Commission for Science & Technology COTR Contracting Officer’s Technical Representative [outdated; now COR] CTO Commonwealth Telecommunication Organization DFID Department for International Development [of the United Kingdom] DOS US Department of State dTS Development & Training Services, Inc. ECOWAS Economic Community of West African States EGAT [USAID] Bureau for Economic Growth, Agriculture and Trade [now incorporated into the new Bureau for Economic Growth, Education, and Environment, E3] FY Fiscal Year GBI USAID - Global Broadband and Innovations Alliance GES Growth Enhancement Support Scheme GIFEC Ghana Investment Fund for Electronic Communications GSM Global System for Mobile Communications GSMA GSM Association ICT Information & Communications Technology IFDC/AFAP International Fertilizer Development Center/African Fertilizer & Agribusiness Partnership INTEGRA Integra Government Services International, LLC or Integra, LLC IQC Indefinite Quantity Contract ISSER Institute of Statistical Social and Economic Research [of the University of Ghana] ITU International Telecommunications Union KA Band Ka band Satellite - covers the radio frequencies of 26.5–40 GHz KII Key Informant Interviews LOE Level of Effort LTE Long-Term Evolution [commonly marketed as 4G LTE] M&E Monitoring and Evaluation MARKETS [USAID/Nigeria] Maximizing Agricultural Revenue & Key Enterprises in Targeted States Project MOC Ministry of Communications MOU Memorandum of Understanding NBS National Broadband Strategy - Kenya NCA National Communications Authority of Ghana NCC Nigerian Communications Commission NEXTT [USAID/Nigeria] Nigeria Expanded Trade and Transport Project NGO Non-governmental Organization Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation vii NITA National Information Technical Agency of Ghana OPEX Operational Expenses PC Personal Computer PMT [dTS] Project Management Team PPP Public Private Partnership PrEng Certified Professional Engineer PS Principal Secretary of Kenya RFP Request for Proposals SADC Southern African Development Community SMP Strategic Management Plan - Nigeria SOW Scope of Work/Statement of Work SSA Sub-Saharan Africa TA Technical Assistance TANZICT Tanzania Information Society & ICT Sector Development Project TCP/IP Telecommunications Communications Protocol/Internet Protocol TCRA Tanzania Communications Regulatory Authority Telecom Telecommunications TOD Task Order Director/Team Leader (dTS) TOPR Task Order Performance Reports TOR Terms of Reference TS Tanzanian Shilling (local currency) TTCL Tanzania Telecommunications Company Limited TV Television UAF Universal Access Fund UCSAF Universal Communications Service Access Fund of Tanzania, formerly UCAF US United States of America USAC Universal Service Advisory Council - Kenya USAF Universal Service/Universal Access Fund(s) USAID United States Agency for International Development USAID/W USAID/Washington USF Universal Service Fund USPF Universal Service Provision Fund Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 1 PROJECT SUMMARY 1. Project Name: Universal Service/Universal Access Fund (USAF) Performance Evaluation 2. Objective: The dTS team is tasked with conducting a final performance evaluation of the USAF Project implemented by Integra, LLC (hereinafter referred to as Integra) under the USAID-funded Contract Number AID-CIO-I-10-00001, Task Order Number AID-CIO-TO-11-00001. The aim of this evaluation is to assess the effectiveness of Integra’s USAF technical assistance in Sub-Saharan African countries. 3. Period of Performance: October 1, 2011 – July 31, 2014 4. Implementing Partners: Development and Training Services, Inc. (dTS) 5. Contract Number: AID-RAN-I-00-09-00015, AID-OAA-TO-12-00007 6. Project Funding: $250,000 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 2 EXECUTIVE SUMMARY The purpose of this report is to provide an independent and objective assessment of the USAID Africa Universal Service and Access Fund Project (USAF), which was implemented in Sub-Saharan Africa (SSA) from April, 2011 through 30 September 2013. USAID’s objective for the USAF Project was “to improve the practices and increase the actual use of current and future telecommunications universal service and access funds in sub-Saharan Africa to accelerate the extension of affordable and sustainable access to telecommunications services (including broadband Internet services where permissible) to those not yet served, tapping innovative technical and business approaches and adapting as appropriate best practices and lessons learned from Africa and elsewhere.”1 The USAF implementing firm awarded the project was Integra Government Services International, LLC, also known as Integra, LLC, and hereinafter referred to as “Integra.” The problem area USAID targeted was to determine what programs or activities should be supported to help achieve increased economic growth in the SSA region. The specific intervention USAID designed to address the problem was the USAF Project, which envisioned that improving USAF functioning in SSA countries and increased access to broadband would contribute to the goal of economic growth. In order to assess the overall results of the USAF Project, USAID issued a TOR to have a performance evaluation conducted on the USAF Project. The evaluation had three objectives: 1. to measure the impact of the USAF Project via a performance evaluation; 2. to measure cost effectiveness of approaches used to achieve the impact of the Project; and 3. to make the evaluation results available and useful.2 In designing the evaluation requirements, USAID recognized the potential difficulty of being able to attribute results to the USAF Project and, given the potential long-term fruition of capacity building efforts, that some of the evaluation findings could be directional3. Therefore, it was anticipated that some of the evaluation findings and conclusions would focus on whether or not the USAF trend was moving in the right direction. It was also recognized that USAFs in Africa lagged behind other regions of the world. A 2010 ITU analysis4 found that of the 45 SSA countries responding, only 25 had laws requiring a USAF, and only 15 had actually collected funds. Of these, only eight countries had actually disbursed any funds to support universal service/access. It was believed that SSA countries have the opportunity in implementing 1 USAID EGAT I&E/ICT, Telecommunications Universal Service Funds or Universal Access Funds (USFs/UAFs), Task Order Award, AID-CIO-TO-11-00001, April 12, 2011 2 The USAID Evaluation TOR anticipated that this objective would include numerous significant “outreach” results. However, due to budget constraints, the objective was narrowed down to require only a final presentation of the evaluation results to USAID interested personnel. 3 At the evaluation kick-off meeting with USAID, it was discussed that “directional” meant that USAF appeared to be heading in an effective direction. 4 Final Third Version of the Report on Universal Access and Service Funds (USAFs) in the Sub-Saharan Africa (SSA) Region,” presented to the ITU Regional Office for Africa on April 30, 2010 by Edgarto Sepulveda. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 3 USAFs to take advantage of the best practices and experiences from other parts of the world, although the high cost of Internet access across much of the SSA countries was understood to present obstacles to increased broadband access. The USAF Evaluation was started in February 2012, and was intended to extend for six months after the termination of the USAF Project (September 2012). Due to a one-year extension of the USAF Project, dTS requested and USAID approved a one-year no-cost change extension of the Evaluation Project. USAID’s TOR had intended that the evaluation be completed in two phases. Phase I was to review available material and to prepare a work plan to be implemented as Phase II. Due to the extension of the project, it was agreed that dTS would monitor the USAF Project activities for some time, and that active evaluation activities would be delayed until a later time. In late 2013, dTS restarted the active evaluation activities by providing a Phase II work plan, and moved forward with key informant interviews in Africa, followed by interviews in the US of USAID, Integra and NetHope personnel associated with the USAF Project. The Performance Evaluation’s findings and conclusions are based on: (1) a review of pertinent documentation; (2) interviews with USAID personnel, (3) extensive interviews with implementing firm personnel, (4) review of Integra project related documents, (5) interviews of key government officials at the organizations receiving technical assistance from Integra, (6) interviews of non-governmental persons in several of the four Core Countries, (7) an electronic survey of selected non-Core countries. Because of a medical emergency in the dTS Team Leader’s family, the one-year extended evaluation period that was to end on 31 March 2014 was further extended for three months – until 30 June 2014. Due to the unexpected departure of the cost effectiveness expert at dTS and the potential unavailability of pertinent USAID personnel for review and comment on the Draft Evaluation Report, another extension was requested and approved. The project completion date is now 31 August 2014. MAJOR FINDINGS AND CONCLUSIONS Integra employed four related forms of assistance in carrying out the USAF Project. It went through a screening process across the countries included in the previously referenced ITU analysis, and selected four countries to receive intensive technical assistance. The counties, referred to in this document as the four Core Countries, were: Kenya, Nigeria, Ghana and Tanzania. Integra obtained a TOR in each of these countries, although in Kenya, a formal government-to-government MOU was required in addition to the TOR. Integra’s success in completing the deliverables of its TORs varied considerably. In Kenya, active support to establish the USAF had to be delayed due to the need to support development of the National Broadband Strategy, a much broader effort than USAF, but one which clearly would encompass universal service in the country. The dTS evaluation found that Integra successfully delivered two of its TOR objectives (analysis/recommendation on a GAP study and outstanding support of the NBS development). It also provided the necessary support that would have resulted in the implementation of the USAF (Objective 3), had not the time limitation of the project expired. In Nigeria, dTS found that Integra successfully met all three of its TOR objectives. The deliverables were: 1) review and evaluation of consultancies on Strategic Planning and impact assessment; 2) Gap analysis planning and requirements, and final analysis report; and 3) a monitoring and evaluation recommendation report. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 4 dTS found that the results in Ghana were not as favorable. Integra had four deliverables in Ghana. These were: 1) Report on data collection framework, objectives, methods and procedures; 2) Report on results of a market study and survey; 3) Report on market and gap analysis, and 4) Report on policy recommendations. Integra was able to complete the first deliverable, but, due to lack of cooperation by the entity conducting the gap analysis (and a one-year delay in completion of the survey), Integra was only able to partially complete deliverables 2 through 4. In Tanzania, although a TOR was signed with the universal service entity, the government process of vetting potential staff for the organization resulted in Integra having to cease activities. A contingency of the agreed TOR had been that the universal service organization would hire adequate staff to be able to receive the technical assistance. Integra did provide a strategic plan template and maintained contact with the CEO of the universal service unit, but the project time constraints ultimately made it impossible to pursue assistance to Tanzania. dTS also found that the other modes of assistance delivered during the USAF project were successful in reaching a wider regional audience than the specific country capacity building. Integra cosponsored and delivered training in three regional workshops, one with the Commonwealth Telecommunications Organization and two with Intel. Three online webinars were developed and delivered, and four detailed USAF written modules were prepared and provided to participants across the region. dTS found that these activities met the TOR objective to reach a regional audience, and concluded that they were done very well. dTS concluded in the evaluation process that where Integra was able to accomplish its TOR objectives, it did so by applying human resources of the highest caliber. Key informants in all four Core Countries stated their desire for continued support from the Integra team and USAID. In Kenya, Integra’s performance in the NBS work was outstanding, and it laid a complete groundwork foundation for getting the USAF up and running. In Nigeria, dTS concluded, via observations during the key informant interviews and review of materials, that Integra’s work significantly contributed to the increased efficiency and transparency of the Universal Service Provisioning Fund (USPF) function, and that the impact of Integra’s efforts will continue to be felt in ongoing activities in Nigeria. The conclusion in Ghana by the evaluator was that Integra completed the amount of its TOR deliverable that was possible, given the obstacles encountered. In Tanzania, dTS concluded that Integra’s suspension of proposed assistance was the correct course of action, and that the preliminary support it had provided was well done. Regarding the cost effectiveness of the assistance provided, dTS found that the assistance provided in Kenya, Nigeria and Ghana helped move the countries forward toward more successful USAF functioning, and that staff finally hired in Tanzania were able to receive training. dTS concluded that the efforts in Kenya and Nigeria were highly cost effective based on the funds to be spent on ICT and USAF. dTS concluded that the activities of the USAF Project produced a cost effectiveness ratio of 162/15. RECOMMENDATIONS dTS provides ten recommendations with this evaluation report, as follows: 5 See Table 2. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 5 1. Further USAF assistance is recommended in Kenya and Nigeria. Assistance to Kenya should be contingent upon the appointment of a new Universal Service Advisory Council (USAC), as the initial USAC was recently disbanded. In Nigeria, a follow-up project would be in order, to assess how well USPF has implemented Integra’s recommendations, and to support any needed corrective actions. Suggested measures are included in the recommendation. 2. It is recommended that a “follow-up” step be designed into any direct assistance to a country, so that USAID has the opportunity to ensure that the initial capacity building has taken hold or to assist in corrective actions. The budget requirements for this follow-up phase must be dedicated at project launch. 3. USAID/W-initiated ICT projects need to consider at the design stage the situation and current priorities in any country USAID Mission where assistance is targeted. This would involve more than briefing the missions; it would require actual consultation with the country Mission. 4. It is recommended that USAID develop a program that would demonstrate how ICT can tie together multiple programs and help to identify the potential synergies between programs. Once developed, the output of this project would be used to further educate country Missions on the benefits of including an ICT analysis in most if not all anticipated programs. 5. dTS recommends that USAID consider the need to be realistic when designing ICT programs intended to have a “regional” impact, as the SSA countries are too many to handle with a single project unless a long-term, high budget program is to be initiated. 6. Where assistance in the ICT area is to be provided, dTS recommends that USAID gather an in￾depth picture of the ICT needs in the country prior to launching. While the Missions already do this in other areas, it is not clear that it is a design requirement for ICT assistance. USAID should take advantage of the US human resources within a targeted country (State Department, Commerce Department, and the Mission). 7. dTS recommends that USAID attempt to identify all donors providing support in ICT in a country or region, then take the lead in creating a cooperative donor framework to avoid duplications. The potential political difficulties of this recommendation are understood, but dTS believes that it needs to be considered. 8. Continued cosponsoring and participation in Intel or other regional USAF related events is recommended as an effective means to increase awareness in SSA countries. However, it is strongly recommended that USAID obtain a detailed list of participants at such events as a resource for future use. Such list would identify the person, country and full coordinates. 9. Consideration should be given to developing a Terms of Reference (TOR) template for use in USAF related projects, such that the implementer could select or eliminate specific items as appropriate. 10. Lastly, dTS recommends that USAID review and consider the suggestions provided in this by key informants in the four Core Countries, USAID Mission personnel, and Integra. These suggestions have been extracted from the interview write-ups and are contained in this recommendation. LESSONS LEARNED dTS offers the following lessons derived from the evaluation experience: 1. Recognition needs to be built into USAF-type assistance projects that the stage of development in a specific country will determine the level of actual impact that can be observed as a result of the assistance. This needs to be taken into account in the design and measurement criteria of such projects. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 6 2. It should be anticipated “up front” in project design that the necessity of a government-to￾government MOU will be a lengthy process, such that the project design time line takes this into consideration. 3. Even well prepared projects require lengthy time frames for completion of technical assistance and capacity-building activities. 4. Results and outcomes do not happen quickly. Thus, measuring the results and outcomes should extend to 2 to 4 years after completion of the project (notwithstanding the ever present issue of counter factual influences). 5. Each of the four Core Countries which were recipients of USAID support in the USAF Project need continued support. Clearly, there is also a serious need for support in USAFs in many other SSA countries. 6. The inclusion of all stakeholders in an undertaking, such as was done in the Kenya NBS effort, is an effective model for other countries. 7. Short-term capacity building will not likely result in long-term retention of all of the guidance provided without some predictable form of follow-up. 8. The existence of a fiber backbone in a country is intimately linked to the potential for success in expanding universal service and access. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 7 I. INTRODUCTION The intent of this Task Order contract is to perform evaluation services related to an innovative USAID-funded project designed to increase the effective use of telecommunications universal service funds in the sub-Saharan Africa region. This project has high visibility within and beyond USAID, and can have significant impact on the availability of affordable access to telecommunications services for the poor. It may also be adapted to other regions. The project, entitled Telecommunications Universal Service Fund/Universal Access Fund (USF/UAF) Project (referred to below as USAF Project or simply Project) was implemented by Integra Government Services International, LLC (Integra) and was funded directly by USAID/AFR/SD/EGEA, in large part under a Task Order and some small funding via a subcontract with NetHope. For purposes of this evaluation, it is considered as one project, although it was funded through the two separate mechanisms. OBJECTIVES & DESIRED RESULTS This evaluation had three objectives: Objectives 1 and 2: dTS was to design and implement an approach to (1) measure the impact of the USAF Project via a performance evaluation, and (2) measure the cost-effectiveness of approaches used to achieve the impact of the Project. The USAF Project’s objectives are stated as follows in the task order to Integra: “To improve the practices and increase the actual use of current and future telecommunications universal service and access funds in sub-Saharan Africa (SSA) to accelerate the extension of affordable and sustainable access to telecommunications services (including broadband Internet services where permissible) to those not yet served, tapping innovative technical and business approaches and adapting as appropriate best practices and lessons learned from Africa and elsewhere. Impact: Based on this, the impacts to be evaluated can be summarized as follows: 1. Increase in the actual use of telecommunications universal service and access funds in SSA and, by doing so, 2. Accelerate the extension of affordable and sustainable access to telecom services (including broadband Internet services where permissible) to those not yet served.” The evaluation statement of work (SOW) did not request a full impact evaluation with the requisite, rigorously defined counterfactual, but rather a performance evaluation focusing on impact: Did the project meet its objectives? Have expected results been achieved? dTS was not expected to focus on questions related to how well the project was managed, how it was perceived, and how well inputs were used. The challenges of attribution of results and achieving results within the project time period were recognized by USAID. dTS was expected to propose and carry out an approach that addressed these challenges to the extent possible. Cost Effectiveness: The USAF Project had a wide variety of options to choose from to achieve impact, some resource-intensive (e.g., technical assistance provided by experts directly to one universal service fund) and others, far less so (e.g., conducting a workshop for several countries together; involving the staff of one or more countries in assistance provided to a specific fund; preparing toolkits for several Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 8 funds to use). The second evaluation objective was to assess which approaches were most cost￾effective, identify key considerations when selecting approaches, and provide some assessment of the relative effectiveness of approaches in meeting Project objectives. Objective 3 – Making Results Accessible and Useful: The third contract objective was to develop summary documents, presentations or other communication materials to make evaluation results accessible and useful to USAID teams, other donors, and universal service funds themselves. This objective required developing and executing an outreach plan. Due to budgetary constraints, the original dTS technical proposal was modified (and accepted by USAID) to reduce the level of effort (LOE) and expected deliverables for this objective, limiting the outreach component to the preparation of this report and its presentation to USAID by the project team leader. BACKGROUND TO THE PERFORMANCE EVALUATION Access to telecommunications services can increase the impact of development activities across all economic sectors within a country, including the education, health, agriculture, government and private sectors. There is a proven link between access to mobile phone services in Africa and economic growth. Because of this, almost all USAID and other donor development projects in SSA use telecommunications-enabled tools and services to increase impact. Unfortunately, the requisite telecom services are often not accessible in a significant portion of the geographic areas where such donor projects focus, including rural and non-rural areas where the poor and extremely-poor reside. A primary reason for this uneven coverage is that these are areas where telecom service providers are less likely to be able to provide services on a viable commercial basis. Extending required infrastructure to these areas and supporting ongoing operations does not generally make financial sense, given relatively low potential customer revenue. Given the importance of access to telecommunications services for development purposes, many countries have established telecommunications Universal Service Funds or Universal Access Funds (USAFs) to provide services to these underserved areas. Based on legislation and regulations, countries establish funds that collect a small percentage of revenue from licensed telecom/information and communications technology (ICT) services providers. The funds are then distributed using a variety of approaches (often a reverse tender process) to provide needed subsidies to telecom services in target areas. Countries vary in the percentage of revenue collected, approach to managing funds, permissible uses of funds, and a variety of other factors. Globally, the USAF track record varies considerably, with most countries falling far short of achieving the full potential benefits of such funds. There are several prerequisites to realizing these benefits. Countries must: 1. Pass legislation and establish the regulations needed to create a USAF; 2. Establish a fund with an entity to operate it; 3. Collect funds and keep their identity separate from general government funds; 4. Disburse funds for their intended public purpose; 5. Establish and maintain an effective monitoring and evaluation process to objectively assess the successes, failures and impacts of universal service projects; USAFs in South America probably have the longest track record, have been studied the most, and offer experience that can be helpful to funds elsewhere. Recently, the International Telecommunications Union (ITU) commissioned a study of the status of USAFs in SSA, yielding some stark findings showing Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 9 that they fall far short of their potential. Forty-five countries responded to the ITU survey. Twenty-five have laws calling for USAFs; of these, only 15 have actually collected funds, and even fewer – eight – have actually disbursed any funds. Even worse, on average, these latter funds have only managed to disburse 13 percent of monies collected. As elsewhere in the world, USAF organizations in SSA are trying to design and operate to achieve maximum benefit. Current challenges include: 1. Moving from legislation to action – steps needed to establish an entity to manage the fund; 2. Actually collecting funds; 3. Managing funds prudently, reflecting good business practices; 4. Disbursing the funds in accordance with policy and legislative guidelines, reflecting (and adapting as needed) best practices elsewhere; and, 5. Monitoring and evaluating impact and adapting practices based on lessons learned directly and via analysis of funds in other countries. Africa’s USAFs have opportunities with potential to help them address these challenges. First, they can learn from funds with long track records. Second, in late 2008, the USAFs formed an association, the African Universal Service and Access Fund Association (AUSAFA) to provide support and help via collaboration between countries. The Commonwealth Telecommunication Organization (CTO) serves as the Association’s Secretariat. Third, several funds in Africa, including those in Senegal, Uganda, Mauritius, South Africa and Ghana, are already operational in differing degrees, and may serve as examples to other African funds more recently formed or now becoming operational. Fourth, the ITU has funded two phases of analytical work regarding USAFs in SSA and considers this an important topic. USAFs in SSA have some unique challenges. The region has the highest priced Internet access due to general reliance on satellites for access to international gateways and between countries. This has begun to change recently with the arrival of several undersea cables and a variety of investments in terrestrial fiber networks by private investors, governments and multilateral organizations. Also, cell phone network providers have expanded dramatically in recent years, are moving to using telecommunications communications protocol/Internet protocol (TCP/IP) for their core infrastructure and are also upgrading their networks to support Internet access. This evaluation is intended to provide USAID with important performance information regarding the efficacy of a project to provide technical assistance to USAFs in countries in Sub-Saharan Africa over a three-year period. It is also intended to provide additional insights for the USAID Global Broadband and Innovations (GBI) Program, which seeks to improve the use of USFs globally. In examining the status of USAFs in SSA, Integra determined that there was significant variation across countries in the level of development and operation of such funds. Such wide variation led Integra to conclude that targeting assistance primarily on a regional basis would not be optimum. Thus, Integra narrowed the intended targets for direct technical assistance to four countries (Kenya, Tanzania, Ghana, and Nigeria – the “Core Countries”). Integra subsequently made a further decision to place Tanzania assistance on an indefinite “hold” status, since there had been no movement in the country to hire the human resources necessary to support USAF activities. IMPLEMENTATION OF THE PERFORMANCE EVALUATION The USAF Evaluation was chartered in February 2012, and the actual project kick-off meeting took place in March 2012. By design, the evaluation was to be implemented while the USAF Project was in its final Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 10 year of operation, and was to be completed six months after the end of the project. Because of the need to support the National Broadband Strategy development in Kenya (see Section V, Findings – Kenya) and funding delays, the USAF implementer found it necessary to request a one-year extension which USAID approved. In order to maintain the originally intended timing relationship to the USAF Project, dTS then requested and received approval from USAID for a one-year extension of the evaluation. It was further agreed that dTS would delay active evaluation efforts for an extended period, but would maintain awareness of project activities and status via monthly monitoring. It was also agreed that dTS would, at the appropriate time, restart more active evaluation activities. During the interim monitoring period, dTS received Integra’s monthly reports to USAID, and after review, had a monthly call with Mr. Otto, the president of Integra, to gain further depth of understanding and request any needed clarifications. Subsequent to the monthly calls, dTS prepared a brief status report for the evaluation’s contracting officer’s representative (COR). It should be noted that throughout the evaluation process, both during the interim monitoring period and afterwards, Integra, its employees, and its consultants fully cooperated with the evaluation team. When the active evaluation period began, dTS prepared a Phase II work plan and moved forward to complete the evaluation. Primary activities were: (1) Visits to the four Core Countries to interview people involved in the USAF and related activities; (2) Electronic survey of selected non-Core countries; (3) Interviews of all USAID and Integra personnel involved in the USAF Project; and (4) Review and analysis of all gathered information for report preparation. The evaluation’s one-year extension was to end on 31 March 2014. Because of a medical emergency in the dTS team leader’s family, it was necessary to request an additional three-month extension, which was approved by USAID. USAID’s flexibility in approving this extension is much appreciated by dTS and the project team leader. Finally, due to the loss of the dTS cost effectiveness expert and the potential unavailability of USAID personnel for review and comment on the draft report, dTS and USAID agreed to a final extension of the project completion date to 31 August 2014. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 11 II. THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE PROBLEM STATEMENT One of USAID’s overarching goals in the SSA region and worldwide is to provide assistance that will either contribute or lead directly to increased economic growth. Although SSA countries are experiencing the same type of population migration from rural to urban environments as other areas of the world, the majority of the regional population still resides in rural areas which are generally characterized by a paucity of technological infrastructure and more than abundant levels of poverty. The telecommunications reform initiatives of the late 1990s and early 2000s, combined with the explosive growth of mobile telephony, have contributed to a rising middle class in most SSA metropolitan areas, but SSA countries have seriously lagged behind the rest of the world in extending services to the rural areas and the unserved poor. By 2010, most SSA countries had adopted universal service programs in one form or another, and many were in the process of expanding their definition of universal service to include universal access to broadband services. A 2010 ITU report, however, found that few of the national USFs had been able to disburse funds to actually extend universal service to the unserved population. The problem for USAID was to determine what programs or activities should be supported to help achieve the increased economic growth goal in the SSA region. USAID’S INTERVENTION IN RESPONSE TO THE PROBLEM USAID initiated the USAF Project in response to the problem, and envisioned it as a way to contribute to economic growth in the long run by assisting SSA countries in improving their universal service funds’ functioning. The USAF Project was chartered as part of the GBI initiative, also implemented by Integra. By assisting SSA countries in improving USAF operations, it was believed that the ultimate impact would be stimulation of economic activity as more and more people were able to access either basic voice telephony or gain Internet access via broadband. As originally envisioned, the USAF Project was intended to be completed over a 12 to 15-month period. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 12 III. PURPOSE OF THE EVALUATION The purpose of the USAF Evaluation is to determine, on an objective basis, whether or not the project implementer, Integra LLC, met the objectives of the terms of reference (TOR), and, if so, how well such objectives were met. The reasons for assessing the relative success of the USAF Project are two-fold: First, it is a necessary management action to measure the successes or failings of a USAID-funded project. Second, USAID desires to obtain “lessons learned” and recommendations derived from the evaluation which can be used as design considerations for future project development. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 13 IV. RESEARCH DESIGN AND EVALUATION METHODOLOGY The SOW of the evaluation Task Order awarded to dTS described two phases of work. Phase I was designing the approach to conducting the work. This required dTS to meet with the Integra team and USAID representatives to understand the project’s approach, types of and accessibility of data to conduct evaluation services, and ways to collect required data (quantitative and non-quantitative). The USAF Project team had been directed by USAID to be cooperative, and has been since the start of the evaluation. dTS could also use other means it deemed useful to conduct the design work. Phase I, as originally envisioned, was not to exceed 90 calendar days. At the conclusion, dTS was to propose an approach to conducting and completing the work described above, including a description of data that could reasonably be collected by the USAF Project, as well as other data that would be collected, including type, source, and approach to collection, quality control measures and other pertinent characteristics. dTS was also to propose the elements of the multi-pronged approach prescribed above, including the range of documents and communications approaches proposed, and outlines for each. At the conclusion of Phase I, the USAID SOW envisioned that dTS would prepare a detailed work plan for Phase II, to be approved by the COR prior to launching Phase II. Integra had requested and received a one-year project extension; dTS similarly requested and received USAID approval for a one-year extension of the evaluation. dTS went into a monitoring mode from July 2012 through June 2013, waiting for critical mass of Integra’s contributions to begin a more active evaluation process. dTS restarted the active Phase II engagement in August 2013, and on August 16, 2013, sent a preliminary draft Phase II work plan to the USAID COR. dTS also recommended modifications in the LOE personnel structure for the project, as one of the ICT experts was unable to rejoin the project due to other commitments. In a USAID/dTS conference call on September 4, 2013, dTS learned that USAID would not consider the LOE modifications until after September, the end of its fiscal year. It was agreed that dTS would continue working on necessary activities with the previously approved LOE, and that a refined Phase II work plan would be prepared and submitted after USAID’s formal decision regarding LOE amendments. Any changes to methodology from the original dTS proposal would be contained in the Phase II work plan, subject to USAID approval. On October 30, 2013, after the dTS restart, an Evaluation Methodology document was provided to USAID, including an overview of proposed methodology, data collection and analysis steps, and the Phase II proposed work plan. Annexes to the document contained the list of key informants; proposed key questions to include in interviews/surveys, proposed guide for telephone interviews, and evaluation design outline. The proposed methodology focused on determining specific impacts attributable to Integra’s work. It was recognized that the primary concentration would need to be on qualitative measures, but that dTS would explore the impact on users or levels of service penetration in a country, new trends that would lead to changes in impact, expert opinions of key informants and a mini-survey of project participants. Proposed potential measures also included: Increased USAF usage (amount of revenue collected and disbursed); changes in telecommunications access and penetration; acceleration of access; and what the Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 14 direction of the USAF in a given country appeared to be at the end of Integra’s assistance. dTS recognized the need to take into account the differing country circumstances under which Integra had provided assistance, as well as the potential impact of rapid technological evolution. As USAID noted in the evaluation TOR, the difficulty in determining what impacts or changes could be directly attributable to Integra’s efforts was not underestimated. To accomplish the evaluation, dTS planned to perform a thorough document review, conduct interviews of key informants (at USAID, Integra, NetHope and in the four Core Countries), and initiate an electronic survey of project participants. Document reviews and key informant interviews became the primary methods employed by dTS during the evaluation. It was not possible to identify USAF Project participants who had received training at the three regional workshops; accordingly, neither the telephone interviews nor the intended electronic survey were conducted as planned. dTS did, however, conduct an electronic survey later in the project. That survey was sent to SSA countries other than the four Core Countries to acquire a general picture of the status of USAFs and determine if there was any awareness of USAID’s USAF activities in SSA. The key informant at the African Union Commission (AU) also offered to send the dTS survey to all its members, with the goal of potentially getting a better response than dTS had received at that point, and, also, to further inform the AU on USAF in the countries. See Annex 7 of this report for additional details. LIMITATIONS USAID recognized in the Evaluation TOR design that it would be difficult to distinguish exogenous influences on USAFs’ development and progress during the life of the project, and also that, due to the nature of the problems being addressed and the assistance to be provided, that some of the results of the implementer’s actions could be “directional;” i.e., observable indicators signal that the USAF is moving toward effective operation. dTS further found during the evaluation process that it was not feasible to apply any significant level of quantitative analysis to measure Integra’s performance. The USAF Project took place during a period in which mobile services – including mobile broadband – experienced significant growth in most SSA countries. It would not have been feasible to measure the incremental impact, if any, on services extended as a result of the project. Also, the intricacies and complexities of the USAID funding approach and resulting delays in the USAF Project were not examined in this evaluation, since the TOR stated that, “The Contractor is not expected to focus on questions related to how well the project was managed, how it was perceived, and how well inputs were achieved.” The inability to measure the impact on direct and indirect USAF beneficiaries placed a serious limitation on dTS’ ability to measure the relative cost-effectiveness of Integra’s efforts. In lieu of being able to measure how many people gained either basic service or broadband access, dTS found it necessary to rely on the same general measure that Integra used: amount of potential USAF planned by the countries going forward. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 15 V. FINDINGS OVERVIEW Integra followed a measured, carefully defined process6 when determining which countries would be prime candidates for assistance, and narrowed down the potentials to about 10 countries. Through further analysis, the final choice of Core Countries included Ghana, Nigeria, Tanzania, and the already selected Kenya. Developments in Kenya and that country’s direct request for assistance presented the first actual opportunity for Integra to formalize an agreement to provide USAF assistance in SSA. Integra quickly learned, however, that Kenya’s requests were expanded beyond direct USAF considerations. The Permanent Secretary of the Ministry of Communications charged the Communications Commission of Kenya (CCK) with the lead responsibility for developing a National Broadband Strategy (NBS) for the country, a policy development much broader than universal service/access, but clearly one in which universal service/access would play a critical role. At that time, the Integra team’s planning activities had progressed to the point where they intended to select three or four additional countries in which to seek agreements for assistance. Although each of the four Core Countries had empowering legislation mandating USAFs, their stages of development varied significantly. All four had a USF organization, but only in Ghana and Nigeria were these organizations actually managing an actual fund. In Ghana, not only was there a dedicated organization for universal service, the Ghana Investment Fund for Electronic Communications (GIFEC), but there were also numerous universal service activities/projects that had already been deployed. In Nigeria, a universal service organization, the Universal Service Provision Fund (USPF), had implemented some rural tele-center projects, albeit with only minimal follow-up. Tanzania, on the other hand, had created the Universal Communications Service Access Fund of Tanzania (UCSAF, formerly UCAF), but it was staffed by only one person, and there was no functioning universal service program. In Kenya, USF responsibility was resident in the communications/ICT sector regulator, CCK. Although USFs had been included in the law since 2009, CCK had yet to collect any funds from operators, and was also embroiled in a legal challenge by broadcasters who did not believe that CCK had the authority to regulate that sector. Given this amount of variation in relative fund operational maturity, it was evident that the assistance to be provided in each country would need to be specifically tailored to that country’s stage of fund development. In addition to targeting the four Core Countries for assistance, Integra played a major role in three regional workshops, one co-sponsored by the CTO and two co-sponsored with Intel. These workshops were attended by USAF officials from many SSA countries, who were exposed via workshop presentations and discussions led by Integra to a sequential framework for developing and administering funds. Workshop topics included strategic planning, USAF program concepts, monitoring and evaluation and data collection/analysis. Attendees were also provided copies of the prepared training modules, which essentially amounted to a detailed set of templates and a framework for creating and effectively managing a USAF. Integra also prepared and delivered three one-hour online webinars covering the first three training modules. Webinars were attended by USAF representatives from as many as seven 6 See Annex 5 for a summary of Integra’s selection criteria. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 16 countries. Although participant feedback was favorable, the relatively small number of participants led Integra and USAID to conclude that developing and offering additional webinars was not the best use of remaining project funds. FINDINGS ABOUT USAF PROJECT PERFORMANCE ACHIEVEMENTS Integra’s success in completing its TOR deliverables in the four countries varied considerably. The team was able to satisfy requirements of the TOR agreements with Kenya and Nigeria. In Ghana, the intended level of analytical support had to be curtailed due to reasons explored more fully below. In Tanzania, a TOR was signed by UCAF and Integra, but it was only possible to complete minimal front-end work. Experiences in each of the four countries and in other universal service/access activities are examined in the following sections. KENYA CCK requested Integra’s assistance in late 2010, and after significant delays a “government-to￾government” MOU was signed in March 2012. MOU objectives were for Integra to support Kenya via: 1. Analysis and recommendations on the ICT gap analysis previously conducted by CCK; 2. Support for the development of a National Broadband Strategy, including supporting a public forum on the strategy; and 3. Assistance with implementation of the Universal Service Fund. When chartering the USAF Project, USAID did not anticipate providing assistance to a country in such an effort as the NBS development. However, given that the organization charged with USAF responsibilities had to manage the NBS effort, USAID agreed that Integra needed to include that work component in its assistance. It also became clear that the role and priorities of a USAF in Kenya would be defined by the NBS. According to input by Integra and CCK, NBS development quickly took precedence over other aspects of the MOU. Mr. Townsend developed a detailed framework and provided it to CCK for use as a guide in developing the NBS. Establishing a national steering committee and subsequent working groups were critical components of the framework. Including all of the significant stakeholders in the process, an innovation previously untried by CCK, was apparently the “spark” that led to the major ICT stakeholders joining the effort. CCK and Integra each expressed to the evaluator that the NBS development effectively raised public awareness of the importance of ICT development in Kenya. Prior to becoming immersed in NBS planning, Integra reviewed the 2010 gap analysis provided by CCK, and delivered foundation planning support for establishing the USF. After the NBS was approved and launched, attention returned to the remaining MOU components. Integra brought Mr. Parvez Iftikhar7 to Kenya to support CCK in further developing the operational framework, governance structure and detailed guidance for the fund’s pilot programs. At CCK, dTS interviewed Ms. Susan Mochache, Assistant Director – Universal Service Obligation; Mr. Muhatia Mutsotso, Universal Service Obligations & Funding; and the Director General, Francis Wangusi. 7 Mr. Iftikhar was CEO of Pakistan’s Universal Service fund, which successfully deployed universal service to remote areas of that country. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 17 These officials expressed their appreciation for the support given by the USAID Integra team in Kenya, and indicated they believed that continued support was needed by CCK. During the interviews, Ms. Mochache indicated that CCK had issued invoices to ICT providers in Kenya, excluding the broadcast segment, but funds have not yet been collected. dTS interviewed Mr. Muruiki Mureithi8, Integra’s local consultant supporting the USAF Project. Integra indicated that Mr. Mureithi’s constant presence and local network was invaluable in supporting the work in Kenya. The evaluation team was unable to schedule an interview with the former Minister of Information and Communications, who had assumed a different cabinet level position. An interview was held with the former Permanent Secretary of the Ministry, Dr. Bitange Ndemo, also Chairman of the Alliance for Affordable Internet (A4AI). He indicated that he was aware of Integra’s activities due to his daily contact with the Minister and ministerial personnel, but did not recall having been directly contacted by Integra. When asked about his perspective on the impact of Integra’s work in Kenya, he said that Integra’s output went to CCK, which then provided updates to him, and then from him to the Minister for policy considerations.9 He further stated that Kenya developed an ICT strategy and ICT board after the launch of the NBS, using the NBS as the foundation. When asked about what advice he would offer USAID in designing and implementing programs like the USAF Project, he used the example of IBM’s approach in Kenya. That company sends senior executives to the Ministry to ask what they want or need. The Ministry then identifies problems where assistance could be of value, and the company then decides whether or not to offer such assistance. He also stated that in Kenya, it is valuable to have a local point person who understands the industry involved and the local politics – clearly, here, referring to Mr. Mureithi’s involvement. dTS also interviewed Messrs. Mervyn Farroe and Albert Waudo, both USAID Kenya Mission personnel. Mr. Farroe said that Integra needed help facilitating the MOU, and that he managed this process. He mentioned that it took nearly a year to get the MOU signed. Mr. Farroe indicated that the USAF Project was not a “heavy lift” for the Mission and that Integra never asked for anything beyond MOU assistance. Mr. Waudo indicated that his involvement in the Project was purely administrative. Mr. Nicholas Nesbitt was listed on Integra’s Attachment C to its Final Report to USAID as a contact. He was President of KenCall during some of the USAF Project period, and is currently the Country General Manager for IBM. Mr. Nesbitt did not recall being contacted by Integra, nor was he involved with the project in any way. He suggested contacting Victor Kyalo, ICT Authority CEO, and Dr. Kate Getao, who is focusing on E-Government at the authority. The dTS evaluation team was not able to arrange interviews with either of these people while in Kenya. Integra clearly delivered on objectives 1 and 2 of the MOU, and provided support that would have resulted in the implementation of the USAF (Objective 3), had the Project’s time limitation not come into play. 8 Mr. Mureithi is CEO of Summit Strategies, Ltd., an ICT consulting firm in Nairobi, Kenya. His many years of ICT experience in Kenya and other parts of Africa, and his local networks in Kenya’s ICT arena made him an optimum choice as Integra’s local presence. See the writeup of Mr. Mureithi’s interview in Annex 9 for additional information. 9 Dr. Ndemo also mentioned his concern that Kenya needs to develop a spectrum policy several times during the interview, and that stated that such policy should be open - that it will be critical to deal with the so-called “white space” frequencies. His view was that any such frequencies awarded to a vendor ought to be returned if not used within one year, and that the total award process must be handled in a public and transparent way. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 18 NIGERIA Nigeria had an established USPF, and had collected and disbursed funds to support primarily rural universal service projects. At the time of USAF Project inception, the USPF had a surplus of about USD $100M (i.e., funds not yet disbursed to provide universal service applications). Integra learned that outside evaluation indicated that less than half of Nigeria’s previous universal projects had achieved their objectives. The recently appointed USPF manager, Mr. Abdullahi Maikano, was given the responsibility to create a new strategy and improved processes for the fund’s operation. He requested Integra’s assistance, and a TOR was agreed to and signed in February 2012. Three main components were included: 1. Support to USPF for current and planned institutional development initiatives; 2. Assistance to develop and execute an ICT Access Gap Study; and 3. Advice on developing and implementing Monitoring and Evaluation (M&E) procedures. Three deliverables emanating from these assistance components were intended to be completed within eight months of launch, and were stated as follows: 1. Review and evaluation of consultancies on Strategic Planning, Impact Assessment; 2. ICT Access Gap planning and data requirements, statistical model file(s), and final analysis report; and 3. USPF Monitoring and Evaluation Recommendations Report. Integra’s responsibilities were to manage and oversee project activities in cooperation with the USPF organization, and to provide inputs, examples, analysis, and recommendations for each assistance component. It was also to participate in meetings, discussions, consultations, and other relevant communications with USPF and other stakeholders in Nigeria. USPF was to ensure adequate staffing and assignments to allow effective collaboration and implementation of the technical assistance provided by Integra. USPF also was to pay for any Integra participation in retreats or other meetings outside of Abuja, and to provide access to data resources, collect additional data as needed, and facilitate stakeholder consultations, as well as review and contribute to all analysis, recommendations and reports. The dTS evaluation activities in Nigeria required two visits to the country. The first visit was shortly before the 2013 Christmas holidays; many of the government officials targeted for interviews were not available. In particular, the Minister of Communication Technology, Hon. Madame Omobola Johnson, was not available during the initial visit. Because of her significant involvement in stimulating the universal service function, a second visit was scheduled to interview her and other officials who were not available during the first trip. The Minister stated that she was introduced to Integra via Professor Raymond Akwule, and that Integra contacted her to discuss potential activities to support USAF. Integra showed her examples of what had been done in other countries. She indicated that the greatest impact of Integra’s work in Nigeria was to help making the new National Strategic Plan for Broadband more realistic, and to show the USPF organization that, with training and good follow-up practices, they did not need to always rely on outsourcing to accomplish activities; they could handle projects internally. She further said that Integra played a “big role” in the retreat held to design the new national strategy, and that they also provided subsequent assistance. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 19 dTS interviewed the USPF Manager, Mr. Abdullahi Maikano, and Mr. Kelechi Nwankwo, Head of Strategy and Corporate Performance Monitoring. Mr. Maikano was joined in the interview by the Head of Connectivity, Engr. Asaju, previously Head of Infrastructure. Mr. Maikano and Engr. Asaju clearly stated their opinion that the new strategic plan’s success was due to the assistance provided by David Townsend and Parvez Iftikhar. Mr. Maikano said that under the old strategic plan, they supported 54 base stations over a five-year period, whereas under the new plan, they were able to implement 20 base stations in just one year’s time. He and Mr. Nwankwo described the transparent processes that were recommended by Integra and adopted by USPF for implementing bids and awards for subsidized projects. Mr. Nwankwo also emphasized that Integra’s assistance caused USPF to change its focus, such that efforts were better targeted to areas where the need was observable. He said that the community center model was improved after the retreat, and that USPF was now able to design a prototype model community center that clearly delineates how the community center is to be funded and must be operated. He was emphatic that M&E will take place for all USPF projects. USAID personnel interviewed in Nigeria included Sharon Pauling, Director of Economic Growth & Environment Office (and Acting Mission Director at the time) and Mr. Toks Aluko, an ICT expert. Also interviewed at USAID were Alefia Merchant and James Lykos, and on the second visit to Nigeria, Mr. Roland Ohroh – EGAT. Ms. Pauling indicated that she was briefed by USAID/W on the USAF Project, but had not been directly involved in the project. Mr. Aluko was aware that Integra provided assistance in the development of the broadband policy. When Mr. Ohroh was interviewed, he informed dTS that he had not been involved in the Project, but that he had some awareness as a result of his involvement in the NEXTT Project. The connection with Integra is that Eric White was working on the ICT aspects of road improvements on the NEXTT Project. At the suggestion of Ms. Pauling and Mr. Aluko, dTS arranged interviews with the NEXTT Program Chief of Party (COP) and also with Markets II Program representatives. In the NEXTT interview, Mr. Monoghan provided a presentation explaining the program, whose prime objective is to improve commerce along a North-South corridor in Nigeria and to improve the Nigerian brand worldwide. For the Markets II Project, dTS interviewed Mr. Gordon Ononiwu, Director, External Relations & Capacity Building (in lieu of COP, Mr. Harvey Schartup, who was out of town). Mr. Ononiwu described the project as having the goal of transforming agriculture in selected sites from a sustenance operation to a commercial farming endeavor. When asked about the role of ICT in the project, he indicated that there has been much talk about the use of ICT in agriculture work, but that it is very slow in arriving. He also cited the problem that rural farmers have a low level of interest, likely owing to lack of power sources in many rural areas. dTS also arranged an interview with the IFDC/AFAP Country Representative, Mr. Scott Wallace, who is supporting the Ministry of Agriculture’s Growth Enhancement Support (GES) program – a five-year plan to get government out of the agriculture business and transform Nigerian agriculture into a commercial market sector. This project is apparently a sub-contract of the Markets II Project. He indicated that in 2013, the program reached 4.5 million of the base (target) of 7.8 million farmers. IFDC provides pelletized fertilizer for deep penetration, where nutrient benefits are slowly released over time, as opposed to the “broadcast” method of fertilizing, where all of the substance is spread simultaneously. The deep penetration method has resulted in a 35 percent increase in rice crop yield where it has been deployed. As a related aspect of this program, Mr. Wallace said that the Minister of Agriculture wanted to provide 10 million mobile phones to rural farmers in 2013, but he did not think that goal had been achieved. He also mentioned that the lack of energy infrastructure in some areas is an impediment to Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 20 farmers using cellular phones. Mr. Wallace explained the current paper process used to identify and classify farmers in order to manage the process of getting seeds and fertilizer to them. It is apparently not working very well; a mechanized platform for the program is under development. He stated that he believes the United Kingdom’s Department for International Development (DFID) was in the final stages of funding two pilot projects to improve the interface processes with farmers. dTS learned that neither the Markets II nor IFDC representatives were aware of the USAF Project in Nigeria, although ICT would clearly play a role in both projects. The final interview in Nigeria was with Professor Raymond Akwule, President of the Digital Bridge Institute. He stated that he heard about the GBI effort at an ITU meeting, and that his involvement had been primarily to advise the Integra team on who to contact in Nigeria and how to negotiate the political environment. His office served as a “war room” for the team to prepare contacts, perform analysis and devise strategy. He introduced Integra to the Minister and appropriate NCC and USPF contacts. Although he said he had not been actively involved in the actual assistance provided by Integra, he was aware of their role in the USPF retreat to work on national strategy and the digital divide issues. dTS found that Integra was able to meet its TOR objectives in Nigeria. GHANA In December 2011, Integra and GIFEC formalized a TOR10 for “Technical Assistance to help close the Digital Divide: Internet/Broadband Penetration in Ghana.” Objectives were to provide assistance that would enhance the capacity and information resources of GIFEC and the Ministry of Communications toward understanding the nature and scope of the “digital divide” in Ghana, and to improve policies and practices to overcome such gaps in ICT access. Areas to be addressed included indicators for measuring penetration and diffusion of services, factors accounting for the divide in Internet and broadband usage, and which near-term policy initiatives should be undertaken by GIFEC. The TOR also included a goal of assisting GIFEC in enhancing the agency’s functions and staff capacity. Activities to be completed were: (1) Data collection methods and implementation; (2) Market and gap analysis; and (3) Policy recommendations. Four specific deliverables were indicated in the TOR: 1. Report on data collection framework, objectives, methods and procedures; 2. Report on results of market study and survey; 3. Report on market and gap analysis; and 4. Report on policy recommendations. These deliverables were to be concluded within six months of project launch, around August 2012. Integra’s responsibilities were to prepare the analytical framework and data collection methods; provide guidance, training, and oversight of data collection; conduct data analysis; provide policy recommendations; and prepare all reports. GIFEC’s tasks were to recruit and organize data collection study team(s); facilitate implementation of data collection tasks; and review and contribute to all analysis, recommendations and reports. 10 A copy of the TOR for Ghana and the other three Core Countries is contained in Annex 5 to this report. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 21 The Integra team prepared a preliminary study report outline and a design for a comprehensive survey. It also identified, in detail, the secondary data that was necessary, and supported GIFEC in developing the survey TOR, which was to be contracted by the Ministry of Communications. GIFEC selected ISSER to conduct the survey. Significant delays were experienced in finalizing the contract with ISSER, however, and survey work did not start until May 2013, some nine months after Integra’s assistance to Ghana was to have been completed. The GIFEC Administrator/CEO indicated to the evaluator that the delay was caused by ISSER wanting more money to conduct the survey than originally agreed. Integra further informed the evaluator that there was a “back-and-forth” period where ISSER wanted to modify survey questions, and additional negotiations took place in that regard. Finally, Integra advised the evaluator that ISSER refused to provide a copy of the completed evaluation, and prepared a series of survey reports on its own. The evaluator was unable to obtain an interview with the ISSER representative responsible for the survey (presumably due to the team visiting Ghana shortly before the Christmas holidays, when many people take early leave to visit their home villages). Another obstacle encountered in assisting Ghana was that the secondary data identified as a requirement for the intended analyses was not obtained by GIFEC. It was believed that the National Communications Authority of Ghana (NCA) would have the needed information. It turned out that NCA does not collect that information. Ultimately, a small amount of secondary data was obtained from ICT service providers, so that Integra advised the dTS evaluator that it was able to enhance the contents of its report. The dTS evaluator was able to conduct an in-depth interview with only one GIFEC representative, Mr. Alfred Boyan, Director of Finance and Administration. At the Administrator/CEO’s request, the interview with Mr. Boyan was interrupted toward the end, so that the GIFEC team could give a prepared presentation to the evaluator. Although not particularly conducive to investigating the impact of Integra’s performance in Ghana, the evaluator viewed the presentation – a picture of activities Ghana has undertaken in the universal services area – with occasional comments and/or clarifications. Following the presentation, the evaluator was able to conduct very short one-on-one interviews with Mr. Attor and Mr. Prempeh, GIFEC CEO and Business Development Manager, respectively. It was not clear to the evaluator that Mr. Attor made any distinction between assistance provided on the strategic plan by Messrs. Townsend and Boayke, consultants under CTO sponsorship, and the assistance provided by Integra. Given Mr. Townsend’s significant involvement in both activities, that is understandable. In addition to the GIFEC session and interviews, dTS interviewed two other government officials in Ghana. Mr. Issah Yahaya, Director, Policy Planning, Monitoring and Evaluation at the Ministry of Communications (MOC) stated that Ghana has transitioned to a broadband strategy and policy for ICT, and that broadband implementation is the responsibility of the National Information Technical Agency (NITA). He then described the current technical ICT developments in the country. He recommended that dTS interview NITA General Director, Mr. William Tevie, and also the NCA Chief Director. dTS was able to briefly interview Mr. Tevie, but was unable to contact the NCA Chief Director. When asked if they were aware or had been contacted during the USAF Project, Mr. Yahaya and Mr. Tevie both said they were not aware of the Project in Ghana. Mr. Tevie stated that he is responsible for planning and implementing all government ICT projects, and cited infrastructure projects in Ghana currently being funded by China, Denmark and the World Bank. He declined to elaborate on the relationship between GIFEC and NITA; the evaluator remains unclear on the distinction and possible coordination roles of these two organizations. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 22 dTS was advised by Integra that the USAID Mission in Ghana had not shown any interest in the USAF Project;, so that one objective of the Ghana visit was to inquire to USAID why interest was lacking at the field level. Since the current Mission Director was assigned to Ghana after the USAF Project period, Ms. Flanagan arranged a dTS interview with Ms. Kim Yee, the Private Enterprise Officer at the Mission. Ms. Yee stated that she had not had exposure to the Integra program, but was aware that Integra had met with a person in the Economic Growth section. Her view was that, at the time Integra was launching and working to support universal service/access in Ghana, the USAID Mission was “struggling” to meet the requirements of the USAID Feed the Future program, and was engaged in an “all hands on deck” effort to support that initiative. She said that the USAF Project timing simply had not been “great,” as no resources could be made available to support project activities. The evaluation team has not seen ISSER’s survey reports, but according to Integra, once it was able to obtain a copy of the completed survey, it was able to provide GIFEC and MOC an improved analytical report on survey findings. The dTS evaluation team found that Integra succeeded in completing Objective 1, but that, due to the significant time delay, lack of full cooperation by ISSER, and unavailability of all the secondary data requested, Objectives 2 through 4 could only be partially completed. TANZANIA In January 2012, Integra and the Universal Communications Access and Service Fund (UCAF, now Universal Communications Service and Access Fund of Tanzania (UCSAF))11 signed a TOR for assistance with the following three components: 1. Market Assessment; 2. Strategic Planning Assistance; and 3. Implementation. Integra deliverables for this TOR were: 1. Report on Market Assessment – to be conducted jointly via UCSAF staff – supported by Integra. The assessment was to include gap analysis components based on review of existing and available data sources and tools, and would result in an analysis or the market to support UCSAF strategic planning. 2. Strategic Planning Assistance – This deliverable was intended to be the main focus of Integra’s technical assistance to UCSAF, and included the steps necessary to develop a draft strategic plan and several other components. 3. Implementation Support – to assist UCSAF in rolling out the Strategic Plan, including organization design, potential near-term project potentials, and any subsequent recommendations, as needed, related to policy options, collaboration initiatives and other opportunities for UCSAF action. Integra’s responsibilities were to manage/oversee project activities in cooperation with Tanzanian personnel, and to provide needed work items to produce the deliverables described above. UCSAF was to ensure adequate staffing and personnel assignments to “allow effective collaboration and implementation” of the agreement. Unfortunately, other than the CEO, Mr. Peter Ulanga, key UCSAF 11 It appears that the name of the fund in Tanzania has been changed to: “Universal Communications Service Access Fund (UCSAF)” based on the business card provided by Peter Ulanga to the evaluation team. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 23 staff members were not hired during the Task Order 1 timeframes; Integra had to suspend further assistance. Integra did prepare a framework for development of the strategic plan and for a market assessment. UCSAF staff personnel were able to attend some of the Integra webinars delivered under Task Order 1, and also attended the final project workshop. dTS visited Tanzania in mid-January 2014 and interviewed six people, starting with the Minister of Communications, Science and Technology, Hon. Makame Mbarawa. The Minister said that he had never worked with USAID, and that there have been no contacts on a government-to-government basis. He was not familiar with the USAID USAF Project, but he did address the UCSAF staffing delay. His statement regarding the delay in staffing indicated that it was caused by the lengthy vetting process when hiring government employees. Although he indicated that he had not been informed of any specific delays, he emphasized that UCSAF is an independent agency. He mentioned that he believed Mr. Ulanga was very involved in various tender projects and was under significant pressure in that regard. He provided information about the history of USF in Tanzania, and indicated that carriers pay into the fund (except for Internet service providers (ISP) who are not yet paying, as there is concern that small ISPs might not survive if required to pay the 0.3% assessment). The Hon. Mbarawa stated Tanzania needs regulation to decide how to utilize the “white space” spectrum resulting from the analog-to-digital TV conversion, and that such regulations are needed by the end of 2014. Details of his description of the various government tenders related to universal service are contained in the interview write-up in this report’s annex. The Minister also indicated that a revised national broadband strategy was currently under development, with a white paper due by July 2014. Mr. Ulanga and others are involved in that effort, which includes planned visits to Nigeria, Kenya and Ghana. Finally, Minister Mbarawa stated that he believes Tanzania and USAID need to work together in these areas, and that he would welcome contact from USAID in that regard. UCSAF had been staffing over time, and Mr. Ulanga had five of his staff join the interview. He said he had tried to work with Integra via informal networking, then signed a TOR in January 2012. No work actually took place because he did not have the needed staff. His first resource did not join until August 2012, followed later by additional personnel. When asked for the reason for the delay, he cited bureaucratic government processes, and indicated he had been using Ministry staff on a loaned basis, but none had ICT skills. James Kilaba (PREng), Deputy Director, ICTs Development, Tanzania Communications Regulatory Authority (TCRA), met with dTS despite being on vacation. He said that he had heard about Integra’s work in May 2012 while attending ICT Week12, and that he had gotten the impression that the TOR was universal, rather than just for Tanzania (possibly a misunderstanding arising from someone’s description of the overall GBI program, under which Integra’s country-specific work was funded). In addition, he had attended the CTO workshop in Dar es Salaam (an event that took place early in the USAF Project, where Integra provided USAF presentations). He recognized the need for Tanzania to review its ICT policy to include broadband. He also stated that TCRA recognized the need to be ready to address spectrum availability, and could certainly use assistance in determining how best to deal with spectrum resulting from the analog-to-digital conversion, indicating that this issue needed to be addressed by the 12 Mr. Kilaba did not elaborate about ICT week, and since it was incidental to the conversation, dTS did not probe to determine what organization sponsored the event. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 24 end of 2014. Mr. Kilaba, when asked about the current prices charged for use of the fiber backbone, stated that the network is not finished; he believes current prices are transitional rather than final. His understanding was that the Ministry was currently receiving assistance from an outside source in developing the broadband strategy and policy, but was unaware of specifics. Mr. Mihayo Wilmore, Managing Director, Uhuruone, was interviewed, and indicated that he had met someone in a Singapore meeting where white space was discussed. He and Bill Garrison of NetHope talked for some time about how to get broadband into schools in Tanzania. He did not clarify how any other contacts with Integra took place, but did mention that he made a presentation about ICT in Education at the Intel/Integra Zanzibar regional workshop. His impression was that Integra acted only as “consultants” rather than “implementers” like NetHope. He indicated that he has had regular email communication with Eric White of Integra. Mr. Wilmore then described efforts to bring broadband into schools in Tanzania; details can be found in Annex 9 to this report. Ms. Kristiina Lahde, Chief Technical Adviser of the Tanzania Information Society & ICT Sector Development Project (TANZICT), was also interviewed. TANZICT is funded by the Government of Finland with the objective to support development of an information society in Tanzania. Although Ms. Lahde had no awareness of or involvement with Integra, she offered insights into some of the internal political aspects of Ministry and ICT functions in Tanzania, and described the work her office had done, and the current evolution for the Tanzania Commission for Science & Technology (COSTECH) to take over the ICT project. Her description of various activities in community outreach/involvement and other ICT efforts are contained in the write-up of her interview in Annex 9 to this report. dTS interviewed Kevin McCown, USAID Policy and Research Advisor at the Mission. He indicated that he learned of the USAF Project approximately a year ago, first from NetHope, then through briefings from Jeff Cochrane (USAID/W) and Bob Otto (he also mentioned an earlier Mission briefing on the project by Bob Otto and Judith Payne – USAID/W), and that Jeff Cochrane had been keeping him up to date on the Project’s status. He understood the Project had not had much impact due to lack of UCSAF staffing. Mr. McGown also stated that he put Integra in touch with a number of potential partners for moving forward in Tanzania, and expressed appreciation for ongoing updates from USAID/W. dTS is aware that Integra provided a template for the strategic plan when it attempted to begin working on TOR deliverables, and that repeated contacts were made with UCSAF to inquire as to anticipated dates for hiring of staff. dTS found that there appears to be a desire on the part of the Minister for USAID assistance, but that UCAF’s only expressed desire for potential further assistance would be for support with market analysis activities. Integra was unable to meet its objectives in Tanzania due to the present country environment and politics, and not through any fault of its own. COST-EFFECTIVENESS MEASUREMENTS - FINDINGS GENERAL One of the objectives of the dTS contract was “to evaluate which approaches are most cost effective, to identify key considerations when selecting approaches, and to provide some assessment of the relative effectiveness of approaches to meeting Project objectives.”13 13 USAID, Task Order Award to dTS (April 12, 2011) Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 25 The objectives of the USAF Project were: to “improve the practices, and increase the actual use of […] telecommunications universal service and access funds in sub-Saharan Africa.” The project aimed to: 1) Increase the actual use of telecommunications USAF in the target countries; and 2) Accelerate the extension of affordable and sustainable access to telecom services to those not served.14 A cost-effectiveness analysis compares the cost of an intervention to its effectiveness (Owens, 1998).15 Ideally, one could look at Integra interventions on a per-user basis and create a cost-effectiveness ratio. In this evaluation effort, a variety of reasons make it impossible to create such a ratio. First, Integra could not provide participant names in the three regional workshops, and dTS was not able to contact them. Second, strategy and policy interventions were not directed at individual users; it is therefore not possible to measure end-user benefits. Finally, it is too early in the process of establishing and improving USAFs to be able to calculate the number of direct beneficiaries related to project interventions. Two measures that are not based on individual users were analyzed: (1) Amount of (additional) money committed to universal access, identified by Integra, as a primary measure of cost-effectiveness; and (2) Number of new policies and/or strategies adopted during or after the project. The first uses a quantitative measure produced from the level of inputs (i.e., financial resources). The second is a qualitative measure based on key informant respondents regarding the number of strategies and policies adopted. Both are reasonable measures when the maximum amount of output (i.e., improved USAFs) has not yet been achieved by the project. Since it was not possible to establish a cost ratio, this section presents the information about funds leveraged as a result of the project (Table 1). The USAF Project had funding of $850,000 to improve USAFs functioning in SSA. The actual interventions achieved by Integra in each country are discussed, and those accomplishments are presented alongside the funds expended to reach the objectives. Table 1: Status of USAF Funds in Target Countries Prior to Integra Intervention (2010) Country Legal Requirement of USAF Established USAF Administration Established Funds collected Amount of Funds Collected in USD Calculated Percentage for most recent full year of sector contributions Ghana 2005 2005 2005 20,994 1.1% Nigeria 2003 2007 2004 146,802 .6% Tanzania 2006 NA NA NA NA Kenya 2009 NA NA NA NA Source: Adapted from Sepulveda (2010) by the author. 14 dTS, (October 30, 2013) Evaluation of Universal Service Funds, Evaluation Methodology, provided to USAID. Washington, D.C.: dTS. 15 Owens, D.K. (1998) Interpreting Cost Effectiveness Analyses, Journal of General Internal Medicine. Volume 13, Issue 10:716-717. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 26 According to Edgardo Sepulveda, in a report to ITU (2010) regarding realizing the full benefits of a USAF, countries must, at a minimum:16 1. Pass legislation and/or establish regulations needed to create a USAF; 2. Establish a fund with an entity to operate it; 3. Collect funds and keep their identity separate from general government funds; and 4. Disburse funds for their intended public purpose. This short list is a useful benchmark for determining how far a country has progressed toward creating a fully functional USAF. Kenya Kenya was the most expensive of the four USAF implementation countries, with a total cost of $256,000.17 When Integra became involved, Kenya was beginning to establish the USAF process. The Kenyan USF was legally authorized in 2009, but was not operational when the Integra project began. Integra provided technical assistance in planning, designing and adopting Kenya’s NBS. The Integra team helped establish the NBS policy framework, procedures, key issues, and objectives, which the Government of Kenya announced and launched. According to Integra’s final report, this strategy commits the government to provisioning 5% of its annual budget (a total of USD $1.1B) through 2017 toward implementation of ICT programs. In 2012, USAID entered into an MOU with the Kenyan government. Through this partnership, the government was able to develop a detailed NBS. Other work on Kenya’s USF was placed on hold during this time. Kenya’s USF has not yet been fully implemented. The Universal Service Advisory Council (USAC) was put in place, an important step to ensuring Kenya’s USF is accountable to both donors and taxpayers. Unfortunately, new legislation was necessary to bring the fund into compliance with the new Constitution, and USAC was disbanded. A new council has now been appointed by the Cabinet Secretary. CCK had issued invoices to ICT vendors for universal service fees, but will need to reissue them. The core management and operational frameworks for the fund have been established, and the framework and plans for pilot project implementation are in place. In summary, Integra assistance in Kenya was helpful in moving the country forward toward creating the structures and institutions required to establish an effective USAF operation. Nigeria In Nigeria, Integra used the second highest total funding, receiving $189,72018 over the life of the project. Integra provided direct technical assistance to improve USAF functioning, including the monitoring and evaluation process. 16 Edgardo Sepulveda, (2010) Report on Universal Access and Service Funds (UASFs) in the Sub-Saharan Africa (SSA) Region to the International Telecommunications Union. Toronto: Sepulveda Consulting. 17 Integra (2013) Global Broadband and Innovations Program, Final Report for the Africa Universal Service and Access Fund Support Project submitted to USAID. 18 Integra final report (2013) Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 27 First, Integra provided support to the Nigeria USF, known as USPF by helping the country develop its new Strategic Management Plan (SMP) for 2013 - 2017. Second, USPF allocated USD $82.8M in FY2013 toward projects including extending the fiber backbone and cellular network coverage. Third, Integra provided technical assistance to USPF management and staff. In summary, Nigeria received the second highest amount of assistance of the four Core Countries. Through Integra’s work, Nigeria was able to move from a functioning USAF, to a much more efficient USAF, with a significant amount of funds allocated to universal service projects. Importantly, Nigeria has adopted all of the guidelines and recommendations provided by Integra. Ghana In Ghana, Integra expenditures were relatively low at $88,484,19 the second lowest amount expended. The effort in Ghana was assessing the implications of the nation’s digital divide at the request of Ghana’s MOC and the Universal Service Fund Administration. The Ghanaian USAF was legally created in 2005; a governing entity was also established that year. Monies were first collected by the USAF in 2005, and first disbursed in 2007. As of 2010, Ghana had an established administrative entity and account in place. This entity is an independent agency, with a board of trustees, a day-to-day administrator, a reasonably￾sized staff and a strategic plan in place. Efforts in Ghana began in 2011, when Integra entered into a TOR with Ghana’s MOC and GIFEC. Project work focused on developing indicators for measuring penetration and diffusion of the Internet in Ghanaian society, determining what factors contributed to the digital divide in Internet usage, and what mid-term policy initiatives GIFEC should embrace to improve Internet and broadband penetration. The Integra team helped GIFEC design, implement, and analyze the national broadband digital divide survey. GIFEC representatives also attended all GBI workshops on USAFs. GBI encountered significant difficulties in Ghana, which impeded its work. Integra was not able to obtain digital divide survey results in a timely manner, and did not receive the secondary data that was stipulated as necessary for developing the analysis and report in the country TOR. In summary, Ghanaian assistance was in the middle of the range. It was effective, although Ghana had a functioning USAF at the time Integra’s assistance began. Integra’s assistance helped the country move toward improved implementation of the USAF. In addition, GIFEC representatives received valuable training through their work with Integra. Tanzania The lowest amount of funds was spent in Tanzania: only $36,294.20 In Tanzania, a TOR was signed, and the CEO of the UCSAF organization was enthusiastic about partnering with Integra. Integra engaged with the UCSAF CEO and agreed to provide technical assistance in the establishment, launch, and capacity building of the fund. Integra provided technical support for preparation of the initial strategic plan. In addition, Tanzania hosted two regional workshops. UCSAF and personnel benefited significantly from extensive training in those workshops, as well as in webinars. 19 Integra final report (2013) 20 Integra final report (2013) Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 28 Unfortunately, the UCSAF CEO was unable to get staff on board during the tenure of the USAF Project; minimal time and funds were ultimately spent in that country. GBI assistance was accordingly suspended after the government did not approve UCSAF staff hiring within the project timeframe. Tanzania received the lowest amount of assistance from GBI, and also did not show much change based on the intervention. Staff, however, did benefit significantly from training provided by GBI. FUNDS LEVERAGED In two of the countries where Integra provided assistance, governments made significant commitments in ICT-related project funding. In Kenya and Nigeria, the technical assistance provided by Integra contributed significantly to the countries’ commitment to spend the amounts shown for ICT and/or universal service. In Kenya, the NBS commitment is to ICT, but - by definition - increasing broadband access is a major component of increasing universal access which, in turn, leads to increased universal service applications (people served). Accordingly, a compelling argument can be made that USAID funds spent supporting these two countries was cost-effective. Although USAID assistance cannot be directly equated to a total "cause and effect" regarding the planned spending in these two countries, the technical assistance was certainly one of the major stimulants leading to the plans. Table 2: Funds leveraged due to Integra Intervention21 Cntry Planned Committed Tendered Contracted Project Total Funds Leveraged Spent/Lever aged Ratio* Nigeria $19,900,000 BTRAIN Project $35,643,075 139 times more funds leveraged than invested $3,148,615 USPF Gap Analysis $12,594,460 BTS Project budget Kenya $80,964,706 2013/14 National budget line item for ICT Infrastructure Development $101,964,706 539 times more funds leveraged than invested. $9,100,000 Operationalizati on of USF (collections) $11,900,000 Seed capital expected to be utilized for the initial projects that shall benefit from GBI's TA (2014) * Ratio of funds spent by Integra to funds leveraged. The total ratio is 162/1 ($137,607,781/$850,000). Source: Nigeria: Secretary USPF; Kenya Treasury of Kenya, and Communications Commission of Kenya. 21 The leveraged funds shown in Table 2 were confirmed by principals in the two countries and represent only a single year’s planned expenditures, i.e., subsequent years’ investments are not included. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 29 STRATEGIES AND POLICIES ADOPTED The second measure of cost-effectiveness is based on qualitative data that is shown in Table 3. Table 3: Qualitative Interventions by Integra to Strengthen USAF Plan Country Strategic Plan in Place in 2010? Strategic Plan in Place in September 2013 After Integra Intervention? USAF Functioning Improved After Integra Intervention? Description of Specific Manner to Improve USAF Functioning? Ghana Yes Yes, and improved. 22 Yes Developed National Broadband digital divide questionnaire & analyzed results Nigeria Yes Yes, and improved existing strategic plan. Yes Helped draft Strategic Management Plan for 2013-2017. Tanzania No No, but partial progress made. No Prepared initial UCAF Strategic Plan. Kenya No Yes, National Broadband Strategy. July 23, 2013. No Established NBS policy framework, procedures and objectives and Universal Service Fund institutional structures and processes. 22 When the Integra Ghana project was initiated, Mr. Townsend was working on a CTO-sponsored project in Ghana to support revising the strategic plan. The CTO and USAID projects progressed concurrently, but Integra’s assistance was not directed toward the strategic plan work. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 30 VI. CONCLUSIONS GENERAL CONCLUSIONS The USAID USAF Project sought to provide technical assistance in SSA to improve universal service/universal access funds’ performance. Given the amount of funding dedicated to the project ($850,000) and the intended project timeline (12 to 15 months), it would clearly have been impossible to improve USAFs in every SSA country. Recognizing such practical limitations, USAID accepted Integra’s recommended approach to narrow the focus of assistance to four Core Countries. However, the USAID TOR made it clear that the maximum regional impact possible was desired, such that USAID encouraged the implementer to develop innovative approaches to reach the widest possible regional audience. Integra answered the call by developing the workshops, modules and webinar mechanisms discussed further in the following sections. The USAID TOR also indicated that project results should be increased coverage of universal service. REGIONAL IMPACT Integra developed an approach that resulted in the majority of assistance being focused on individual countries (addressed in the individual country sections). Nevertheless, dTS concludes that, on balance, Integra met its deliverable requirements to develop and roll out programs that had a regional impact. The co-sponsorship, preparation and presentation/engagement in three regional workshops attended by representatives of many SSA countries provided ample material for attendees to absorb and take home for implementation. Additionally, the three webinars and four “Tool Kit” modules that were developed and made available were targeted to regional audiences, and were well received by participants and recipients. UNEXPECTED CONSEQUENCES As with most complex undertakings, the unfolding realities on the ground did not precisely align with initial project plans. For example, in Kenya Integra had embarked on a pre-assistance assessment of potential target areas for engagement, but the CCK request for immediate support – and the subsequent request for NBS development assistance – required adjusting the planned project rollout sequence. Integra and USAID demonstrated flexibility by allowing a modification of the plan when the actual situation on the ground required technical assistance other than what had been estimated. dTS concludes that Integra and USAID made the right decision in providing technical assistance to the Kenya NBS development. PRIMARY FACTOR CONTRIBUTING TO SUCCESS On an overall project basis, dTS believes that where Integra was able to accomplish its TOR objectives, it did so by applying human resources of the highest caliber, primarily Messrs. Townsend and Iftikhar, who brought in-depth subject matter expertise and on-the-ground experience to the effort. Furthermore, key in-country officials in the four Core Countries, stated their desire for continued support from the team. COUNTRY SPECIFIC CONCLUSIONS Performance results in the four Core Countries and other assistance projects are addressed in this section. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 31 KENYA Ms. Mochache of CCK stepped forward at the CTO/Integra workshop (December 2010) to request assistance from Integra in launching Kenya’s USAF. At some point between December 2010 and April 2011, there was a mandate from the Ministry for CCK to lead the development of a broadband strategy for the country. As a result, Ms. Mochache requested Integra’s assistance to move forward with what became known as the National Broadband Strategy (NBS). Integra had already prepared some of the framework and planning support items to assist setting up the Kenyan USAF, but it was clear that no further progress in that regard would be possible until the NBS was developed and promulgated by the government. Though unexpected, Integra and USAID decided that supporting CCK on the NBS was a good idea, especially since universal service/universal access would, by definition, be an integral consideration in any national strategy involving broadband. Another consideration was that Integra felt that a robust NBS in Kenya could also serve as a potential model for other countries that did not yet have such a national strategy or policy. Mr. Townsend’s insertion of the notion to engage all stakeholders, including ICT vendors, was an innovation for Kenya in stakeholder management, and the relative buy-in of vendors to support the effort can be attributed to the all-inclusive public process adopted. dTS concludes that providing in-depth technical assistance to Kenya in the NBS development, and including public comments and stakeholder review sessions as critical steps in the NBS development process was the optimum decision at that time, and that Integra’s performance supporting that effort was outstanding. To refuse to provide such assistance would effectively have eliminated Kenya as a candidate for USAF support within the project’s intended life span. Since Kenya is one of the leading East African countries in many ICT aspects,23 excluding it from the project would have left a noticeable hole. dTS further concludes that the objective to provide analysis and recommendations on the ICT gap analysis previously done by CCK was fully satisfied. The final objective in the TOR (assistance with USAF implementation) has not yet happened. Integra provided the policy, framework and specific governance guidelines to support the fund establishment, which was expected to be fully operational by the end of 2013. That has not yet happened. dTS learned in late July 2014 that a new advisory council has been appointed, which should remove any impediments to issuing invoices to the vendors for universal service fees and moving forward with USAF implementation. dTS concludes that Integra’s efforts supporting CCK in getting the USAF up and running were performed very well. NIGERIA The timing of USAF Project involvement in Nigeria could not have been better. With a new Minister of Communications and Technology and new USPF manager – both with a genuine commitment to the mandate to accelerate ICT access and adoption across the country – Integra’s assistance was readily welcomed. Nigeria had collected the largest amount of funds of the 35 countries responding to the 2010 ITU study.24 By 2011, however, it had a surplus of USD $100M in funds that had been collected but not 23 In addition to being the home of the M-Pesa cellular “banking” initiative, Kenya has implemented a robust ICT regulatory environment, and played a leading role in establishing the East African regional fiber optic network. 24 Ibid: Sepulveda, Table 9, Page 21, Nigeria Total Sector Contribution $146.802 (USD – Thousands) Total Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 32 disbursed for universal service projects. The Ministry had hired the local KPMG firm to assist in SMP development, and Integra was invited to participate in a lengthy retreat. The USPF manager and Minister requested Integra’s assistance in developing the new strategic plan. Much of the retreat’s success and improvements in the new SMP are attributed to Integra’s input and work items during that retreat. Integra also provided USPF with M&E training and detailed guidelines, which appear to have been implemented. dTS concludes that the objectives in Nigeria were all satisfied in a manner that significantly contributed to the increased efficiency and transparency of USPF functioning. Nigeria is a complete success story for the USAF Project – and the impact of Integra’s work will continue to be felt in ongoing initiatives in the country. GHANA Integra provided GIFEC with a preliminary study report outline, designed the survey questionnaire, identified the secondary data necessary for the intended analytical report, and assisted with preparation of the TOR for the survey contract – all in preparation for the survey to be executed and the secondary data to be provided. The survey was not successfully completed as survey results were significantly delayed, ISSER refused to cooperate with Integra, resulting in less information than the TOR stipulated, and insufficient time remained to further pursue obtaining additional secondary market data. dTS concludes that Integra completed as many of its objectives as could have been done under the constraints, and that the report prepared by Integra will be of significant value to GIFEC and MOC in increasing the overall focus and efficiency of the USAF in Ghana. TANZANIA Although Integra and UCSAF had agreed and signed an assistance TOR, and Integra prepared a framework for developing a strategic plan and market assessment for UCSAF, full implementation of the assistance outlined in the TOR was contingent upon UCSAF bringing the necessary staff on board. The TOR was signed in January 2012, but UCSAF did not hire additional staff until May 2013. In May 2012, Integra suspended the TOR commitment, but maintained ongoing contact with Mr. Ulanga at UCSAF to determine when and if staff would be hired. Ultimately, Integra elected to redirect the funds that would have been spent on Tanzania to other areas. dTS found the preliminary support provided by Integra to Tanzania to be well done. That Integra was unable to go forward and meet its objectives in Tanzania was beyond its control. dTS concludes that suspending the Tanzania effort, based on TOR contingency requirements, was the correct course of action. OTHER CAPACITY BUILDING ACTIVITIES dTS concludes that the topics and materials presented by Integra at the regional workshops, were well organized and relevant to the theme of the workshops. Not having been present at any of the workshops, dTS can only gauge effectiveness by the quality of the handouts, reports from some officials in the Core Countries, and responses to the survey sent to non-Core Countries. Based on those inputs, dTS believes that both the workshops and the detailed Tool Kit modules prepared and distributed by Integra were of high quality. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 33 An item dTS believes is a USAID shortcoming is not having required a detailed list of attendees (including full coordinates) at the three regional workshops. dTS was advised by Integra that it did not have such lists, and Intel told Integra that it is against company policy to release such names/lists. COST-EFFECTIVENESS Integra expended the most funds in assisting the Kenyan government on ICT-related projects, and was effective in moving Kenya from having a non-functional USAF on the books to creating an NBS and making tangible progress toward creating a viable entity to operate a USAF. In addition, the Kenyan government has planned to commit USD $11.9M for seed capital to be utilized for initial ICT-related projects recommended by Integra.25 Kenya plans to use another USD $9.1M to operationalize USF collections. Importantly, Kenya has committed $80,964,706 as a national budget line item for ICT infrastructure development.26 The Nigerian project utilized the second largest amount of funds. Through Integra’s work, Nigeria was able to move from a functioning USAF to a much more efficient USAF. Further, Nigeria has contracted over USD $21M to ICT-related projects, and has tendered an additional USD $12.5M to universal service projects. Importantly, Nigeria adopted all of the guidelines and recommendations provided by Integra. Ghanaian assistance was in the middle of the range, and was of average effectiveness. The country already had a functioning USAF. Integra’s assistance helped Ghana move toward improving their USAF implementation. In addition, GIFEC representatives received valuable training through their work with Integra. The least funds were spent by Integra in Tanzania. Likewise, Tanzania did not show much change based on the intervention. However, staff did benefit significantly from training provided by Integra. Based on this assessment, it is the view of dTS that all four interventions were, in fact, cost-effective, and that the Kenyan and Nigerian interventions were highly so. The following table shows the cost￾effectiveness results found by dTS. Table 4: Cost Effectiveness Assessment by Country Country Cost of Intervention (USD) Capacity Building Provided Due to Intervention? National Strategy Put in Place or Strengthened Due to Intervention? Additional Funds Committed by National Government to ICTs or Broadband Deployment? Cost-Effectiveness Assessment Kenya $256,061 Yes Yes Yes: $21M planned. Highly Cost-Effective Nigeria $189,720 Yes Yes Yes: over $21M contracted, and an additional $12M tendered. Highly Cost-Effective 25 Letter from the Communications Commission of Kenya to Integra. 26 2013/14 Budget Policy Statement, Treasury of Kenya Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 34 Ghana $ 88,484 Yes Yes No Cost-Effective Tanzania $ 36,294 Yes Yes No Cost-Effective Source: Integra report and dTS analysis. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 35 VII. RECOMMENDATIONS The following dTS recommendations were developed based on comments and suggestions from Core Country officials, USAID and Integra interviews, and from the evaluator’s observations and on-the￾ground experience in SSA countries. They are offered with the hope that they will provide USAID with additional input to consider when designing and implementing ICT and/or universal service/universal access projects going forward. 1. DTS recommends that USAID provide funding to continue technical assistance in Kenya and Nigeria. Such assistance must be initiated at the earliest possible time to avoid a loss of continuity in USAF developments in these two countries. Kenya. In the case of Kenya, it is further recommended that the actual start of technical assistance be contingent upon appointment of the new USAC. The recommended project length for renewed assistance in Kenya is a minimum of two years, although three years would be optimal. dTS believes that such time will be needed for the USAF to reach a level of mature functioning where it should be possible to measure the impact of its activities on direct beneficiaries. dTS recommends that measurement criteria for the additional assistance to Kenya should include, but not necessarily be limited to, the following:  Issuance of invoices to ICT vendors;  Collection of fees from ICT vendors;  Design and implementation of a number of pilot projects;  Establishment and operation of a disciplined M&E process at CCK to effectively measure project success or failure;  Measurement by CCK of the number of additional people gaining access to both basic service and broadband (separate measures); and  Annual measurement by CCK or the Ministry of Information, Communications and Technology of the amount of Kenya’s national budget spent on ICT, as committed by the NBS. Nigeria. Further technical assistance is recommended for Nigeria to reinforce the M&E process. dTS recommends that USAID support further assistance to determine how effectively USPF is applying the cluster analytical concepts for evaluating potential telecenter locations and evaluation of how well USPF has used the M&E guidelines that were adopted. Once any gaps in performance have been identified, it is recommended that Integra work with USPF to determine required corrective actions. An objective should be for USPF and the Ministry to take the lead in determining corrective steps, rather than having Integra simply provide detailed corrective actions. dTS also recommends that USAID/W, in conjunction with the Mission, seek to identify potential synergies and linkages between the NEXT and Markets II projects and USAF programs. Once linkages between the three efforts have been identified, the recommended project length for the USAF component would need to be established, taking into account the deliverables and timelines of the two other projects. It is further recommended that USAID consider supporting Nigeria’s Broadband Transmission Infrastructure Project (BTRAIN), which will require a potential project length of at least two years. Such support would need to go beyond assistance in developing the tender and awarding the bid, to Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 36 include assistance in developing and implementing effective project management of the actual infrastructure rollout. dTS recommends that measurement criteria for Nigeria support include, but not be limited to, the following:  For M&E assistance – completion of gap analysis and USPF-developed corrective actions adopted and implemented;  For additional telecenter or other USAF projects implemented by USPF, an objective measurement of the number of additional people who gained access to either basic service or broadband;  For BTRAIN, measurements need to be developed following the BTRAIN project timeline, i.e., USAID’s project deliverables would be geared to support USPF creating and meeting the necessary planning and operational implementation of the infrastructure; and  For establishing the synergies and linkages between NEXTT, Markets II and potential follow-on USAF projects, measurements will need to be developed once such relationships and specific deliverables have been identified. 2. In a similar vein to the recommended additional technical assistance to Kenya and Nigeria, dTS recommends that a specific follow-up phase be a component of any direct country technical assistance. The TOR and deliverables for such assistance would include this component, with a return visit to the country planned and budgeted as part of the assistance. Funding for follow-up activities for in-country technical assistance projects must be dedicated at project launch. The reinforcement component would include the USAID implementer assessing the status of the particular function in the country when the “revisit” occurs, and then specific follow-up support to either correct deficiencies or further support successes. The specific timing for the follow-up would vary by project and/or country. 27 3. Projects to be designed and funded by USAID/W need to consider, up front, the situation and current priorities in any country USAID Mission where the project might be targeted. It may not be advisable to adopt an absolute operating rule not to implement without local Mission buy-in, but such consideration would certainly provide a good guidepost as a project design criterion. This recommendation implies that there would be actual consultation between local Missions and USAID/W prior to final project design and/or launch, which, by definition, would add some amount of time and expense to project development. However, if USAID/W is expected to provide service and support to the Missions (contrasted to having a “Headquarters versus Field” organizational arrangement), this recommendation would fit well into that management alignment. In the case of ICT projects like the USAF Project, it is understandable that a given Mission might not have specific sector expertise in-house. However, there is nothing mysterious about ICT or its applications, and other recommendations in this report will address that situation. 4. ICT can be the viewed as the “glue” or linkage that not only ties together different initiatives, but also helps identify potential synergies among seemingly disparate programs. In almost any developing country, there are initiatives to improve some aspect of socio-economic or political functioning. dTS recommends that USAID consider developing a program which would not only take into account the 27 See suggestions made by key country officials and Integra experts in this regard in Recommendation 10. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 37 general social benefits of expanded ICT and broadband access, but also one that specifically demonstrates how ICT fits into or supports different programs and projects planned or being implemented by USAID. dTS suggests that developing such a program would highlight the potential synergies that should emanate from looking across individual programs, and that it would also further equip USAID to encourage Ministries in developing countries to collaborate on cross-sectorial programs. A second requirement of such a program would be for USAID/W to implement a program to actively share with country Missions the benefits to be realized by including an ICT analysis in most, if not all, programs in the field. To recognize such benefits, it would be necessary to educate Mission personnel on potential ICT applications across a broad spectrum of USAID programs. dTS recommends that, to be effective, this program would need to be advertised across the Agency, and would be presented face-to-face at the Missions, rather than via telephone, electronic or paper-only communications. Such an effort, combined with planned follow-up training, perhaps bi-annually, would also offset the potential deficit of a Mission not having ICT expertise at any given time. 5. dTS recommends that USAID think realistically when designing and implementing a “regional” program. To realize a significant regional impact in the ICT arena, deliverables expected by USAID should not include broad measurements such as: “increasing the penetration of broadband access in the region.” Unless the project includes a large budget and long implementation period, dTS recommends that ICT projects aimed at the regional level be targeted toward joint activities with existing SSA organizations. This recommendation is not meant to preclude USAID assistance in strengthening an existing SSA ICT or other organization. 6. Where a country is targeted for assistance, dTS recommends that consideration be given to developing an in-depth picture of the needs of the country in the ICT sector prior to launching. It is understood that Missions already do this type of analysis in other areas, but it is not clear that it takes place for ICT projects. Considering the human resources that the US maintains in countries, it is recommended that USAID establish a project planning approach where the ICT political environment and identifiable power structure networks and players in a country are identified and analyzed up front. In addition to the Missions, US Department of State and US Commerce Department individuals located at the embassies can provide significant local intelligence on how things are done in any given country – and by whom. From an even broader perspective, dTS recommends that, in the very early stages of project consideration, USAID/W, possibly accompanied by the local Mission, arrange a meeting with the Permanent Secretary of the Ministry, possibly followed by a meeting with the Minister28 to ask where the country believes it could use USAID assistance (see comments in Recommendation 10 by the former PS of Kenya in this regard). While such meetings may not always result in a list of assistance areas that USAID is ready to take on, the fact that the Agency is trying to understand how the country views ICT problems can develop a relationship where “selling” the power structure on a project is not always necessary. 7. dTS recommends that USAID seek to identify all donor organizations providing support in the ICT sector in a given country or region, and, once identified, that USAID take the lead in creating a “cooperative” donor framework for assistance programs. It is recognized that it may not be possible 28 The minister-level position is not so named in every country. For example, in Kenya, Cabinet Secretary is the current title for what was, until recently, the Minister. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 38 to always coordinate every donor-supported project in a regional area, but dTS believes that the identification aspect alone would be worthwhile. To use the universal service/universal access needs as an example, any ICT infrastructure project planned or being implemented in a country would likely have an obvious linkage to USAF considerations. If, for example, a portion of a fiber backbone is being planned or rolled out in a part of a country, the potential impact on universal service and access is evident. Another benefit to be derived from donor coordination would be the avoidance of potential program duplication. Also, as evidenced in the USAF Project, the combination of USAID-funded support (Integra) at the CTO regional workshop in Kenya meshed well with both donor organizations’ objectives, i.e., CTO gained the additional workshop depth provided by Mr. Townsend’s participation (without having to pay for his services), and USAID was able to begin socializing USAF Project potentials with USAF country representatives. dTS recommends that, in the case of SSA, such an effort be approached either under the auspices or in coordination with an African regional organization, such as the African Union Commission. 8. dTS recommends continued co-sponsoring or participation in Intel or other regional USAF-related workshops as an effective way to increase SSA country officials’ awareness of the requirements for effective USAF operation. Since USAID funds are used to support these workshops, dTS believes that a detailed list of all SSA country attendees should be obtained from each workshop. Such lists would be valuable to USAID in implementing future USAF or other ICT assistance efforts and for evaluations in SSA countries. dTS recommends that obtaining such listings be made a requirement for inclusion in workshops or other co-sponsored events. 9. It is recommended that USAID consider including the creation of a USAF TOR template as part of any future USAF project. Such a template would contain specific provisions specifying the respective roles/responsibilities of the USAID implementer and the recipient country or agency within a country. By definition, the template would contain provisions that would not apply in every country or project. At a minimum, the template would provide guideposts for specific country TOR development, and could also be of value as a Mission resource. 10.The final recommendation in this evaluation is for USAID to review and take into serious consideration the comments and suggestions from Core Country key officials, USAID Mission and Integra personnel that are contained in the interview write-ups in Annex 9, parts of which are shown below: Q. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/ PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? NIGERIA – MINISTER JOHNSON IT WOULD BE HELPFUL TO A COUNTRY IF USAID COULD TAKE A LONGER TERM VIEW IN DESIGNING PROJECTS. SPECIFICALLY, IN THE CASE OF THE PROJECT WORK DONE BY INTEGRA IN NIGERIA, IT WOULD BE A GOOD IDEA FOR THEM (INTEGRA) TO COME BACK IN A YEAR OR TWO – DO AN ASSESSMENT OF PROGRESS, THEN ASSIST IN DETERMINING GAPS AND WHERE TO GO AT THAT TIME. BUILDING EXPERTISE IN THESE TYPES OF FUNCTIONS TAKES TIME, AND A Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 39 COUNTRY COULD USE ASSISTANCE FROM EXPERTS WHO ARE TRULY FAMILIAR WITH THE PROBLEMS AND CHALLENGES. NIGERIA – MR. MAIKANO - USPF HEAD CITING THE BTRAIN PROGRAM, HE SAID THAT HE NOW REALIZES THAT THIS PROJECT WILL TAKE AT LEAST 18 MONTHS TO ACCOMPLISH, AND THAT 60 TO 70% OF THE EFFORT WILL BE CIVIL WORK. IF INTEGRA IS TO BE INVOLVED IN A PROJECT LIKE BTRAIN, WHERE INFRASTRUCTURE ROLLOUT IS INVOLVED, USAID WOULD NEED TO ENSURE A COMMITMENT OF A MINIMUM OF 18 MONTHS, CONSIDERING DEPLOYMENT ISSUES SUCH AS RIGHT-OF-WAY ACQUISITION. NIGERIA – SHARON PAULING - USAID GET THE MISSIONS INVOLVED UP FRONT – BEFORE PROJECT LAUNCH. ASK THE RELEVANT QUESTIONS: SHOULD NIGERIA BE INVOLVED? WILL IT FIT WITH CURRENT OR PLANNED PROGRAMS (IN THE COUNTRY)? ALSO, WHAT LESSONS HAVE BEEN LEARNED FROM OTHERS? HAVE ANY REDUNDACIES BEEN TAKEN INTO ACCOUNT? HAVE ANY LEVERAGE AREAS BEEN IDENTIFIED (OR – HAS THE QUESTION EVEN BEEN ASKED)? NIGERIA – ROLAND OROH – USAID HE SAID THAT USAID/W NEEDS TO CONSIDER HOW TO INTEGRATE ANY PLANNED PROJECTS WITH MISSION ACTIVITIES OR PRIORITIES BEFORE LAUNCHING. THERE IS A NEED TO GET THE MISSION PROGRAM OFFICES INVOLVED IN ADVANCE. IF A COUNTRY IS TARGETED FOR A PROGRAM, THE MISSION NEEDS ADVANCE INVOLVEMENT AND COORDINATION. GHANA – MR. YAHAYA DIRECTOR MOC MR. YAHAYA SAID THAT THERE IS A NEED FOR MORE COLLABORATION BETWEEN MOC AND USAID, AND THAT HE WOULD LIKE TO SEE MORE USAID CONTACT. KENYA – DR. NDEMO – FORMER PERMANENT SECRETARY DR. NDEMO OFFERED AN EXAMPLE OF HOW A COUNTRY SHOULD BE APPROACHED. HE SAID THAT IBM (AS A REAL EXAMPLE) SENDS SENIOR EXECUTIVES TO THE MINISTRY TO ASK WHAT THEY WANT. THE MINISTER THEN IDENTIFIES WHAT IT BELIEVES IS A PROBLEM WHERE ASSISTANCE COULD BE OF VALUE. IN THAT WAY, THE ENTITY CONSIDERING PROVIDING ASSISTANCE CAN DETERMINE IF IT WANTS TO PROVIDE SUCH ASSISTANCE – AND ALREADY WILL HAVE BUY-IN AT THE MINISTERIAL LEVEL. HE ALSO SAID THAT IT IS VALUABLE IN KENYA TO HAVE A LOCAL POINT PERSON WHO UNDERSTANDS THE INDUSTRY AND THE LOCAL POLITICS, WITH THE POINT PERSON BE INVALUABLE IN GETTING APPOINTMENTS WITH THE NECESSARY PLAYERS. KENYA – MR. MUREITHI – INTEGRA LOCAL CONSULTANT Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 40 HE STATED THAT CONSIDERATION NEEDS TO BE GIVEN TO THE NEED TO MANAGE THE RELATIONSHIPS BETWEEN USAID AND THE RECIPIENT ORGANIZATIONS AT ALL LEVELS. THE ROTATION OF PEOPLE IN VARIOUS POSITIONS CAN CAUSE PROBLEMS UNLESS THE RELATIONSHIPS ARE “REFRESHED” ON AN ONGOING BASIS. IN SOME CASES, THE TOP LEVELS INVOLVED SIGN OFF ON AN ITEM, BUT LOWER LEVEL RELATIONSHIPS DO NOT ALWAYS GET ESTABLISHED. KENYA – MS. MOCHACHE – ASSISTANT DIRECTOR CCK SHE THOUGHT THAT USAID NEEDS TO GAIN A BETTER UNDERSTANDING THAT THINGS MOVE SLOWLY IN AFRICA, I.E., TO BE MORE AWARE OF THE CULTURAL DIFFERENCES IN AFRICA. SHE FELT THAT INTEGRA’S PROJECT WAS A SUCCESS, BUT EXPRESSED UNCERTAINTY ABOUT WHETHER OR NOT IT WOULD BE EXTENDED BY USAID. THE RESULT OF THE UNCERTAINTY HAS BEEN “A LOT OF UNNECESSARY ANXIETY.” KENYA – MR. WANGUSI – DIRECTOR GENERAL CCK HE STATED THAT THERE IS A NEED TO BUILD IN ADEQUATE TIME TO ENSURE SUSTAINABILITY OF THE CAPACITY THAT HAS BEEN BUILT. KENYA – MR. FARROE & MR. WAUDO – USAID (INTERVIEWED TOGETHER) MR. FARROE STATED THAT, FROM THE COUNTRY’S PERSPECTIVE, USAID/W NEEDS TO REACH OUT EARLY AND UNDERSTAND WHAT THE LOCAL FOCUS IS. IT IS NOT COMPLETELY NECESSARY TO FOLLOW THE MISSION’S PRIORITIES, BUT EARLY-ON COORDINATION AND LINKS ARE CRITICAL – EVEN WHERE THE MISSION IS NOT ENGAGED DUE TO ITS CURRENT FOCUS. MR. WAUDO SAID THAT THERE IS A NEED FOR MORE SHARING OF DOCUMENTS AND REPORTS, AS SOMETIMES THE MISSION DOES NOT RECEIVE STATUS REPORTS. TANZANIA – MR. ULANGA – CEO UCAF HE SAID THAT FOR ANY INFRASTRUCTURE PROJECT, AT LEAST FIVE YEARS IS NEEDED FOR SUPPORT. HE ALSO INDICATED THAT HE APPRECIATES THAT EACH COUNTRY IS DIFFERENT, SO THAT COUNTRIES CAN LEARN FROM EACH OTHER. HE THOUGHT MORE RETIONAL OR MIXED TYPE WORKSHOPS WOULD BE BENEFICIAL AND SAID THAT THE WORKSHOPS SHOULD NOT BE LIMITED TO AFRICAN COUNTRIES. HE ALSO SAID THAT HE BELIEVES UNIVERSAL SERVICE FUNDS ARE LOOKING AT EXTENDING INFRASTRUCTURE, BUT THAT HE THINKS THEY NEED TO MOVE TOWARD APPLICATION AND TRAINING ROLES, SUCH AS HOW TO USE ICT TO IMPROVE SOCIAL BENEFITS. Q. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? NIGERIA – MINISTER JOHNSON Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 41 YES, SYNERGIES WITH VARIOUS PROGRAMS OUGHT TO BE TAKEN INTO ACCOUNT, EVEN IF IT IS JUST BY PROVIDING EXAMPLES OF SYNERGIES DISCOVERED IN OTHER COUNTRIES. GHANA – MS. YEE – USAID SHE STATED THAT NEW USAID/W PROGRAMS SHOULD BE CONSIDERED IN TERMS OF THE COUNTRY DEVELOPMENT COOPERATION STRATEGY (CDCS), WHICH INCLUDES ALL MAJOR PLANS FOR THE COUNTRY. THE IDEA WOULD BE TO INSURE THAT USAID/W PROJECTS CAN BE INTEGRATED OR WOULD SUPPORT MISSION PROGRAMS OR PROJECTS. SHE INDICATED THAT THE CURRENT MISSION DIRECTOR WOULD BE INTERESTED IN LEARNING MORE ABOUT HOW ICT COULD BE A COMPONENT OF MOST PROGRAMS. Q. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? NIGERIA – MINISTER JOHNSON SHE STATED THAT SHE BELIEVES THAT USAID’S FOCUS ON BROADBAND IS A VERY GOOD IDEA. ALSO, FOR NIGERIA, THE QUALITY OF THE PEOPLE ON THE INTEGRA TEAM WAS VERY HELPFUL, I.E., HAVING PEOPLE WHO HAVE SIGNIFICANT EXPERIENCE IN THE FIELD IS IMPORTANT. SHE SAID THAT SHE WOULD VERY MUCH LIKE TO SEE INTEGRA COME BACK TO FURTHER ASSIST NIGERIA. Q. IS THERE ANYTHING THAT YOU WOULD WANT TO RECOMMEND THAT USAID CONSIDER DOING DIFFERENTLY IN DESIGNING AND IMPLEMENTING PROGRAMS SUCH AS THE TASK ORDER 1 EFFORT? INTEGRA – MR. TOWNSEND – PROJECT MANAGER USAF FIRST – GIVEN THE KENYA EXPERIENCE, TRY TO AVOID MOU ARRANGEMENTS THAT COULD LEAD TO SIGNIFICANT DELAYS. SECOND – THERE IS A NEED FOR USAID TO BE BOTH FLEXIBLE AND TO HAVE A “LONG VIEW.” THIRD – PROJECTS NEED TO INCLUDE STEPS RECOGNIZING HOW TO KEEP THINGS GOING IN COUNTRIES RECEIVING ASSISTANCE. INTEGRA – MR. IFTIKHAR – CONSULTANT AS IN NIGERIA, HE SAID THAT FOLLOW-UP ASSISTANCE NEEDS TO BE PROVIDED TO KENYA. AREAS SUCH AS HOW TO EVALUATE BIDS FOR PILOT PROJECTS, CONTRACT NEGOTIATIONS, ETC., NEED TO BE FURTHER SUPPORTED. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 42 VIII. LESSONS LEARNED There are lessons that can be taken from the USAF Project experience in Sub-Saharan Africa. 1. In countries where a USAF is not fully operational, the amount of time between the delivery of technical assistance/capacity building and actual impact on the number of additional people served will be much greater than where technical assistance is provided to a country with an already functioning USAF. In the latter case, assistance is provided to support improvements. While this may appear obvious from an analytical perspective, it has a bearing on the management of a project and the potential amount of project time to be considered when launching such assistance projects. In the case of the USAF Project, the support provided to Kenya (ignoring the significant time spent on NBS development) needed to extend beyond the time constraints of Task Order 1. In Nigeria, on the other hand, the assistance provided was translated into relatively immediate action, in that the USPF organization modified its tendering processes and instituted M&E steps that are currently being applied in practice. 2. In a similar vein, where it is necessary to obtain a “government-to-government” MOU, as was the case in Kenya, USAID needs to maintain flexibility in modifying project objectives, timelines and, even, budgets. To the extent that project design has been able to incorporate the steps described in dTS Recommendation 6, the necessity for such an MOU would have been identified at the beginning of the project, and any potential delay for the MOU negotiation process could be built into the project timeline. 3. Although it may not be possible to make precise estimates, project planning needs to take into account that even well planned and prepared projects will likely require lengthy time frames for completion of the technical assistance and capacity-building activities. This is a planning item where inclusion of up-front analysis of a given country’s styles and processes for decision-making could contribute to improved project plans. 4. Capacity-building outcomes and results generally do not take place immediately after the technical assistance is provided. Given an anticipated lag between assistance delivery and results of such assistance, measurement of the results and outcomes ideally would take place somewhere around 2 to 4 years subsequent to project completion. The impact of influences other than the assistance being measured presents evaluation challenges, regardless of when the evaluation takes place. 5. There is a need for additional technical assistance in each of the four Core Countries to aid them in either continuing to establish a functioning USAF or to maintain improvements resulting from the initial USAF project. There is also a serious need for assistance in the USAF area in many other SSA countries, as both ICT penetration and the level of access to broadband in SSA remains the lowest of any region in the world. 6. Including all stakeholders in an undertaking, such as the NBS development in Kenya provides an effective model for other countries in creating and adopting their national strategies. The inclusion of ICT vendors in any national strategy negotiation can result in gaining their support for such strategy, or, at a minimum, can lessen their opposition. Encouraging countries to also include media advertising of the draft strategy and a true public comment process can bring the strategy to the public’s awareness. It follows, of course, that the government must meet its stated objectives and/or regularly explain results to the public. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 43 7. Short-term institutional capacity building will not likely result in long-term retention of all of the guidance provided. Without some predictable form of follow-up to assist in correcting any faulty implementation efforts and further encouraging initiative where positive steps have been made, capacity may atrophy. In addition to further direct technical assistance, follow-up could take any number of forms, such as: an effective regional organization for USAF representatives (“regional” would not necessarily mean the entire SSA region); regional or sub-regional workshops, or; scheduled Skype or teleconferences, either with a group of countries or a single country Agency. 8. The existence of a national fiber backbone and achievement of real progress in expanding USAF applications on a broad scale are intimately linked. Any future USAID USAF project must identify the current and planned status of a national broadband backbone prior to completing a country’s project design. Without a fiber backbone within a reasonable distance to rural population hubs, such areas must rely on microwave or satellite for broadband access, which presents both capacity limitations and prohibitive expense. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 44 ANNEXES Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 45 ANNEX 1. STATEMENT OF WORK/TERMS OF REFERENCE Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 46 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 47 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 48 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 49 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 50 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 51 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 52 ANNEX 2. COR DESIGNATION LETTERS FOR THE USAF EVALUATION Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 53 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 54 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 55 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 56 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 57 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 58 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 59 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 60 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 61 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 62 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 63 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 64 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 65 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 66 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 67 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 68 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 69 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 70 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 71 ANNEX 3. GBI USAF TOR UNITED STATES OF AMERICA AGENCY FOR INTERNATIONAL DEVELOPMENT 1. Country of Performance: US Adv, & Asst. Services Yes [ ] No [X] 2. Contract (Incorporating FAR and AIDAR Clauses): Contract No: AID-CIO-I-10-00001 Order No: AID-CIO-TO-11-00001 NEGOTIATED PURSUANT TO THE FOREIGN ASSISTANCE ACT OF 1961, AS AMENDED AND EXECUTIVE ORDER 11223 3. CONTRACTOR (Name and Address): INTEGRA 4350 N Fairfax Dr. Ste. 900 Arlington, VA 22203-1624 CAGE: 5XJM7 DUNS: 961739898 4a ISSUING OFFICE: U.S. Agency for International Development M/CIO/CMS 2PY North 1300 Pennsylvania Avenue, NW Washington, D.C. 20523 4b ADMINISTRATION OFFICE: -Same as above￾5. TECHNICAL OFFICE: U . S . Agency for Intl Development Joe Duncan, EGAT/I&E/ICT 1300 Pennsylvania Ave. NW FRB Rm.3.08-064 Washington, DC 20523 Phone: 202-712-0474 6. PAYING OFFICE. SUBMIT INVOICE TO: ei@usaid.clov 7. EFFECTIVE DATE: 04/12/2011 8. ESTIMATED COMPLETION DATE: 09/30/2012 9. ACCOUNTING AND APPROPRIATION DATA: BBFY: 2010; FUND: DV; OP UNIT: AFR/SD; SOC: 4100301; OBLIGATION TOTAL: $ 500,000.00 10. The United States of America, represented by the Contracting Officer signing this Order, and the Contractor agree that: (a) this Order is issued pursuant to the Contract specified in Block 2 above and (b) the entire Contract between the parties hereto consists of this Order and the Contract specified in Block 2 above. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 72 11a. NAME OF CONTRACTOR: INTEGRA BY: 11b. UNITED STATES OF AMERICA Agency for International Development BY: NAME: Robert Otto TITLE: President DATE: NAME: • Michael A. Clark TITLE: Contracting Officer DATE: AID 1420-61 (Rev'd) Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 73 USAID EGAT I&E/ICT Telecommunications Universal Service Funds or Universal Access Funds (USFs/UAFs) Indefinite Quantity Contract Task Order Award AID-CIO-I-10-00001 AID-CIO-TO-11-00001 Date: April 12, 2011 USAID FROM THE AMERICAN PEOPLE United States Agency for International Development Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 74 PART I 1.0 GOVERNMENT POINT-OF-CONTACT INFORMATION ........................................................................... 3 PART II - STATEMENT OF WORK (SOW) .............................................................................5 1.0 BACKGROUND .................................................................................................................................................... 5 2.0 OBJECTIVE .......................................................................................................................................................... 6 3.0 SCOPE .................................................................................................................................................................... 7 4.0 CONSTRAINTS AND ASSUMPTIONS ............................................................................................................. 7 5.0 TASKS .................................................................................................................................................................... 8 5.1 Task 1— Management and Reporting Requirements ................................................................................8 5.2 Task 2 — Conduct Phase 1: Undertake a series of Capacity Building Workshops across Sub-Sahara Africa that Includes Training and Discussion on Best Known Practices in Place within Sub-Sahara Africa along with Proposed Global Best Known and Innovative Practices For USFs/UAFs .......................................9 5.3 Task 3 — Conduct Phase 2: Provide Technical Assistance (TA) to a select set of 3-4 USFs/UAFs thought to be open to adoption of Best Known Practices along with proposed Innovative Practices ..............9 6.0 PERIOD OF PERFORMANCE .........................................................................................................................10 7.0 PLACE OF PERFORMANCE ...........................................................................................................................10 8.0 DELIVERABLES ................................................................................................................................................10 9.0 SPECIAL REQUIREMENTS ............................................................................................................................12 9.1 Inspection and Acceptance Criteria .......................................................................................................... 12 9.2 Quality Assurance .......................................................................................................................................12 9.3 Terms and Conditions ................................................................................................................................ 12 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 75 PART I GOVERNMENT POINT-OF-CONTACT INFORMATION 1.0 GOVERNMENT POINT-OF-CONTACT INFORMATION Contracting Officer (CO) Michael Clark USAID M/CIO/CMS 2PY North 1300 Pennsylvania Ave. NW Washington, DC 20523 Telephone Number: 703-666-5613 Facsimile Number: 703-666-5640 Alternate Contracting Officer (CO): Yvonne Wilson USAID M/CIO/CMS 2PY North 1300 Pennsylvania Ave. NW Washington, DC 20523 Telephone Number; 703-666-5653 Facsimile Number: 703-666-5640 USAID Contracting Officer's Technical Representative: Joe Duncan USAID EGAT I&E/ICT, Rm. 3.08-064 1300 Pennsylvania Ave. NW Washington, DC 20523 Telephone Number: (202) 712-0474 USAID Alternate Contracting Officer's Technical Representative: Micah Globerson USAID EGAT I&E/ICT, Rm. 3.08-133 1300 Pennsylvania Ave. NW Washington, DC 20523 Telephone Number: (202) 712-1407 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 76 PART I 2.0 TABLE OF RATES Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 77 PART II - STATEMENT OF WORK (SOW) 1.0 BACKGROUND Telecommunications Universal Service Funds or Universal Access Funds (USFs/UAFs)1 are being used by an increasing number of countries globally as a viable approach for extending access to telecommunications services to those living in rural areas where the licensed service providers often find providing services commercially unviable. These funds are typically formed through a small percent being collected from the revenue of licensed service providers—most typically as defined in a country's telecommunications law or supporting/implementing regulations. Many countries have telecommunications laws calling for the establishment of USFs/UAFs. A subset of these countries has actually set up such funds, though a smaller subset of countries are actually operational in collecting and dispensing funds. And even where fully operational, there are often ineffectiveness and/or inefficiencies in their operations. A number of countries with funds set up are not fully operation. And yet other countries have yet to authorize the establishment of a USF/UAF. USFs/UAFs in Africa generally face the same challenges of funds elsewhere with regards to designing and operating the fund such as to achieve maximum benefit. Several current challenges include; 1) how to define which licensed service providers must make contributions to the fund and are thus eligible for receiving funds to support their rural build out, 2) how to best take into account the virtual explosion of mobile network build out that through a liberalized telecom environment is reaching further into rural areas, 3) how to address extending broadband into rural areas where in addition to extending the reach, training and content are essential value-added elements, 4) how to leverage the fund to increase socioeconomic gain possible through the wise use of these funds, 5) how to move from a simple on-going subsidy model to a more sophisticated investment model, where the collected funds are leveraged beyond simply subsidizing the potential shortfall of costs versus revenue, to where they are managed such as to become an investment fund for infrastructure build out, and possibly other innovative and complementary uses. Africa's USFs/UAFs have specific opportunities and challenges unique to sub-Saharan Africa. First, such funds in Africa are generally being established subsequent to the establishment of funds in other regions and therefore can take into account lessons learned and effective best practices derived from successful preexisting funds.- Second, the country-level funds already in place and planned within Africa have already organized an association (African Universal Service and Access Fund Association (AUSAFA) to provide support and help in their collaboration. This Association was established in December 2008 through the Commonwealth Telecommunication Organization (TCO). Several funds in Africa, including those in Senegal, Uganda, Mauritius, South Africa and Ghana) are already serving as examples and offering insights to other African funds more recently formed or just now becoming operational. AUSAFA provides an opportunity for leverage of USAID's funding provided through this Some funds are called universal service funds and some universal access funds so the universal service and access fund term is used to refer to all such funds. 2 Attachment A provides selected references that Africa's ASAF's can tap, including a thorough analysis of models and practices of USAF's in Latin America, perhaps the continent with the most experience with such funds. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 78 PART II project. Third, funds in Africa need to adapt to the specific telecommunications, business and geographic environment that exist within. Only in recent years has there been broad-based market liberalization, specifically in the mobile arena, and with this, an explosive build out in many countries. Also, at this time, and over the next several years, for the first time in history, Africa will be served with wide-scale access to international undersea fiber. In the fall of 2009, USAID did, through its West Africa Trade Hub project, and in partnership with Intel, participate in a CTO-AUSAFA event, including AUSAFA's first annual meeting in Livingston, Zambia. At this event David Townsend, the principal author of the Regulatel Report for Latin America, as well as the lead in providing technical assistance for shaping Vietnam's USF, along with Darrell Owen, who was the lead for the Vietnam Last Mile Initiative (LMI) project under which the USF work was undertaken, participated in providing key presentations to the AUSAFA members. David Townsend's engagement prior to, and subsequent to the Zambia event in supporting several African USFs, established a base for USAID's future engagement. Preliminary to the technical assistance approach provided in this task order was a recent USAID￾sponsored activity, "Increasing the Use and Impact of Telecommunications Universal Service Funds in Sub Saharan Africa" (hereinafter the "SSA Assessment"). Conducted as part of the Global Broadband and Innovations (GBI) Program, that activity was an assessment intended to serve as the basis for this follow-on task order. It mapped and provided strategies for reforming Sub Saharan Africa's USF environment, including best practices and learning sharing, the identification of prime candidates for targeted technical assistance, recommendations on strategies to facilitate improvement to USF management and financial practices, and cooperation and funds-leveraging with the African Universal Service and Access Fund Association (AUSAFA) and the Commonwealth Telecommunications Organization (CTO) to coordinate on common USF reform objectives. This Task Order provides USAID's Africa Bureau with a unique opportunity to quickly add value in this critical area. Attachment provided in this Task Order: Attachment A — Selected References Attachment B — Performance Requirements Summary 2.0 OBJECTIVE The objective of this task order is to improve the practices and increase the actual use of current and future telecommunications universal service and access funds in sub-Saharan Africa to accelerate the extension of affordable and sustainable access to telecommunications services (including broadband Internet services where permissible) to those not yet served, tapping innovative technical and business approaches and adapting as appropriate best practices and lessons learned from Africa and elsewhere. The work under this Task Order should improve the USF/UAF awareness, capacity, and practices across sub-Sahara Africa, with regards to taking into account the current telecom dynamics, and creating best practice models for adoption. The work will leverage to the extent feasible, CTO's engagement as well as that of the recently created AUSAFA with regards to capturing an expanded base of current practices, and promulgating the best-of-breed practices taking place within Africa, but also adding to these, Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 79 PART II newer more advanced models for consideration. The objective will also focus technical assistance on working with a small subset of countries with USFs already in place and positioned such as to move quickly to become the next generation of USFs on the continent. 3.0 SCOPE The scope of this Task Order is to meet the above-stated objective by using a variety of means, including, but not limited to approaches such as; Use of and reference to practical and web-accessible toolkits (including materials form the ITU, World Bank, and other sources found to be of value, Case studies and best practices derived from within Africa as well as from outside sources,  Peer-to-peer activities at capacity building,  Presentations at seminars and workshops, and  Participating in events/activities with interested universal service funds or related associations. The scope is also be limited by the funds available for this task order. The scope is also limited by the time, with an estimated performance within which all work must be completed as being a period of no longer than 18 months from the time the Task Order is put into place. 4.0 CONSTRAINTS AND ASSUMPTIONS The following constraints and coordination apply to this task order:  This Task Order requires the Contractor/Subcontractor team have demonstrated professional background in working with USFs/UAFs for a period of no less than 10 years in providing technical assistance to multiple USFs/UAFs in helping setting up, shaping, and becoming operational.  This Task Order requires COTR approval of Chief of Party;  Requires coordination with USAID Africa Bureau Office of Sustainable Development, EGEA Division, USAID Bureau of Economic Growth Agriculture and Trade (EGAT), and USAID Missions in Africa;  Requires coordination with, where tasks are complementary or where AUSAFA offers an appropriate forum for convening, teaching, presenting, and reviewing draft deliverables), with the AUSAFA (via its secretariat at CTO, Commonwealth Telecommunications Organization www.cto.int );  Requires compliance with both Section 508 and USAID Graphic Standards (available at www.usaid.gov/branding ) or any successor branding policy, where applicable. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 80 PART II 5.0 TASKS 5.1 Task 1— Management and Reporting Requirements 5.1.1 Subtask 1— Task Order Management The Contractor shall be responsible for the effective management and administration of all efforts undertaken under the Task Order. The Contractor shall be responsible for ensuring that all work activities are performed in a timely and cost effective manner while maintaining the highest quality of performance. 5.1.2 Subtask 2 — Work plan Development with Three Phases The Contractor shall develop draft and final versions of a Work Plan for this effort in two phases. The Work Plan will establish two phases of work as reflected below. The initial Work Plan will reflect the proposed work for each of the two Phases. Further, as the first Phase nears completion, the Work Plan for the body of work reflected in the subsequent Phase will be reviewed for potential adjustment and refinement, and submitted to the COTR for consideration-approval before work begins. The Work Plan for each of the three Phases the Contractor shall provide as a minimum, the following: a. Describe the technical approach, organizational resources and management controls to be employed to meet cost, performance and schedule requirements; b. List the specific tasks to be accomplished; c. Document planning assumptions and decisions; d. Document the project timeline for each task activity; e. Identify the task order deliverables and the deliverable dates for Government review and acceptance; f. Develop a distribution list and deliverable format for all task order deliverables; g. Provide the draft Work Plan within ten (10) business days of receipt of the task order; and h. Provide the final Work Plan within five (5) business day of receipt of USAID feedback. 5.1.3 Subtask 3 — Reporting Requirements The Contractor shall develop and deliver Monthly Task Order Performance Reports (TOPR) that provide valid and timely program status information as of the report date to the designated USAID point(s) of contact not later than the tenth (10th) business day after the reporting period. The TOPR shall include but not be limited to the following: Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 81 PART II a. Work performed during the reporting period including: accomplishments, problems/issues and recommendations for corrections; b. Work to be accomplished during the next reporting period; and c. Any issues to USAID that may occur during project implementation. 5.2 Task 2 — Conduct Phase 1: Undertake a series of Capacity Building Workshops across Sub-Sahara Africa that Includes Training and Discussion on Best Known Practices in Place within Sub-Sahara Africa along with Proposed Global Best Known and Innovative Practices For USFs/UAFs At a minimum, the contractor shall: The first phase (Phase 1) of the Work Plan will include the following body of work:  Translate the materials collected and published in "Increasing the Use and Impact of Telecommunications Universal Service Funds in Sub Saharan Africa," (hereinafter the "SSA Assessment") into 2-3 day seminar/workshop format;  Work closely with the ITU, InfoDev, the World Bank, CTO, AUSAFA, the GSM Association, regional governmental organizations such as ECOWAS and COMESA), etc., as appropriate, to undertake a series of regional, sub-regional, and country-specific seminars and workshops. These will consist of short sessions on the USF/UAF topic being built into events being sponsored by other organizations, as well as seminars/workshops undertaken as a product of this Task Order.  Update/refine and finalize the publishable series of documents coming out of the SSA Assessment that: 1) presents the current state of USFs/UAFs across Sub-Sahara Africa that significantly improves upon currently available information, and 2) presents a Best Known Practices for consideration by USFs/UAFs across Sub-Sahara Africa  Refine the short-list of specific countries with USFs/UAFs that were identified in the SSA Assessment as holding the most potential for becoming candidates for additional/focus technical assistance through Phase 2; and  Review and update/refine the Work Plan for Phase 1 with needed adjustments based on work nearing completion in Phase I. 5.3 Task 3 — Conduct Phase 2: Provide Technical Assistance (TA) to a select set of 3-4 USFs/UAFs thought to be open to adoption of Best Known Practices along with proposed Innovative Practices. The second phase (Phase 2) of the Work Plan will include the following body of work:  Assess short-list of USFs/UAFs found to be suitable for providing focused technical assistance (TA) to move them forward with regards to their advancing the Best Known Practices within Sub-Sahara Africa, and make final decision;  Develop a draft detailed work plan for each USF/UAF engagement; Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 82 PART II  Negotiate these work plans into MOU with national USF/UAF to frame/structure the level and nature of TA to be provided through this Task Order, with targeted results;  Provide TA in accord with each MOU;  Conclude the engagement with a proposed plan of action for moving each of the supported USFs/UAFs forward beyond the level of TA being provided through this Task Order; and  Prepare a final documented report that captures the work performed, issues surfacing, issues addressed, progress made, results, and future direction for moving the USF/UAF forward. Report should also put forward recommendations for possible USAID's future engagement and potential for replication in other countries. 6.0 PERIOD OF PERFORMANCE 04/01/2011 — 09/30/2012 or 18 Months from the date of task order award. 7.0 PLACE OF PERFORMANCE Work under this Task Order shall be performed at the Contractor's/Subcontractor's in the U.S., as well as in Africa. All domestic and international travel that is billable to this Task Order must be approved in advance by the COTR—with the travel request to include destination/s, length of travel, travel-related costs, and objectives. 8.0 DELIVERABLES Table 8-1 summarizes the task order deliverables identified so far and defines the distribution list. At the culmination of Phase 2, additional deliverables will be defined. Table 8-1 Task Order Deliverables Contract Line Item Number Deliverable Task Reference Distribution 0001 A [Reserved] 5.1.1 Per Government-Approved Preliminary Work Plans for Phases 1 & 2 0001B Initial Work Plan for Phase 1 and 2 Draft Final-Approved 5.1.2 COTR 0001C Monthly TOPR 5.1.3 Per Government-Approved Work Plans, Phase I 0002A [Reserved] 5.2.1 Per Government-Approved Work Plans, Phase I Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 83 PART II 0003 Updated/refined and finalized the publishable series of documents coming out of the SSA Assessment that; 1) presents the current state of USFs/UAFs across Sub-Sahara Africa that significantly improves upon currently available information, and 2) presents a Best Known Practices for consideration by USFs/UAFs across Sub-Sahara Africa 5.3 0004 Refined short-list of specific countries coming out of Phase 1 with USFs/UAFs that hold the most potential for becoming candidates for additional/focus technical assistance through Phase 2 5.3 0005 Refreshed Phase 2 Work Plan Draft Final-Approved 5.3 Per Government￾Approved Phase 2 Work Plan 0006 Finalized short-list of specific countries being provided TA during Phase 2 5.4 Per Government￾Approved Phase 2 Work Plan 0007 Detailed Work Plan and MOU for each country being supported with TA during Phase 2 5.4 Per Government￾Approved Phase 2 Work Plan 0008 Plan of action for moving each of the supported USFs/UAFs forward beyond the level of TA being provided through this Task Order 5.4 Per Government￾Approved Phase 2 Work Plan 0009 Final documented report that captures the work performed, issues surfacing, issues addressed, progress made, results, and future direction for moving the USF/UAF forward 5.4 Per Government￾Approved Phase 2 Work Plan [Reserved] 5.6 Per Government￾Approved Work Plan Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 84 PART II 9.0 SPECIAL REQUIREMENTS 9.1 Inspection and Acceptance Criteria Final inspection and acceptance of all work performed, reports and other deliverables will be performed at the place of delivery. Refer to Attachment B — Performance Requirements Summary for performance requirements and standards. 9.2 Quality Assurance The Government will evaluate the contractor's performance under all tasks of the task order, including specified deliverables summarized in paragraph 8. The COTR will record all surveillance observations. When an observation indicates defective performance, the COTR will require the contractor task manager to initial the observation. The initialing acknowledges the contractor's awareness of the observation, but does not constitute concurrence. Any action taken by the Contracting Officer as a result of the surveillance will be in accordance with the terms and conditions of the IQC and the GSA schedule. 9.3 Terms and Conditions a. Designated COTR: Joe Duncan (202) 712-0474 b. Alternate COTR: Micah Globerson: (202) 712-1407 c. The project team should include an advisor or organization with strong experience related to universal service funds. d. Local Labor Specifications: From the Government's viewpoint, there is also a desire to engage regional (African) specialist(s) to engage on the task order in order to reduce project risk and insure the field appropriate/relevant deliverables. e. Government Furnished Equipment/Information: The Contractor will be provided access to associated documentation needed for the performance of this task order. f. Travel: International travel within Africa may be required and authorized by the COTR. Any travel must be pre-approved. Requests for approval must include: estimated trip dates, traveler(s), destination(s), purpose, estimated costs. g. Special Format Requirements: Mutually agreed upon between COTR and Contractor. h. Security Requirements: Security requirements are outlined in the IQC. i. Invoices: Progress payments will be based on completion or achievement of a JOB and/or monthly reporting/implementation requirement. Monthly payments will be paid based on a fixed amount for each hour worked by specified classes of labor. Vouchers and supporting statements or certificates properly identifiable with the IQC and TO shall be submitted as follows: ei@usaid.gov. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 85 PART II ATTACHMENT A — SELECTED REFERENCES RELATED TO TASK ORDER Universal service fund section of ITU/InfoDev Toolkit for ICT: http://www.ictregulationtoolkit.org/en/Section.2097.html General information on the newly formed (2008) African Universal Service and Access Fund Association (AUSAFA): http://cto.int/Default.aspx?tabid=230 Stern, Peter A. and D. Townsend, New Models for Universal Access in Latin America, Regulate/World Bank (PPIAF)/ECLAC Project on Universal Access for Telecommunications in Latin America, August 2006. (excerpts available from jpayne@usaid.gov ) ITU Model Universal Service Fund Policies and Procedures, www. i tuarabic.org/coe/2005/UASP/Fin al %20Documents/2nd%20day/08,09,10%20- %20ITU%20Models.opt , February 2005. Next Generation Networks and Universal Access, The Challenges Ahead, http://www.itu.int/ITU￾D/treWEvents/Seminars/GSR/GSR07/discussion papers/Eric Lie universal service.pdf , Global Symposium for Regulators (GSR) Discussion Paper, ITU, February 2007. Presentation on experience in Senegal: http://cbdd.wsu.edu/edev/universalaccess/Senegal.ppt Model Universal Access Policies: Entrepreneurial Strategies Stimulating Telecommunications Market Development, report to the Commonwealth Telecommunications Organization (CTO) by David N. Townsend & Associates (DNTA), May 2003, http://www.cto.int/Portals/O/docs/research/CTO EntrepStrat.pdf Sepulveda, Edgardo and M. TeIrault, Minimum Subsidy Auctions for Provision of Public Access Telecommunications in Rural Areas, Chapter 5 in Trends in Telecommunication Reform 2003 from the ITU. http://www.cto.int/Portals/O/docs/research/CTO-ITU Trends03 Chapter5.pdf Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 86 PART II ATTACHMENT B — PERFORMANCE REQUIREMENTS SUMMARY Explanation of Columns Outcomes Area within the overall contract where the Government describes what it wants to accomplish as the end result of this contract. Requirement Requirements represent organizational needs. Standard Standard represents the performance baseline against which the contractor will be measured. Acceptable Quality Level The percentage of maximum allowable error or variation from the standard. Performance Incentive Measurement Interval The frequency with which the incentive adjustments will be exercised. Monitoring Source The source, or data collection methodology, the organization will use to measure contractor performance. Incentive/ Disincentive The methodology used to reward good performance or punish poor performance. Calculation The formula used to quantify contractor performance. PERFORMANCE REQUIREMENTS SUMMARY (continued) Outcome Requirement Standard Acceptable Quality Level Performance Incentive Measurement Interval Monitoring Source Incentive/ Disincentive Calculation Maintain a high Deliverables are Deliverables Deviation Measured By USAID 100% Positive Performance = number level of quality high quality and are written may be made Deliverable; Non- inspection of all incentive = None of valid written assurance on all complete. clearly and only when monetary deliverables by complaints during the deliverables. concisely, providing the COTR and the Performance Incentive COTR. Negative incentive = life of the task order. Refer to USAID Contractor Calculated at end contract For example: paragraph 8, Deliverables stakeholders a clear have agreed to deviate of task order. discrepancy report placed in Satisfactory understanding from the COTR has no the official Performance = 1 or less Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 87 PART II of all standard. more than one (1) contract file valid written complaints deliverables. valid written when the during the life of the complaint during contractor has task order. Final the life of the task more than one (1) documentation order; Minimal valid written Unsatisfactory must be in COTR complaint during Performance = 2 or Microsoft intervention the life of the more valid written Suite: Word 2000 and Excel required. task order. complaints during the life of the task 2000 or current order. USAID version Final documentation must be spell checked, grammar checked, and accepted by the Government. PERFORMANCE REQUIREMENTS SUMMARY (continued) Outcome Requirement Standard Acceptable Quality Level Performance Incentive Measurement Interval Monitoring Source Incentive/ Disincentive Calculation Meet the Deliverable 100% of No deviation Measured By USAID 100% Positive Performance = number delivery dates delivery dates are delivery dates from the Deliverable; Non- inspection of all incentive = of deliverables specified by the met or exceeded. specified by the standard monetary deliverables by None. identified in the task COTR and the COTR and the unless a Performance COTR. order (Refer to Work Plan, if applicable. Work Plan, if applicable. are deviation is approved by Incentive Calculated at end Negative incentive = Paragraph 8, Deliverables) delivered met or exceeded. the COTR and a task order modificatio n is issued by the Contracting of task order. contract discrepancy report placed in the official contract file when the contractor's on-time divided by the number of deliverables required by the contract (Refer to Paragraph 8, Deliverables), expressed as a percentage. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 88 PART II Officer. performance falls below the AQL For example: range of 100% for this task Deliverables delivered order. on-time = 6 Deliverables required by the contract = 7 Performance = 6/7 or .857 x 100 = 85.7% of the deliverables met the delivery dates specified in the project plan. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 89 ANNEX 4. GBI WORKPLAN 1030 15th Street NW, Suite 555, Washington, DC – Tel 202 898 4110 – www.integrallc.com TO: Joe Duncan, COTR, GBI Program FROM: Robert Otto, GBI IQC Team Leader DATE: 05 July 2011 SUBJECT: Revised Work Plan for Task Order AID-CIO-TO-11-00001 Telecommunications Universal Service Funds or Universal Access Funds (USFs/UAFs) INTEGRA presents the following revision to the Work Plan for the subject Task Order. This revision is designed to better align the two pieces of the Africa/SD funded Universal Service Fund Support Project. It supports the implementation of a single, coherent, program of technical assistance and toolkit development, as well as workshops in support of these objectives. 1. Summary Objectives and Targets The central objective of this project is to help accelerate the utilization of the many millions of dollars in collected and available Universal Service Funds in Sub-Saharan Africa (SSA), to achieve their designed purpose of expanding access to Information and Communication Technologies throughout the region. The widespread delays and barriers in disbursing of these monies, as well as the operational and administrative constraints of the Funds themselves, have only recently begun to be recognized on a broad level. As the magnitude of monies collected continues to increase in comparison with the amounts actually released to finance ICT investments, national and international institutions are acknowledging the need to build capacity and expand knowledge of best practices rapidly within USF administrations, telecommunications regulators, and the ICT sector in general in order to unleash the great potential of these Funds to begin closing the ICT access gaps that prevail across Africa. Much study remains to be done as to the specific causes and potential solutions for the blockages of these Funds, and alternative methods and tools for channeling needed resources into delivering ICTs to underserved locations and populations. (Some of this will be covered under the parallel SSA Assessment project; see Section 3 discussion below.) There is little doubt as to the potential positive development impacts of these critical resources. Allocating even a small portion of these capital funds, collected mainly from the very healthy profits of the commercial (largely urban-based) Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 90 telecommunications industry, toward the more equitable availability of electronic communications services and information technologies will return benefits to rural communities, low income households, as well as government agencies and even private sector providers themselves. Through the work covered by this Task Order, the INTEGRA project team will seek to increase the effectiveness of Universal Service Fund monies in creating concrete and measurable impacts in on telecommunication access in selected countries. The team will further contribute to the broader growth of ICTs throughout Sub-Saharan Africa through the spread of ideas, resources, and knowledge concerning USF best practices. During the course of the work on this Task Order, together with the companion SSA Assessment, we will define (and revise) specific target goals for the scope of these impacts. These target estimates will be based upon our review of the current status of ICT access as well as of USF operations in the region, the levels of cost and resources required to expand such access, and the degree of support that key USF administrations will require to implement their mandates effectively. The following sections highlight our initial view of these prospective targets. Project Impact Targets Direct Targets: The measurable impacts of direct technical assistance to selected USFs in 3-4 countries are likely to vary greatly, based upon a variety of factors, many of which will be beyond the influence of this project, or of the USF administration. Nevertheless, we can set some preliminary (and conservative) targets for achievable results from these efforts. Given that the project team has already initiated early consultations with the administration in Kenya, we can use the conditions and opportunities in that country as an initial baseline. Assuming that TA will be provided to three additional countries (each smaller than Kenya), we can make assumptions about the likely levels of spending, and anticipated impacts on rural ICT access, over a 3-year horizon. This approach assumes that the first projects funded by a USF under a technical assistance agreement with USAID will be smaller, pilot activities that help ensure the viability of the process and build capacity within the USF for larger deployments. For purposes of this forecast, the impacts of USF spending are estimated in terms of overall “rural ICT access growth”, which combines natural market development together with USF-subsidized growth, recognizing the two are synergistically related. In this context, “new ICT users” include both new mobile/voice customers plus Internet users, as measured by ITU and national telecommunications regulatory authorities (hence, a citizen who obtains both a mobile phone as well as Internet access would be counted as two “new users”). Target USF spending ($) and rural access growth (new rural ICT users) Country Year 1 Year 2 Year 3 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 91 Kenya $2.5m, 100,000 $5.0m, 200,000 $8.0m, 500,000 Country 2 $1.5m, 50,000 $2.5m, 125,000 $5.0m, 250,000 Country 3 $500k, 25,000 $1.5m, 75,000 $3.0m, 150,000 Country 4 $500k, 25,000 $750k, 50,000 $1.0m, 100,000 Indirect Targets: In addition to the impacts of direct technical assistance, a primary goal of this project is to create incentives and resources to support more rapid implementation of Universal Service Funds by numerous other national administrations throughout Sub-Saharan Africa. Although it may be difficult to ensure or monitor specific adoption of the methods and knowledge disseminated through the project’s activities, we do anticipate that there will be a measureable increase in USF disbursements, and hence of ICT access, in the immediate 1-3 year period of the project’s implementation and following its completion. Of course, most of this growth will be driven by increased private sector investment, but USF policies will help accelerate such market trends. Based upon the recent trends in this region and the estimated level of unspent funds available, we would tentatively define the following forecast: Current and forecast penetration (all SSA) 2011 2012 2013 2014 Mobile telephone 50% 60% 70% 80% Internet (incl. mobile Internet) 5% 10% 25% 45% Note that this forecast will be substantially updated upon completion of the parallel work on the SSA Assessment project, which will involve direct and in- depth review of the most recent status of USFs and ICT access in the entire SSA region. 2. Approach and Tasks This section presents the project team’s planned approach to completion of the designated activities of the Task Order, and achievement of the objectives described above. For each Task, we describe the work to be done, the deliverables to be provided (where appropriate), and the estimated time frame for completing the task. Task 1. Management and Reporting Requirements Subtask 1.1 – Task Order Management INTEGRA will be responsible for the effective management and administration of this project. INTEGRA management will ensure that work activities are performed in a timely and cost effective manner and with an optimal level of performance and impact. This will include close collaboration with the USAID leadership overseeing the project, regular reporting and consultations among all project participants, an effective communications strategy for all major reports, workshops, and other milestones, and Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 92 timely response to feedback and inputs at all stages. The Project Manager will be Mr. David N. Townsend, an acknowledged leading expert in Universal Access/Service around the world, with more than a decade of experience working with African USAFs. The expert team working with him will include personnel with extensive international USAF experience. They will also include individuals with previous and/or ongoing collaborations with Mr. Townsend, which will ensure smooth and efficient coordination of all tasks. The INTEGRA internal management team also has strong experience working within Africa on development issues, and managing broad projects of this nature. Direct administration responsibilities for this project will be handled by Eric White. Subtask 1.2 – Work Plan Development The INTEGRA team managing the project has developed this initial work plan, covering both phases I and II of this Task Order, to be delivered within ten (10) business days of USAID’s countersigning of the Task Order. After receiving comments on this work plan from USAID, the team will submit a final version to USAID within five (5) business days. Deliverable 1: Final Work Plan for two phases of the Task Order. Time Frame: Month 1, five days after receipt of USAID’s comments. Subtask 1.3 – Reporting Requirements The INTEGRA team will submit monthly reports to USAID, to arrive no later than the tenth (10th) day of the month. Monthly reports will highlight progress made on achieving the Task Order’s objectives, a plan for the work to be done in the next month, and any issues that need to be addressed. Deliverable 2: Monthly Reports Time Frame: The report for month 1 will be delivered no later than the tenth day of month 2. Each monthly report following this first will be delivered according to the same time schedule. Task 2. Undertake a Series of Capacity Building Workshops The Africa USF activity includes a workshop component that is aimed at facilitating dialogue among USF leaders, training them in best known practices, (both locally and globally), improving USAID’s understanding of major issues with USF operations, and making administrators aware of new innovative trends in USF management. This work will be undertaken by the INTEGRA team in consultation with other agencies and groups, including (but not limited to) CTO, ITU, InfoDev, the GSMA, and regional governmental organizations. Task 2 will be accomplished through the following steps. Note: In the task order the completion of “Task 2” depends upon the prior completion Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 93 of the “SSA Assessment” (the shorthand name for the “Increasing the Use and Impact of Telecommunications Universal Service Funds in sub-Saharan Africa” work funded by AFR/SD). However, this workplan acknowledges that, due to funding issues, it will not be possible to finish the SSA assessment before the workshops must be conducted. Consequently the work under this task order will be carried out concurrently to the SSA assessment (which will now be more about developing toolkits than researching USFs). Subtask 2.1 – Revisit the work plan for phase 2 of this Task Order that was previously produced under subtask 1.2. As the INTEGRA team has rescoped the program after discussions with David Townsend and taking account of budget and time realities, it will be necessary to review the work plan to incorporate this new scope. INTEGRA will submit a draft and final revised work plan for this task order (final work plan within five business days of receiving USAID’s comments on the draft work plan). Deliverable 3: Revised Work Plan, draft and final. Time Frame: Month 3 Subtask 2.2 – Plan a training and information gathering workshop in sub￾Saharan Africa The initial workshop will kick off the SSA USF work by conducting training and learning about the many challenges USFs in Africa face. This will serve both to help target technical assistance and tool development work, but will also serve as raining for the USFs. GBI will pursue a partnership with another organization; potentially InfoDev, CTO, or a regional body, to ensure that new material is disseminated at the workshop as part of the training USF administrators receive there. Time Frame: This workshop will be conducted by month 5. Subtask 2.3 – Produce a short list of countries with USFs that hold the most potential for successful TA. The list will consist of 6-10 countries that are the best prospects for direct TA, based upon both inquiries made under the concurrent SSA Assessment activity and on the workshop previously conducted as part of this Task Order. (Note, however, that Kenya will be included as the first country, as discussed below.) Deliverable 4: Refined short list of specific countries (coming out of workshop and concurrent SSA Assessment activity) with USFs that hold the most potential for becoming candidates for additional/focused technical assistance. Time Frame: Final list to be delivered by October 15th 2011. Subtask 2.4 – Translate the Materials collected in the concurrent SSA Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 94 Assessment into a 2-3 day final seminar and/or workshop format. These materials will include a series of presentations and participatory activities designed to stimulate thought, discussion, and learning among sub-Saharan African USF administrators. The presentations will start from the current format of the Intel￾USAID workshops and be modified to include new information emerging from the SSA Assessment. The workshop format will serve as both a dissemination activity for the toolkits and modules developed under the SSA assessment, and as an opportunity for GBI to provide a debrief on the TA activities in which they engaged. It will serve as a wrap-up of the complete SSA USF program, and offer steps that USFs can take going forward after the program completes. Time Frame: Months 14-16. Workshop in month 16. Subtask 2.5 – Update and refine the publishable series of documents emerging from the SSA Assessment, accounting for feedback received at the workshops. The INTEGRA team will incorporate feedback from the final workshop into a revised (and final) draft of the USF toolkit. Deliverable 5: Updated documents emerging from the SSA Assessment Time Frame: Month 16. The document will be presented and disseminated at the final workshop. Task 3. Provide Technical Assistance (TA) to a select set of 3-4 Universal Service Funds Task 3 is aimed at producing tangible, effective outcomes in individual countries that are well prepared to benefit from targeted technical assistance. The basis for the work will be the broad experience of GBI’s team of consultants, as well as the findings and methods developed during the concurrent SSA Assessment as that process unfolds. Note on Kenya: Though the TA phase of the project has yet to begin, a project has already been developed in Kenya, and will soon be implemented. This early work is entirely consistent with the project objectives, and affords a valuable opportunity to begin substantive work in a representative country, while contributing to the overall knowledge gathering and best practice planning efforts underway under the SSA assessment activity. The main tasks and deliverables for task 3 include the following. Subtask 3.1 - Develop Recipient-Specific Work Plans and Memorandums of Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 95 Understanding for Technical Assistance As agreements are reached with individual countries (and relevant USAID missions) to provide technical assistance, the project team will develop a country-specific work plan in each case. We anticipate that there will be three or four countries ultimately selected, depending upon the scope of assistance required, the readiness of the USAF administration, and other factors. In some cases, assistance may be provided in conjunction with, or as follow-up to, other support activities that the Fund may be receiving or may have recently undertaken. The country work plans will include key activities to be conducted, delineation of responsibilities, milestones, benchmarks for success, resource/time commitments of all parties, and a plan of action for moving each country’s USF forward beyond the TA being provided through USAID. It is expected that the ork plans will be completed in a staged manner. The work plans will also be done in close consultation with USAID and USAID Missions, and with the active participation and buy-in of the particular USAF. Final approved work plans will be the subject of negotiated Memoranda of Understanding among the USAID Missions and the national USAF administration or other government representative. Deliverables 6 and 7 (required separately for each country receiving TA): Respectively a specific work plan for each country and an MOU between the recipient USF and USAID. (Note that a draft work plan for Kenya is already under development.) Time Frame: Work plans will be developed as agreements are reached with selected TA recipient countries. Those completed first will begin TA earlier, while other agreements and plans are in development. All work plans will be developed during Months 5-11. Subtask 3.2 - Provide Technical Assistance for each country/work plan Based on the country-specific work plans and memoranda of understanding, the project team will deliver technical assistance to the targeted USAFs. This support may be adapted as circumstances change, to focus the work on yielding the most useful results for each country. The work will involve a series of two to four visits to each country, as appropriate, together with ongoing communications with USAF staff and other officials, and the development of relevant documentation, analyses, data, and guidelines in connection with the assistance. For each intervention, we will prepare a report which summarizes the activities undertaken, results achieved and anticipated, and lessons learned as well as tools and materials developed under the assistance, which can be shared among other USAFs. Each summary report will also include recommendations for further steps by the USAF, as well as potential follow-up support. Note on Kenya: As explained above, work on direct assistance to the Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 96 Communications Commission of Kenya (CCK) on the Kenya USF has launched at the outset of the project schedule. The advantages of this initiative include gaining additional time to collaborate with the CCK on their USF planning, over the course of the entire contract period. Time frame: Months 5-15 (excluding Kenya) Subtask 3.3 – Finalize the list of countries to receive Technical Assistance When the period of technical assistance begins (month 5) we will have a target list of technical assistance countries, but it won’t be a finalized list of those USFs that will actually receive TA from GBI. During the initial months of the TA engagement, GBI will be administering TA programs in some countries, while looking for opportunities to pursue them in other countries. During this process new information about which countries are best suited to receive TA may come to light, and it will be incorporated into the final decision as to where to offer TA. The final list of countries receiving TA will be submitted to USAID as soon as it is prepared, but no later than month 10. Deliverable 8: Final list of countries to receive TA Time Frame: Month 10 Subtask 3.4 – Finalize individual technical assistance engagements At the conclusion of GBI’s engagement with each individual country, a summary report of actions taken and results achieved will be written and presented to USAID. A key portion of this project summary will be a plan, worked out with the government, of how they can take what they have learned in their engagement with USAID and leverage it into making further improvements in the operations of their USFs. Deliverable 9: For each country, a summary report which includes a plan for how the country will move forward beyond the level of TA they received under the program. Time Frame- All reports completed by month 15 Subtask 3.5 – Prepare a Final Report Following completion of all tasks, the project team will prepare and deliver a summary Final Report on the project’s accomplishments, which will describe all work completed, key deliverables, as well as the impacts and lessons learned from the project, and recommendations for further follow-up work. This report will be based substantially upon the findings arising from all project tasks and prior deliverables; a draft version will be circulated among key stakeholders for comment, and will then be finalized taking account of relevant feedback. Deliverable 10: Final Report Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 97 Time frame: Delivery of Final Report by the end of Month 17. 3. Planning assumptions Key assumptions and other material considerations that underlie the tasks and activities described in this plan include the following: • Coordination with SSA Assessment Project and Preliminary Kenya Engagement: As indicated above, this Task Order has been developed by USAID in conjunction with a parallel procurement for the conduct of the “SSA Assessment” of the status of US/UA Funds throughout Sub-Saharan Africa. Taken together these two procurement actions make up Africa/SD’s SSA USF Project. They will occur simultaneously and build off of each other to form one coherent project. • The technical assistance project in Kenya allows for a number of key advantages, including direct and timely assistance to a newly formed USF in a strategically important country, practical experience with the procedures for developing an MOU for such Technical Assistance, and mutual learning in a hands-on environment that will be relevant to all other aspects of this work. All of these benefits justified taking advantage of the opportunity offered earlier this year, and support our decision to begin TA work early in the Kenya case. • Cooperation with National USF Administrations and USAID Missions: The project tasks, execution, and time frames assume that the project team will be able to obtain timely and effective cooperation from USF administrations in the target countries, as well as from local USAID missions in those countries. Each of these is critical to the team’s ability to provide assistance within a given country. The team will take initiatives to reach out to key officials responsible for USF decision-making throughout the various stages of the project, and will only pursue Technical Assistance options with those administrations that are both interested and capable to respond effectively to USAID-GBI support. This process will involve several steps, from initial contacts and communication to specialized needs assessment and ultimately negotiation of a custom Memorandum of Understanding with the relevant local authorities. If such procedures encounter substantial delays or roadblocks, the team may have to reconsider the practicality of offering TA to a particular country USF within the time frame of the Task Order, and pursue alternative recipients. Similarly, the team’s ability to execute an effective MOU and Technical Assistance effort within any country will be dependent upon the cooperation and support of the local USAID mission, which will ultimately be the signatory to the MOU. We assume that such cooperation will be readily achieved, and will Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 98 undertake close communication with missions in candidate countries early in the process of selecting TA recipients, to ensure buy-in and involvement by the missions. If any cases arise where mission support is impractical for any reason, we will, after pursuing all relevant channels to resolve the impediments, turn to alternative countries for Technical Assistance. • Coordination and Resource Sharing with Regional and International Institutions: Another key assumption for optimal completion of the project objectives, specifically for the broad dissemination and sharing of the ideas and resources generated by the team’s work, is effective cooperation with international and regional institutions that are also involved with ICT access and development initiatives. We assume that these organizations (e.g., ITU, World Bank, CTO, AUSAFA, COMESA, SADC, ECOWAS, etc.) will welcome consultations and mutual initiatives toward common goals in expanding Universal Access, even though each may have its own internal strategies and priorities. Ideally, we will find avenues for developing shared resources and cooperative programs, to multiply the effectiveness of each group’s activities, while avoiding excess duplication of efforts. The project team will take early and positive initiatives to reinforce relations with these institutions, and make them aware of our near and long-term goals. Where any barriers to cooperation are encountered, we will work with our counterparts to overcome them, and consult closely with USAID to ensure transparent and good faith communications. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 99 ANNEX 5. INTEGRA SELECTION CRITERIA FOR COUNTRIES TO RECEIVE INTENSIVE TECHNICAL ASSISTANCE Country Selection Criteria The following material was extracted from Integra’s Global Broadband and Innovations Program: Final Report for the Africa Universal Service and Access Fund Support Project, dated October 18, 2013. In order to identify and select among countries to be candidates for intensive technical assistance under the project, the project team developed and refined a series of qualifying criteria, with the aim of ensuring that the recipients of GBI support would be those best able to benefit therefrom. The criteria employed fell into three categories, with a number of specific elements under each. These can be summarized as follows: 1. Essential prerequisites: Basic conditions that must be met as a prerequisite for GBI support. a. Priority countries for AFR and USAID (15 specific countries) b. Established or planned USAF, sufficiently developed to merit assistance (according to the 4-level ranking system explained below) c. USAF administration active interest in receiving assistance d. Country government and local USAID mission engagement and cooperation e. Financial or other resource contribution by the Fund 2. High priority factors: The most important (practical) considerations to apply in determining eligibility and priority. These were defined as: a. USAF administration has necessary capacity to receive assistance b. Assistance likely to yield significant impacts on USAF performance c. The Fund’s challenges are exemplary for other Funds d. Recent or ongoing capacity building initiatives by the Fund 3. Additional considerations: Factors that will weigh in favor of selecting a country or Fund(s), if the priority factors are generally equivalent among candidates. Examples included: a. Diversity of issues and support requirements b. Opportunities for multi-country and/or institutional cooperation With respect to the status of USAFs in a given country (#1b above), the team employed the following ranking system:  Category 1: No established USAF legislation, policy, or institution;  Category 2: Recently established USAF, little progress to date in launching operations;  Category 3: Established USAF that has been under-performing, e.g., collecting funds without disbursing, slow implementation of objectives, etc. These may be sub-classified according to two groups: o 3a: Funds where impediments to success are mainly political or legal in nature (e.g., lawsuits, political disputes over use of funds); o 3b: Funds whose low performance is mainly due to internal factors: lack of capacity, need for planning, insufficient technical resources, etc.; Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 100  Category 4: Well established USAF that has been operating with relative success, but may still benefit from new perspectives, strategic realignment, etc. The project team conducted considerable research and engaged in discussions and contacts with Fund administrations and governments in more than a dozen countries to assess and validate these criteria. The final selections were: Kenya, Nigeria, Ghana, and Ghana. The background, objectives, and activities for each resulting in-country project are described in the following sections. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 101 ANNEX 6. GBI TORS FOR CORE COUNTRIES NIGERIA Preface: dTS received a scanned copy of the signed final TOR. Legibility was reduced due to high compression of pictures contained in the scan. In the interest of clarity, dTS obtained a copy of the draft TOR from Integra and adjusted them to match the contents of the final TOR exactly. Terms of Reference Technical Assistance: Nigeria USPF Institutional and Strategic Development 1. Introduction These Terms of Reference represent the agreed scope of cooperation between the United States Agency for International Development (USAID) Global Broadband and Innovations (GBI) program, and the Nigeria Universal Service Provisioning Fund (USPF), in association with the Nigeria Ministry of Communication Technology (MCT) and the Nigeria Communications Commission (NCC). This cooperation provides USAID/GBI technical assistance to the USPF and the associated agencies to help foster more effective utilization of resources toward national Information and Communication Technology (ICT) development in Nigeria. 2. Objectives The main objective of this cooperation is to help enhance the capacity and operations of the USPF, toward the goal of further developing the Fund’s role in stimulating wider access to ICTs in Nigeria. Specifically, this project will involve cooperation between GBI and USPF, on a set of tasks that will support the Fund’s ongoing institutional and strategic development processes, in the areas of Strategic Planning, Impact Assessment, ICT Gap Analysis, and Monitoring and Evaluation. 3. Scope of Work The project will consist of four main components: 1. Focused Advisory Support to the USPF in connection with current and planned institutional development initiatives 2. Assistance in the development and execution of an ICT Access Gap Study 3. Advice on development and implementation of Monitoring and Evaluation procedures Key tasks under each component are as follows: Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 102 1. Focused Advisory Support Under this first component, GBI’s experts will provide a range of advisory support to the USPF in relation to various current and pending initiatives to review and update the Fund’s mandate and operations. These include, specifically, the planned consultancies on development of a new Strategic Plan and conduct of an Impact Assessment study. In addition, the GBI team will advise and assist USPF in other related planning and institutional development initiatives and consultations. Beyond these activities, the GBI team will provide as-needed advice and support to USPF officials. 1.1. Review Terms of Reference, initial work plans, interim work product for USPF consulting projects on: (1) USPF Strategic Plan, and (2) USPF Impact Assessment Study. Provide advice, comments, recommendations regarding any enhancement or adjustments that may be warranted. Assist with oversight and follow-up of consultant activities as requested. 1.2. Provide input and advice on further USPF strategic planning and institutional development activities in relation to the above consulting projects, and other USPF management initiatives. These may include, as appropriate, assistance with and possible attendance at a planned Board Retreat in Nigeria, anticipated Study Tour visit(s) to representative Funds in other countries, and other meetings, consultations, or internal deliberations. 1.3. Provide advice and consultations upon request in relation to USPF planning, operational, and management issues as they arise, and in connection with the other activities of this Technical Assistance. 2. ICT Access Gap Study For this component, the GBI team will work with to define and develop an ICT Access Gap Study for Nigeria, with emphasis on both telephony and particularly Broadband access. The goal of the study will be to assist the Fund to understand and evaluate the current status and market potential of the ICT sector, and to identify the location, size, nature, and conditions of geographic and demographic gaps in network coverage and service availability. This analysis will allow more focused and strategic utilization of USPF resources, while providing valuable guidance to the Fund in support of its Mandate. The study will depend upon the availability of reliable and updated data on the status of telecommunications networks, utilization, customer demand, and a number of other factors, and will employ established market modeling and analytical techniques, adapted to Nigeria’s unique conditions. This work will be a collaborative effort between the GBI experts and Nigerian officials, with input required from multiple stakeholders to be successful. 2.1. Develop recommended analytical framework, study objectives, methodology. Identify GBI and Nigeria study team participants, assign tasks, establish timetable, milestones. This initial step will be launched with a study team meeting in Abuja. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 103 2.2. Establish study data requirements and expected sources, collection methods. This will likely be based primarily upon existing primary sources, especially licensed operator network deployment and service records, other NCC-based information sources, national census and other socio-economic and demographic databases, and other public records. Implementation. The Nigerian study team will be primarily responsible for data collection, while the GBI team will help specify required inputs and formats, review data as obtained, and recommend adjustments as necessary. 2.3. Define the analytical methodology modeling structure, assumptions, and other parameters for the study, based upon the available data and existing GBI templates. GBI’s experts will lead this work, in consultation with designated USPF and other Nigerian staff. 2.4. Conduct analysis of market data, generate tentative findings on ICT access gaps, review with appropriate officials and stakeholders; produce sensitivity analysis and modifications as necessary. Generate final gap analysis report. The GBI team will oversee and assist with this activity, while final calculations and reporting will be primarily the USPF team’s responsibility. 3. Monitoring and Evaluation This phase of work will involve analysis and development of Monitoring and Evaluation criteria and methods for oversight of USPF funded ICT deployments. The goal is to assist the USPF in verifying and enforcing the proper utilization of its funding support by private contractors, and in analyzing the cost-effectiveness of USPF programs. This work will align with the pending Impact Assessment study, while focusing primarily upon establishing appropriate methods for USPF project oversight. 3.1. Review the work of the Impact Assessment consulting team in evaluating the past and current status of USPF project implementation, monitoring, and oversight standards, including reporting requirements, data collection, verification criteria, and enforcement methods. Develop critical evaluation of the Impact Assessment study’s findings on effectiveness, gaps, and needs. The GBI team will work with USPF staff to conduct this review. 3.2. Develop principles, standards, methods for enhanced monitoring and evaluation of USPF projects, including criteria to be included in tender procedures, contractual reporting and compliance obligations, and processes for conducting verification audits, as well as enforcement standards. The GBI team will provide recommended approaches and international examples in support of USPF staff. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 104 3.3. Establish implementation planning steps for introducing new monitoring and evaluation procedures, including task responsibilities for USPF staff and any necessary TORs for outside contractors. Determine timetable for initial implementation trials. GBI will provide advice and assistance to USPF staff on these elements. 4. Outcomes and Deliverables The Technical Assistance will produce a series of expected deliverables, depending upon the specific requirements developed under each component of the work, which the USAID/GBI team will prepare in coordination with USPF: 1. Review and evaluation of consultancies on Strategic Planning, Impact Assessment. 2. ICT Access Gap planning and data requirements, statistical model file(s), and final analysis report. 3. USPF Monitoring and Evaluation Recommendations Report. 5. Timeline The assignment is intended to be concluded within 8 months from project launch. 6. Responsibilities of the Parties The Parties to these Terms of Reference are committed to working together to achieve the project objectives and identified outcomes. The specific responsibilities to be undertaken by each party include: USAID/GBI: Manage and oversee project activities in cooperation with USPF; provide inputs, examples, analysis, and recommendations as appropriate for each component of the assistance. Participate in meetings, discussions, consultations, and other relevant communications with USPF and other stakeholders. USPF: Ensure adequate staffing and personnel assignments to allow effective collaboration and implementation of the Technical Assistance. Cooperate with GBI team in project planning, management, and implementation; provide local resources and facilitate collaboration as needed. Provide financing for any requested participation of GBI team in official Retreat or other meetings outside Abuja. Provide access to available data resources and collect additional data as needed; organize and facilitate stakeholder consultations; review and contribute to all analysis, recommendations, and reports. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 105 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 106 GHANA Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 107 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 108 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 109 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 110 KENYA Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 111 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 112 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 113 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 114 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 115 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 116 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 117 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 118 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 119 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 120 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 121 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 122 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 123 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 124 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 125 TANZANIA Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 126 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 127 ANNEX 7. EVALUATION METHODOLOGY – DETAILS Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 128 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 129 Prepared for the United States Agency for International Development, USAID. Contract Number RAN￾I-00-09-00015, Task Order Number AID-OAA-TO-12-00007, Evaluation of Universal Service and Access Funds. Implemented by: Development & Training Services, Inc. (dTS) 4600 North Fairfax Drive, Suite 402 Arlington, VA 22203 Phone: +1 703-465-9388 Fax: +1 703-465-9344 www.onlinedts.com Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 130 EVALUATION SERVICES IQC FINAL PERFORMANCE EVALUATION OF THE UNIVERSAL SERVICE ACCESS FUND (USAF) IN SUB-SAHARAN AFRICA EVALUATION METHODOLOGY October 30, 2013 DISCLAIMER The authors' views expressed in this deliverable do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 127 CONTENTS Contents ............................................................................................................................................ I. Background, Rationale, and Objectives ...................................................................................... II. Overview of Methodology ............................................................................................................ III. Data Collection and analysis ...................................................................................................... A. Document review .............................................................................................................................................. B. Key Informant Interviews ................................................................................................................................ C. Electronic Survey ............................................................................................................................................... D. Cost Effectiveness Analysis .............................................................................................................................. E. Data Analysis ....................................................................................................................................................... IV. WorkPlan .................................................................................................................................... Sub-Annex 1. List Of Key Informants ............................................................................................. Sub-Annex 2. List Of Key Questions .............................................................................................. Sub-Annex 3. Key Informant Interview Guides ............................................................................. Sub-Annex 4. Draft Evaluation Report Design .............................................................................. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 144 I. BACKGROUND, RATIONALE, AND OBJECTIVES The dTS team is tasked with conducting a final performance evaluation of USAF, a project implemented by Integra, LLC (hereinafter referred to as Integra). The aim of this evaluation is to assess the effectiveness of Integra’s USAF technical assistance in Sub-Saharan African countries. As originally envisioned, this Task Order was to provide USAID with important performance information regarding the efficacy of a project to provide technical assistance to USAFs in (an estimated) 3 to 6 countries in Sub-Saharan Africa over an (anticipated) 3 to 5 year period. It was also intended to provide insights for the USAID Global Broadband Innovation (GBI) Program, which is working to improve the use of universal service funds on a global basis. During the first few months of the USAF Project’s operation, Integra narrowed down the intended targets for direct technical assistance initially to four countries (Kenya, Tanzania, Ghana, and Nigeria). This SOW for this evaluation has three objectives: 1. To measure the impact of the Project via a performance evaluation; 2. To measure cost effectiveness of approaches used to achieve the impact of the USAF Project; 3. To develop a summary document and presentation to make the results of this SOW accessible and useful to USAID teams, other donors and to universal service funds themselves. As stated in USAID’s Terms of Reference (TOR) for the USAF Project, the impact to be evaluated can be summarized as follows: 1. Increase the actual use of telecommunications universal service and access funds in SSA and, by doing so, 2. Accelerate the extension of affordable and sustainable access to telecom services (including broadband Internet services where permissible) to those not yet served. The TOR does not request a full impact evaluation with the requisite rigorously defined counterfactual, but a performance evaluation focusing on impact which answers the following questions: Did the project meet its objectives? Have expected results been achieved? dTS is not expected to focus on questions related to how well the project was managed, how it was perceived, and how well inputs were achieved. The challenges of attribution of results and achieving results within the project time period are recognized by USAID, and dTS is expected to propose and carry out an approach that addresses these challenges to the extent possible. The USAF Project has a wide variety of options to achieve impact, some resource intensive (e.g., technical assistance provided by experts directly to one universal service fund) and some far less so (e.g., conducting a workshop for several funds together; involving the staff of one or more funds in assistance provided to a specific fund; preparing toolkits for several funds to use). The second objective of this evaluation is to understand which approaches are most cost effective; determine key considerations when selecting approaches in the future; and provide assessment of the relative effectiveness of approaches to contribute to meeting project objectives. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 145 The TOR for the evaluation stated a third objective to develop a variety of summary documents, presentations or other approaches to communication to make the results accessible and useful to USAID teams, other donors and to universal service funds themselves, including developing and executing an outreach plan. Due to budgetary constraints, the original dTS technical proposal was modified (and accepted by USAID) to reduce the LOE and expected deliverables for this objective. The agreed objective is for dTS to prepare a comprehensive final report and deliver an in-person presentation to USAID of the evaluation’s findings and recommendations. Project Background: The goal of USAF is to assist core countries in sub-Saharan Africa to increase the effective use of telecommunications universal service funds in sub-Saharan Africa. It is an innovative project with high visibility within and beyond USAID, and could have significant impact on the availability of affordable access to telecommunications services for the poor. Further, it may be adapted to other regions. The project, entitled Telecommunications Universal Service Fund/Universal Access Fund (USF/UAF) Project (referred to below as USAF Project or simply Project), is being implemented by Integra and is funded by USAID/AFR/SD/EGEA. For purposes of this evaluation, it is to be considered one project although it has been funded through two mechanisms, both providing funding to Integra, the contractor implementing the project. Access to telecommunications29 services can increase the impact of development activities across all sectors, including education, health, agriculture, government and the private sector. There is a proven link between access to mobile phone services in Africa and economic growth. Because of this, almost all USAID and other donor development projects in SSA use telecommunications-enabled tools and services to increase their impact. Unfortunately, the requisite telecom services are often not accessible in a significant portion of the geographic areas where such donor projects focus, including rural and non￾rural areas where the poor and ultra-poor reside. A primary reason for this uneven coverage is that these are areas where telecom service providers are less likely to be able to provide services on a viable commercial basis. Extending the needed infrastructure out to these areas and supporting on-going operations does not make financial sense, given relatively low potential customer revenue. Given the importance of access to telecommunications services for development purposes, many countries have established telecommunications Universal Service Funds or Universal Access Funds (USFs/UAFs) to provide services to these underserved areas. Based on legislation and regulations, countries establish funds that collect a small percentage of revenue from licensed telecom services providers. The funds are then distributed using a variety of approaches (often a reverse tender process) to provide needed subsidies to telecom services in target areas. Countries vary in the percentage of revenue collected, approaches to managing the funds, permissible uses of the funds, and a variety of other factors related to USAFs. Globally, the track record of USAF funds varies considerably, with most countries falling far short of achieving the full potential benefits of such funds. To realize these benefits, there are several prerequisites. Countries must:  Pass legislation and establish the regulations needed to create a USAF. 29 Many countries in SSA and throughout the world have now expanded the concept of universal access to include access to broadband ICT services, i.e., it is no longer just basic telecommunications services that need to be expanded to rural and underserved areas. At the same time, the lines between telecommunications and other ICT services have also blurred due to technological advances. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 146  Establish a fund with an entity to operate it.  Collect funds and keep their identity separate from general government funds.  Disburse funds for their intended public purpose. USAFs in South America probably have the longest track record, have been studied the most, and offer experience that can be helpful to funds elsewhere. Recently, the ITU (International Telecommunications Union) commissioned a study of the status of USAFs in SSA, yielding some stark findings showing that these funds fall far short of their potential. Forty-five countries responded to the survey. Of these, 25 have laws calling for USAFs; of these, only 15 have actually collected funds and even fewer – eight – have actually disbursed any funds. Even worse, on average, these latter funds have only managed to disburse 13 percent of the monies collected. As elsewhere, USAFs in SSA are trying to design and operate to achieve maximum benefit. Current challenges include: 1. Moving from legislation to action – the steps needed to establish an entity to manage the fund; 2. Actually collecting funds; 3. Managing the funds prudently, reflecting good business practices; 4. Disbursing the funds in accordance with policy and legislative guidelines, reflecting (and adapting as needed) best practices elsewhere; and 5. Monitoring and evaluating impact and adapting practices based on lessons learned directly and by other funds. Africa’s USAFs have specific opportunities that have the potential for helping them address these challenges. First, they can learn from funds with long track records. Second, in late 2008 the USAFs formed an association, the African Universal Service and Access Fund Association (AUSAFA), to provide support and help via collaboration between countries. The Commonwealth Telecommunication Organization (CTO) serves as the Association’s secretariat. Third, several funds in Africa, including those in Senegal, Uganda, Mauritius, South Africa and Ghana, are already operational in differing degrees, and may serve as examples to other African funds more recently formed or just now becoming operational. Fourth, the ITU has funded two phases of analytical work regarding USAFs in SSA and considers this an important topic. USAFs in SSA have some unique challenges. The region has the highest priced access to the Internet because of the general reliance on satellites for access to international gateways and between countries. This has begun to change recently with the arrival of several undersea cables and a variety of investments in terrestrial fiber networks by private investors, governments and multi-lateral organizations. Also, cell phone network providers have expanded dramatically in recent years, are moving to using Internet Protocol (TCP/IP) for their core infrastructure, and are also upgrading their networks to support Internet access. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 147 II. OVERVIEW OF METHODOLOGY dTS had planned to conduct the evaluation in two phases. The first phase was the work needed to design the approach to conduct the work described above. This design work required dTS to meet with the USAF Project team to understand the project’s approach, types of and accessibility of data to conduct these evaluation services, and ways to collect required data – both quantitative and qualitative. The USAF Project team was directed by USAID to be cooperative, and thus far, has been. At the conclusion of the first phase, dTS was to prepare a detailed work plan for the second phase and this evaluation methodology. Because of a one-year extension of the USAF Project, USAID agreed that dTS would go into a monitoring mode (after it also received a one-year extension) until such time that it believed more active evaluation activities were warranted. dTS has launched the active evaluation stage and provided a Draft Phase II Work Plan to USAID. As part of the draft Phase II Work Plan, dTS also submitted to USAID a request for modification of the project LOE (with the LOE remaining within the original budget) and anticipated project-related travel. This evaluation design document has been prepared to provide additional depth and foundation to the work activities reflected in the Phase II Work Plan. This evaluation is not designed as a rigorous impact evaluation exercise. Our team will, however, strive to identify observable outcomes and impacts of the project and to obtain independent measures of these outcomes/impacts. Wherever possible, we will analyze data on activities and results disaggregated by gender. The methodologies available have certain limitations from an empirical standpoint, notably the absence of statistical checks and experimental findings to counter biases in the information we collect.30 The table below outlines the various methods of data collection and analysis that will be used during the evaluation. The evaluation will start with a thorough document and literature review, and then proceed with key informant interviews conducted by the dTS team before and during the country visits, along with electronic surveys and collection of project internal data. There are two distinct components of this project that will be implemented in parallel: Impact Evaluation: Methodologically, the focus of the evaluation design will be on measuring the specific impacts attributable to the USAF Project outputs. Essentially, this will be done by determining the change in available indicators over a period beginning as close as possible to the start of the USAF Project’s. Since the initial USAF reports indicate that there is limited quantitative evidence that of the project impact on number of users or penetration of the networks, the evaluation team will concentrate on qualitative evidence of the new trends that eventually lead to the changes in the main impact indicators and expert opinions of key informants in each county about the extent to which the project contributed to the development of these trends. The evaluation team will also collect data from project participants about the effectiveness of particular project interventions through mini-surveys. Cost-effectiveness analysis: The cost-effectiveness evaluation will make use of all of the data collected through the impact evaluation, but will require several additional data collection and analytic 30 Also worth noting is the timing of the evaluation, at a point before the final outcomes of the current project can be observed. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 148 activities as well. Essentially, we will analytically relate the impacts identified through the impact evaluation to data on the resource costs of the services delivered by the USAF Project. First step will be to identify the costs of technical assistance being offered by the project in each country. This will be done on the basis of the reviews of USAF Project documents and discussions with INTEGRA and USAID personnel. Through the key informant interviews, the evaluation team will collect expert opinion about the effectiveness of activities in each country. It will also attempt to identify if there were other activities in the country or region that lead to the similar results. As the result of this work, the evaluation team will provide measures of the following: cost per direct beneficiary (by country) and estimated cost per indirect beneficiary (final recipient of the telecom service), and cost of similar interventions done by other donors or under other projects, if available. We recognize that our analyses will have to deal with the challenges associated with combining cash and in-kind amounts, and with varying country measures and contexts. We also assume that that it will be necessary to develop appropriate units of each type of assistance that we can ultimately associate with dollar costs in the final cost-effectiveness calculations, while at the same time controlling for differences in the USAF stage of development and country contextual factors. Table 1: Description of the Evaluation Methods and their Associated Purpose and Expected Outcomes Method Purpose of Method Expected Outcomes Pre-trip activities Document review Analysis of project activities and project deliverables Collect information on the project structure and achievements, and construct list of key informants, partners, etc. Determine overall environment for the project implementation Record documented project outcomes at the regional and country level Developing a list of project activities for cost-benefit analysis Review of project structure List of project activities List of recorded project outcomes List of project partners and key informants Information on project impediments and outcomes. Key informant interviews Collect information about the project, validate evaluation methodology and tools, and collect informants opinions about effectiveness of USAF Tabulation of typical responses about the project performance Tabulation of typical expert opinions about project outcomes and impacts Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 149 Method Purpose of Method Expected Outcomes Information about the project activities, deliverables, and cooperation with main stakeholders Collect informant opinions about project outcomes at regional, country, and firm level Obtain information about the costs of project activities and estimated outcomes of these activities Expert assessment of project outcomes and impacts List of suggested program improvements Extended list of the key informants for the in￾country data collection Field-based activities Key informant interviews Collect information about the project, validate evaluation methodology and tools, and collect experts opinions about effectiveness of USAF Information about the project activities, deliverables, and cooperation with main stakeholders Collect informant opinions about project outcomes at regional, country, and firm level Lead for the secondary sources of data for the in-country project impact Tabulation of typical responses about the project performance Tabulation of typical expert opinions about project outcomes and impacts Expert assessment of project outcomes and impacts List of suggested program improvements List of participants in the project activities Electronic survey of participants in the project trainings and study tours. Collect primary data on the possible impact of the project on the capacities of Core Countries telecommunication institutions in sub-Saharan Africa.31 Quantitative measures of perceived changes in human and organizational capacities; Indication of the project success to produce sustainable changes in the capacity Review of documents provided by Integra Collect primary information Core Countries Integra’s USAF activity in sub-Saharan Africa Objective measures of the potential USAF impact in sub-Saharan Africa. Validation of project reporting Post-field activities 31 See Annex 7 for a list of persons and organizations being contacted for interviews. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 150 Method Purpose of Method Expected Outcomes Participatory discussions with Integra and USAID Collect information about the project structure and performance of USAF Additional information about project activities, best practices, and recorded outcomes and impacts Expert assessment of project outcomes and impacts Expert recommendations on suggested improvements and future activities To assess the impact of the more intensive TA to the targeted countries, we propose to supplement the approaches above with visits to four of the target countries plus contact with the AUSAFA and possibly other mentoring organizations. The purpose of the country visits is to explore in greater depth the linkage between USAF Project activities and the identified impacts. The visits to the target countries will include key informant interviews with: (i) personnel at the country USAFs; (ii) government regulators and policy-makers; (iii) service providers; and (iv) potentially, a number of other institutions and organizations in any given country that might be able to supply relevant input. For example, there may be a Ministry of Rural Development or its equivalent, or units within the Ministries of Education and Health that deal with ICT deployment issues. In many countries, UNDP plays an important role. Various NGOs may also be engaged in universal service-related activities. Each country visit will be conducted by our Team Leader, assisted during the visit by a local administrative/research aide. Pending the initial reviews and discussions with USAID and INTEGRA, we propose the following measures of impact: (i) Increased USAF Usage: We propose to measure USAF use in terms of revenue collections and disbursement of funds expended to increase access (as opposed to expended for operations) in the core countries where funds are being spent. This data may be available from each of the USAFs for the period from when they began operation, but in some countries, like Kenya and Tanzania, this data may be less relevant to the USAF work. Our analyses will compare the income and expenditure data for the period since the USAF Project’s initial interventions against data for prior years. We will consider using increases in access/telecoms across SSA countries as a counter-weight measures. (ii) Telecom Access: Changes in the number of people with access will be based on population data available from national statistical offices, data from the country USAF, and data from official sources relevant to institutional users (e.g. schools wired, telecenters). Our initial calculations will be of the population residing in the geographic boundaries of the area(s) receiving new levels or types of service, and penetration defined as, for example, number of fixed lines per 100 population; percentage of population within walking distance of a telecenter, or percentage of schools wired up. (iii) Accelerated Access: This indicator of impact requires a comparison against a measure of what the level of access would have been in the absence of the USAF Project. We propose comparing the access data from USAF Project target areas to historical data on telecommunication service expansion over the past decade. It is expected that only Nigeria and Ghana will have sufficient information for this measure. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 151 (iv) Increased Take-up: For this impact indicator we propose to measure changes in telecom penetration per 100 inhabitants based on user data either from the USAF or the USAF supported service provider. (v) Increased Capacity: These measures will emerge from the interviews and document reviews conducted during the evaluation design phase when the nature and intended outputs of the various forms of project assistance become clearer. We anticipate they will be qualitative measures based on key informant interviews and document reviews. (vi) Direction of USAF in the Country: Given the potential that the results of Integra’s interventions may well only be realized over a longer time frame than is allotted to the evaluation effort, we will observe to the extent practical whether or not the interventions have increased the country’s capacity in developing and operating the USAF in a manner such that the “directionality” of the USAF appears sound. In finalizing and then implementing the impact evaluation design it will be important to identify and address various important methodological and practical challenges. These include:  How to Tailor Measures of Impact to Country Circumstances: We will be focusing on the four core countries (Kenya, Tanzania, Ghana and Nigeria). Each of these countries was (and is) at different stages of USAF implementation when the USAF Project began – and remain so today. It will be important to tailor measures of impact to country circumstances such as the variable costs and technical challenges to implement projects and provide service to targeted region(s).  Technological Evolution: Telecom/ICT technology is in a constant state of flux. What is cutting edge today may not be appropriate or cost-effective tomorrow. In assessing increases in access and usage, it will be important to develop simple measures of the relative cost-effectiveness of newly available platforms and services.  Attribution of Impacts to the USAF Project: Determining the extent to which USAF Project technical support can be credited with positive changes in impact indicators will pose the greatest analytic challenge. One threat to attributing positive changes to the USAF Project assistance is the possibility that the change would have occurred in any event, for example, simply due to the passage of time, increases in the availability of government funding, improvements in the macro-economic climate, licensing of new operators (We recognize that universal service obligations are part of the licensing process in some countries), or the influence of initiatives from other donors. This can be addressed partially through identification of appropriate comparison regions where available and reviews of country USAF data and reports, as well as with interviews of USAF and government officials. To sort out the effects of these influences from the impacts of the USAF Project we propose to collect survey and in-person interview data from targeted USAF personnel, country government officials, and service providers. III. DATA COLLECTION AND ANALYSIS A. DOCUMENT REVIEW Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 152 The document review will be conducted in two phases: the pre-field documents and literature review, and the ongoing review of documents collected during the field activities. The goal is to make the document review and the evaluation as comprehensive as possible. The pre-field document and literature review has been conducted during the first year of the project, with the following objectives: 1. To learn about the components and activities comprising the project; 2. To assess the strength of the causal link drawn between the above project elements and the anticipated project results in the project planning and implementation materials; 3. To glean some sense of “lessons learned “ and adjustments made during the USAF Project; and 4. To inform the development of the evaluation methodology and field data collection tools. The pre-field document and literature review included USAF administrative and technical documents, such as annual reports and work plans discussing similar projects or documenting changes in Core Countries’ access to telecommunications/ICT services. USAID provided Integra reports, the USAID TOR for USAF Project and other Integra material current to time of kick off meeting. Integra also forwarded all reports to dTS. dTS has established an ongoing USAF Project library which will contain all documentation received or prepared by dTS relating to the project, including all email correspondence. This literature review is being conducted by both the Team Leader and the Evaluation Specialist, with support from the research assistant. The latter will concentrate to a large extent on the review and analysis of technical documents concerning initiatives in the area of telecommunications universal service funds in sub-Saharan Africa. The on-going document review during the field activities will include documents that become available to the evaluation team as a result of the key informant interviews, such as policy analyses, drafts of laws and regulations, and reports with related discussions. Much of the qualitative data reflected in these documents will be used to analyze the host government’s needs and USAF’s proposed solutions in the legal-regulatory context. B. KEY INFORMANT INTERVIEWS Key informant interviews (KII), our central data collection method for this evaluation, will be used to deepen our understanding of the USAF Project, to obtain experts/stakeholders opinion on the success of USAF activities and progress towards objectives, and to assess the project’s past or future impact. In particular, we will use the key informant interviews to: 1. To hear the views of project partners and counterparts on the quality of Integra’s cooperation with them and the success of project activities; 2. To hear expert assessments of changes in Core Countries telecommunications services and any USAF contributions to those changes 3. To corroborate influence and claimed outcomes; and 4. To assess the sustainability of the USAF effort. Our initial list of key informants includes USAF Project management, USAID/AFR/SD/EGEA Core Countries representatives, local independent experts from NGOs and international organizations. The initial list of the key informants will be expanded and corrected based on our interviews with USAF Project management and recommendations from USAID and other stakeholders. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 153 Key informant interviews commenced during the first week of the project field work. This first wave of the key information interviews was conducted by the Team Leader and the other ICT expert32 in Washington, DC. The purpose of this work activity is to determine what organizations in the SSA area are actively involved in supporting USAF initiatives, and to determine the potential impact of such involvement (if any) on USAF progress in SSA countries, and to clarify the list of the key informants in the countries of interest. The second wave of the KII will start with the beginning of the in-country field work. The Team Leader will travel to the four core counties to conduct the interviews. In certain cases of overlap in subject matter interest, the interview may be concurrently conducted by the team members. Key informants will be contacted in advance to schedule interviews. If key informants are not able to meet during the time available in-country, dTS will seek to complete the interviews by phone or, if available to the person, via Skype or a written questionnaire. A semi-structured questionnaire with open-ended questions will be used as a basic guide to conduct key informant interviews. It will be used to guide the key informant interviews and form the basis for the meeting summaries developed in English in electronic format. The questionnaire may be revised based on the information obtained during the interviews. If necessary, a brief statement about the purpose of the study and voluntary participation in the survey will be translated into local languages in Core Countries for the discussions. Interviewers will read the statement and answer respondents’ questions before proceeding with the interview. However, given that the English language is generally spoken and understood in the four core countries, we do not anticipate the need for such translation. Key informant interview information will be analyzed as soon as possible after the interviews are conducted. The experts will identify most typical responses to the questions during the key informant interviews, and will tabulate the typical responses from all experts. A separate list of “outlier” responses will be created for further investigation. Follow-up interviews will be conducted by the Evaluation Specialist or the local expert if clarification is necessary. C. ELECTRONIC SURVEY The evaluation team plans to conduct a small-scale electronic survey of the participants of the project training, workshops, study-tours and other capacity-development activities. The survey will ask three groups of questions to determine the following: 1. How useful were the learning experiences provided? Please comment on the topics, structure, facilitators and trainers involved. 2. Do participants consider the activities important for their capacity building? 3. Did the capacity building result in any changes within the participant’s organization? If so, please describe them. The project team will rely on Integra to provide lists and the contact information for the project’s capacity building workshops and webinars participants. Once such information is obtained, the evaluation team will finalize the questionnaire and send it to the participants by email. If email is not 32 The other ICT expert referenced here was Mr. Morell, who was a team member until the evaluation went into a monitoring mode, and who was unable to rejoin the effort due to other commitments. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 154 available or the participant is not responding, the team will follow up with two phone calls and will attempt to conduct the survey over the phone. Since the survey sample will not be drawn randomly from a proper sampling frame, it will not be possible to generalize the results of the survey. Still, the results—combined with other data—could indicate the impact of the USAF Project on the capacities of the institutions. If Integra is not able to identify all participants at the various workshops, we will seek to obtain such listings from the sponsors of the events (Intel, CTO, etc.). D. COST EFFECTIVENESS ANALYSIS The cost-effectiveness analysis will be used by the Evaluation Team to collect internal data and analyze the cost of different project activities compared with the benefits of the activities. The Evaluation Specialist will analyze the project budget and talk to the project team to determine a set of standard activities implemented by the project, such as trainings, seminars, technical assistance by an expat to client, research, etc. The evaluation team will request Integra to provide a detailed project budget to estimate total costs for each activity. The evaluation team will work with Integra to determine that the estimates of the costs are accurate and all indirect costs are properly included in the cost of activities. In parallel, the team will collect information about the direct and indirect beneficiaries of each of the project activities. The information will be collected from Integra reports, the key informants/experts, or secondary data in country. The evaluation team will calculate the ratio of activity costs to the number of direct beneficiaries of the activities. E. DATA ANALYSIS Qualitative Data Analysis will rely on two primary methods: 1. Tabulation of Qualitative Results – The team will construct a list of main achievements of the project based on different information sources. It will collect claims supporting or disproving these achievements in a table, and will analyze frequency and substance of this evidence to determine the degree to which each achievement can be considered proven. 2. Cross-Verification of the data – The Evaluation Team fully anticipates that data analysis may present competing and even conflicting claims. For each statement derived from the qualitative data, the team will attempt to find at least one piece of supporting evidence in each of the sources of qualitative data: literature/document review, key informant interviews, and secondary data sources. Quantitative Data Analysis will consist of the following:  Calculation of the quantitative indicators for the possible project impacts and outcomes at the country and individual levels – to the extent possible. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 155 IV. WORK PLAN The evaluation will be implemented according to the division of responsibilities listed below and the step-by-step work plan outlined in the table. Phase II Work Plan A Work Plan for the Phase I was sent to USAID on 6 April 2012. While development of the Phase II Work Plan was the prime deliverable due from Phase I, the Phase II Work Plan was delayed due to the project being moved into a monitoring mode. Therefore, developing the Phase II Work Plan was one of the first deliverables for this active engagement period. An initial Draft Phase II Work Plan was provided to USAID. The submission of the revised Phase II Work Plan has been held up pending USAID approval of requested LOE allocations among the dTS team. The Phase II Work Plan will contain as much specificity as possible, given the potential for varying approaches and timelines in the Core Countries. At a minimum, the Phase II Work Plan will reflect an assessment by dTS of the components necessary to meet USAID’s requirements for the evaluation. Step-by-Step Work Plan: Phase Activity Responsible Party(s) Time Schedule Award Finalize team arrangements dTS Project Management Team (PMT) February 1, 2012 Task 1: Literature Review and Evaluation Methodology Preparation Clarification call with USAID Core Countries and Request/gather background documents from USAID Core Countries & Integra PMT & Team Leader March 12, 2012 Team building PMT, Evaluation team March/April, 2012 Review background documents Evaluation team April/May, 2012 Prepare and submit revised draft work plan to USAID Team Leader April 11, 2012 Prepare and submit final evaluation methodology and data collection tools, and receive USAID/Core Countries approval Evaluation team Oct, 2013 Arrange logistics and schedule for field work PMT, Team Leader Nov, 2013 Task 2: Conduct Fieldwork Travel to Core Countries Evaluation team Dec, 2013 & Jan 2014 Orientation meetings with USAID & local partner Evaluation team Dec, 2013 & Jan 2014 Finalize site-visit interview protocols, including key informant and mini-survey selection; begin Core Countries interviews and surveys Evaluation team Dec, 2013 & Jan 2014 Key informant interviews and surveys in Evaluation team Dec, 2013 & Jan Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 156 Core Countries 2014 Task 3: Report preparation and Briefing Submit Draft Reports Complete data collection and analyses Evaluation team Feb, 2013 Prepare summary of main findings and preliminary conclusions Evaluation team Feb, 2013 Hold partner validation meeting with stakeholders and debriefing with USAID Evaluation team Feb, 2014 Team returns to U.S. Evaluation team Feb, 2014 Prepare & submit draft final report Evaluation team, PMT June 24, 2014 Task 4: Submit Final Report Respond to USAID comments on draft report; submit final report Evaluation team, PMT July 31, 2014 Task 5: Updates Regular updates or briefings to USAID Evaluation team Ongoing Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 157 SUB-ANNEX 1. LIST OF KEY INFORMANTS Respondent(s) Organization Country Contact Information (if available) Date Interviewed USAID Albert Waudo, Program Manager USAID Kenya awaudo@usaid.gov +254 (20)-862-2000 USAID U.S. Embassy Complex United Nations Avenue Gigiri P.O. Box 629, Village Market 00621 Nairobi, Kenya 01/24/2014 Alefia Merchant, Agriculture Development Officer USAID Nigeria amerchant@usaid.gov +234 (9)-461-9300 7, Mambilla Street Off Aso Drive Maitama District, Abuja, Nigeria 12/16/20143 Darrell Owen, Consultant USAID USA darrell_owen@msn.com 1300 Pennsylvania Ave NW, Washington, DC 20004 04/22/2014 Eric Johnson, Economic Growth Office USAID Tanzania ejohnson@usaid.gov Not interviewed James Lykos, Private Enterprise Development Officer USAID (formerly Director of Research & Planning, EnGenCore) Nigeria 7, Mambilla Street Off Aso Drive Maitama District, Abuja, Nigeria E-Mail: abujainfo@usaid.gov 12/16/2013 Jeffrey A. Cochrane, Division Chief – ICT USAID USA jacochrane@usaid.gov 1300 Pennsylvania Ave NW, Washington, DC 20004 04/24/2014 Joe Duncan, Acting Team Leader E3 ICT, Alternate COR for USAF Project USAID (now holds different position) USA jduncan@usaid.gov 1300 Pennsylvania Ave NW, Washington, DC 20004 04/24/2014 Judith Payne, Contracting Officer’s Representative (Former), USAF Project USAID USA jpayne@usaid.gov 1300 Pennsylvania Ave NW, Washington, DC 20004 04/23/2014 Kevin Martin, Private Enterprise Officer USAID Liberia kemartin@usaid.gov Not interviewed Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 158 Respondent(s) Organization Country Contact Information (if available) Date Interviewed Kevin McCown, Policy & Research Advisor USAID/Tanzania Tanzania kmccown@usaid.gov +255-22-229-4490 (x4463) 686 Old Bagamoyo Road, Msasani P.O. Box 9130, Dar es Salaam 01/17/2014 Kim Kim Yee, Private Enterprise Officer Office of Economic Growth, USAID/Ghana Ghana kyee@usaid.gov +233-(0)-302-741-729 USAID No. 24 Fourth Circular Road, Cantonments P.O. Boc 1630; Accra, Ghana 12/20/2013 Lily Beshawred, Director, Program Dev. & Implementation USAID East Africa Ibeshawred@usaid.gov Not interviewed Mark Maessick, Program Officer USAID Kenya mmaessick@usaid.gov USAID U.S. Embassy Complex United Nations Avenue Gigiri P.O. Box 629, Village Market 00621 Nairobi, Kenya Not interviewed McDonald Homer, Director, Africa Bureau USAID USA Mhomer@usaid.gov 1300 Pennsylvania Ave NW, Washington, DC 20004 05/30/2014 Mervyn Farroe, Supervisory Program Officer USAID Kenya mfarroe@usaid.gov USAID U.S. Embassy Complex United Nations Avenue Gigiri P.O. Box 629, Village Market 00621 Nairobi, Kenya 01/24/2014 Michael Jones, Deputy Director, Agriculture, Business and Environment USAID Kenya Mjones@usaid.gov USAID U.S. Embassy Complex United Nations Avenue Gigiri P.O. Box 629, Village Market 00621 Nairobi, Kenya Not interviewed Michael Warui, GDA Specialist USAID East Africa Mkibinge@usaid.gov Not interviewed Nino Nadiradze, Office of Economic Growth USAID Ghana nnadiradze@usaid.gov USAID No. 24 Fourth Circular Road, Cantonments P.O. Boc 1630; Accra, Ghana Not interviewed Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 159 Respondent(s) Organization Country Contact Information (if available) Date Interviewed Roland Oroh, Agri Business Project Manager USAID - EGAT Nigeria roroh@usaid.gov USAID, Murjanatu House I Zambezi Crescent, Maitama PMB 519, Garki, Abuja 02/04/2014 Sharon Pauling, Director, Economic Growth (current Acting MD) Economic Growth & Environment Office, USAID/Nigeria Nigeria spauling@usaid.gov USAID, Murjanatu House I Zambezi Crescent, Maitama PMB 519, Garki, Abuja 12/13/2013 Toks Aluko, ICT Expert Economic Growth & Environment Office, USAID/Nigeria Nigeria taluko@usaid.gov USAID, Murjanatu House I Zambezi Crescent, Maitama PMB 519, Garki, Abuja 12/13/2013 International Organizations Kas Kalba, President Kalba International, Inc. National Telecommunications USA kalba@comcast.net 116 McKinley Avenue New Haven, CT 06515 USA Not interviewed Peter Silarszky, Senior Economist World Bank GICT USA psilarszky@worldbank.org Not interviewed Russell Southwood, President Balancing Act UK info@balancingact-africa.com 54 Walnut Tree Walk London SE11 6DN UK Not interviewed Tim Kelly, Lead ICT Policy Specialist World Bank GICT USA tkelly@worldbank.org Not interviewed Government Officials Abdullahi Maikano, Secretary Nigerian Communications Commission Nigeria maikano@ncc.gov.ng Plot 423, Aguiyi Ironsi Street, Maitama District, Abuja 02/04/2014 Abi Jagun, Special Assistant (Post & Telecom.) Federal Ministry of Communications Technology Nigeria Abiodun.jagun@commtech.gov.ng P.M.B. 12578 Federal Secretariat Complex Phase I, Annex III, Shehu Shagari Way Abuja FCT Not interviewed Alfred Boyan, Director, Finance & Administration GIFEC, Ministry of Communications Ghana Alfred.boyan@gofec.gov.gh P O Box CT 5625, Cantonments, Accra. 12/19/2013 Bitange Ndemo, Permanent Secretary, Ministry of Communication and Technology Ministry of Communications (now University Of Nairobi - Business School) Kenya bitange@jambo.co.ke Telposta Towers, Kenyatta Ave. Koinange Street P.O Box 30025-00100, Nairobi Kenya 01/21/2014 Bwalya Mwango, Manager, Projects, Universal Access Zambia ICT Authority Zambia bmwango@zicta.zom Not interviewed Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 160 Respondent(s) Organization Country Contact Information (if available) Date Interviewed Catherine Gitau, Head of e-Government ICT Authority Kenya +254 202 2211 960 +254 722 376 659 Not interviewed Excellence Favor, Board Member National IT Authority Uganda Excellencefavor@yahoo.co.uk Not interviewed Francis Limo Kollum, Board Member Communications Commission Of Kenya Kenya fkollum@bictoengineering.com fkollum@gmail.com + 254 (722) 524142 Not interviewed Francis Monlose Communications Commission of Kenya (CCK) Kenya No contact information available Not interviewed Francis Wangusi, Director General Communications Commission Of Kenya Kenya wangusi@cck.go.ke +254-20-4242000 CCK Centre, Waiyaki Way, P.O. Box 14448, Nairobi 00800, Kenya 01/22/2014 Hamad Masoud Hamad, Minister Ministry of Infrastructure & Communications, Rev. Gov’t of Zanzibar Zanzibar hamasoud@yahoo.com Not interviewed Haruna Idrissa, Minister Ministry of Communications Ghana harunaanugs@yahoo.co.uk Not interviewed Hon. Haruna Iddrisu, Minister Ministry of Trade & Industry Ghana yawmensah.sym@gmail.com harunanugs@yahoo.co.uk Not interviewed Hon. Samuel Losuron Poghisio, Chairman Of The Kenya Civic Aviation Authority Minister Of Communication And Technology Kenya Info@kcaa.or.ke Not interviewed Issah Yahaya, Director Policy Planning, Monitoring & Evaluation, Ministry of Communications Ghana info@moc.gov.gh 233-302-666465 P.O. BOX M38 Accra, Ghana 12/20/2013 James Kilaba, Deputy Director, ICTs Dvlp’t Tanzania Communications Regulatory Authority Tanzania kilaba@tcra.go.tz +255-22-24-12-011 Mawasiliano Towers Sam Nujoma Road Plot No. 2005/5/1C P.O. box 474 Dar es Salaam, Tanzania 01/20/2014 Joel Imitara, Assist Director, Competition, Tariffs & Analysis Communications Commission of Kenya Kenya imitara@cck.go.ke Not interviewed Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 161 Respondent(s) Organization Country Contact Information (if available) Date Interviewed Kelechi Nwankwo, Head of Strategy & Corporate Performance Monitoring Universal Service Provision Fund Nigeria knwankwo@ncc.gov.ng 12/12/2013 Kofi Attor, Administrator/CEO GIFEC, Ministry of Communications Ghana Koffiattor82@yahoo.com P O Box CT 5625, Cantonments, Accra 12/19/2013 Kwaku Ofosu Adarkwa, Chief Director Ministry of Communications Ghana kwaku.ofosu-adarkwa@moc.gov.gh; ofosuadarkwa@yahoo.co.uk Not interviewed Lasantha De Alwis, Corporate Secretary Commonwealth Telecommunications Organisation United Kingdom l.dealwis@cto.int Not interviewed Louis Moahlodi, Chairman: Board of Directors Universal Services & Access Agency of South Africa South Africa louismoahlodi@yahoo.com Not interviewed Makame Mbarawa, Minister Ministry of Communication, Science and Technology Tanzania makame.mbarawa@gmail.com or makame.mbarawa@mst.go.tz +255-22-211-6531 Jamhuri Street P.O. Box 2645 Dar es Salaam, Tanzania 01/15/2014 Matano Ndaro, Director, Competition, Tariffs & Analysis Communications Commission of Kenya Kenya ndaro@cck.go.ke CCK Centre, Waiyaki Way, P.O. Box 14448, Nairobi 00800, Kenya Not interviewed Nader Khalil, Executive Manager, Universal Service Project National Telecom Regulatory Authority Egypt nhusseiny@tra.gov.eg Not interviewed Omobola Johnson, Minister of Communications & Technology Federal Ministry Of Communication Technology Nigeria Personal Assistant to Minister: Joy.Udusegbe@commtech.gov.ng P.M.B. 12578 Federal Secretariat Complex Phase I, Annex III, Shehu Shagari Way Abuja FCT Email:roundtable@commtech.gov.ng 02/04/2014 Paarock Asuman Vanpercy, Director General National Communication Authority (NCA) Ghana nca@ghana.com Not interviewed Paul Mendy, Senior Finance Manager Public utility Regulatory Commission (PURA) Gambia psm@pura.gm Not interviewed Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 162 Respondent(s) Organization Country Contact Information (if available) Date Interviewed Peter Ulanga, Chief Executive Universal Communications Service Access Fund Tanzania ulanga@ucaf.go.tz +255-222-700000 2 nd Floor TTCL Kijitonyama Building Ali Hassan Mwinyi Road, Dar es Salaam PO Box 33114 01/17/2014 Phillip Prempeh, Strategic Planning GIFEC, Ministry of Communications Ghana Prempeh.phillip@gmail.com P O Box CT 5625, Cantonments, Accra. 12/19/2013 Raymond Akwule, President Digital Bridge Institute, Nigerian Communications Commission Nigeria rakwule@yahoo.com 8 P.O.W. Mafemi Crescent, Off Solomon Lar Way, Utako District – Abuja P.M.B. 5168, Wuse Post Office 02/05/2014 Robert Weller, Chief, Technical Analysis Federal Communications Commission USA Robert.weller@fcc.gov Not interviewed Seyi Onabanjo, Special Assistant for Local Content Federal Ministry if Communications Technology Nigeria Seyi.onabanjo@commtech.gov.ng Not interviewed Susan Mochache, Assistant Director, Universal Service Obligation Communications Commission of Kenya Kenya smochache@cck.go.ke +254-20-4242213 CCK Centre, Waiyaki Way, P.O. Box 14448, Nairobi 00800, Kenya 01/22/2014 Tope Fashedemi, Director, e-government Federal Ministry of Communications Technology Nigeria Tope.fashedemi@commtech.gov.ng Not interviewed Victor Kyalo, Deputy CEO/TCIP Programme Manager ICT Authority Kenya vkyalo@ict.go.ke Not interviewed Vusi Ngcobo, Member: Board of Directors Universal Services & Access Agency of South Africa South Africa Vusi.ngcobo@yum.com Not interviewed William Tevie, Director General National Information Technology Agency (NITA) Ghana info@nita.gov.gh 12/20/2013 Yahya Zakaria Osman, Technical Director GIFEC, Ministry of Communications Ghana yosman@gifec.gh P O Box CT 5625, Cantonments, Accra. Not interviewed Bashir Kamara, Project Manager Commission – Universal Access Development Fund Sierra Leone Princebash2@yahoo.com Not interviewed Public & Private Counterparts David Townsend, Consultant Integra USA DNT@dntownsend.com 17 Lawrence Road Swampscott, MA 01907 05/13/2014 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 163 Respondent(s) Organization Country Contact Information (if available) Date Interviewed Eric White, Managing Associate Integra USA ewhite@integrallc.com 1030 15th St NW Washington D.C. 03/21/2014 Parvez Iftikhar, Consultant Integra USA pi@piftikhar.com parvez@piftikhar.com t: +92 51 2291108 c: +92 300 855 5200 piftikhar@hotmail.com 04/10/2014 William B. Garrison, Former Senior DCOP & Project Support for Task Order 1 NetHope (Formerly) USA wbgarrison@gmail.com Tel: 703-647-9459 Address not obtained 05/28/2014 Private and Public Organizations, Others Moctar Yedaly, Head of Telecoms & Post Division, Dept. of Infrastructure (Head of ICT) African Union Commission Ethiopia yedalym@africa-union.org yedaly@africa-union.org moctaryedaly@yahoo.com Not interviewed Ferruh Gurtas, Corporate Affairs Director, ME Intel Turkey, Africa Ferruh.gurtas@intel.com Not interviewed Jane Wertz, Head of Sales & Marketing, East & Central Africa Helios Towers Tanzania jwertz@heliostowersafrica.com Not interviewed John Roman, Senior Manager, Global Spectrum Assessment & Policy Intel USA John.m.roman@intel.com Not interviewed Jussi Hinkkanen, Vice President, Gov’t Relations, Africa Nokia South Africa Jussi.hinkkanen@nokia.com Not interviewed Kristiina Lähde, Chief Technical Advisor TANZICT Tanzania kristiina@tanzict.or.tz 0756 487306 +255-75-648-7306 DTBI/3rd floor, COSTECH-Building PO Box 4302 Ali Hassan Mwinyi Road Kijitonyama, Dar es Salaam, Tanzania 01/16/2014 Michele Castegnaro, Sales Director, Strategic Industries Alcatel Lucent Kenya Michele.castegnaro@alcatel￾lucent.com Not interviewed Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 164 Respondent(s) Organization Country Contact Information (if available) Date Interviewed Mihayo Wilmore, Managing Partner UhuruOne Tanzania Mihayo.wilmore@uhuruone.com +255-222-12-4713 Suite 503, 5th Floor Harbour View Towers Samora Avenue, PO Box 20656 Dar es Salaam, Tanzania 01/15/2014 Muriuki Mureithi, CEO Summit Strategies Summit Strategies, Ltd. Kenya mureithi@summitstrategies.co.ke +254-20-3673925 Eden Square Complex 7th Floor Chiromo Rd Westlands, Nairobi 01/21/2014 Nicholas Nesbitt, Chief Executive Officer KenCall (now Country General Manager, IBM) Kenya mnesbitt@kencall.com +254-20-283-4000 +254-721-458-458 The Atrium, 4th Floor, Chaka Road PO Box 35475 00200 Nairobi, Kenya 01/23/2014 Alf Monoghan, Chief of Party NEXTT Project, CARANA Corporation Nigeria amonaghan@nigerianextt.org +234-0-708-871-5948 9A Lungi Street, Wuse II, Abuja, Nigeria 12/18/2013 Gordon Ononiwu, Director, External Relations & Capacity Building MARKETS II Project, IFDC Nigeria gononiwu@nigeriamarkets.org 23 Ona Crescent, Off Lake Chad Crescent, Maitama, Abuja, Nigeria 12/18/2013 Scott Wallace, Country Representative IFDC/AFAP Nigeria swallace@ifdc.org +234-0-806-594-3178 No. 6/Plot 141 Ogbagi Street, Off Oro-Ago Crescent, Cadastral Zone II, Garki, Abuja, Nigeria 12/17/2013 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 165 SUB-ANNEX 2. LIST OF KEY QUESTIONS USAF Evaluation: Key Questions i. To what extent, and how did the project shape the legal and regulatory environment for a stable, transparent, and resilient USAF in Core Countries?  What legal-regulatory changes have occurred during the project in the areas targeted by USAF?  Do the changes contribute to a stable, transparent, and resilient telecommunications sector?  What activities did USAF carry out that have a plausible causal connection to those legal￾regulatory changes?  How did those activities contribute to the changes, e.g. by means of drafting assistance, technical analysis, or capacity-building activities?  How did the relevant officials come to value or solicit the inputs from USAF?  What other influences could reasonably account for those legal-regulatory changes?  Were the changes reversed? If not, will they be sustained? ii. To what extent, and how did the project affect the capacity of the Core Countries telecommunications institutions to implement reforms and raise their performance to international best practices?  Has the capacity of the telecommunications sector improved? In what ways – e.g. better-trained personnel, more developed infrastructure, greater efficiency, improved access to and sharing of information?  Do the changes contribute to raising the sector’s performance to the level of international best practice?  What activities did USAF carry out that have a plausible causal connection to those improvements in capacity?  How did those activities contribute to the changes, e.g. by means of training, information, technical assistance, provision of equipment or software?  How did the relevant officials come to value or solicit the inputs from USAF?  What other influences could reasonably account for those changes in capacity?  Were the changes reversed? If not, will they be sustained? iii. To what extent, and how relevant were the project activities designed to assist vulnerable Core Countries with telecommunications awareness, and increase demand for provision of adequate customer￾oriented telecommunications services and proper government supervision? How was gender addressed in these interventions?  Did Core Countries’ telecommunications system respond effectively to the stresses on their vulnerable populations resulting from the global crisis?  In what ways did these responses (a) provide access to telecommunications to the vulnerable, (b) increase public awareness and understanding of telecommunications universal service funds, and (c) increase the demand for and supply of telecommunications services that have appropriate consumer safeguards and supervision, and are relevant to the needs of vulnerable Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 166 Core Countries?  Which of the USAF components were actually relevant to Core Countries’ needs?  What activities did Integra carry out that have a plausible causal connection to the telecommunications universal service funds response to the needs with respect to vulnerable populations?  How did those activities contribute, e.g. by means of training, information, public outreach, technical assistance?  What other influences could reasonably account for the quality of the telecommunications universal service funds response?  How did the relevant partners and counterparts come to value or solicit the inputs from USAF?  Was gender addressed in the telecommunications system’s response with respect to vulnerable populations? Was it addressed in other areas of policy addressed by USAF, e.g., capacity￾building, deepening telecommunications services? How was gender addressed, e.g. building gender equity into management, staffing, training and technical assistance activities – and disaggregating statistics by gender? Did USAF itself contribute in this area, and if so, how? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 167 SUB-ANNEX 3. KEY INFORMANT INTERVIEW GUIDES DRAFT INTERVIEW GUIDE – USAF PROJECT FOR USE IN TELEPHONE INTERVIEWS OF SSA PARTICIPANTS ID INFORMATION: NAME: ____________________________________________________________________ POSITION: _________________________________________________________________ DATE OF INTERVIEW: ______________________________________________________ INTERVIEWER: _____________________________________________________________ INTRODUCTION: THANK YOU FOR TAKING TIME TO COMPLETE THIS TELEPHONE SURVEY. THE PURPOSE OF THIS SURVEY IS TO GET YOUR FEEDBACK ABOUT A USAID UNIVERSAL ACCESS FUND PROJECT IN SUB-SAHARAN AFRICA COUNTRIES. USAID FUNDED THE UNIVERSAL SERVICE ACCESS FUND (USAF) PROJECT TO SUPPORT SUB-SAHARAN AFRICAN (SSA) COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. INTEGRA IS THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM. USAID HAS HIRED MY COMPANY, DTS, TO CONDUCT A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. THEREFORE, WE ARE TRYING TO GATHER AS MUCH INFORMATION AS POSSIBLE REGARDING INTEGRA’S EFFORTS AND ACTIVITIES. PLEASE ASSIST US IN THIS EFFORT BY RESPONDING TO THE FOLLOWING QUESTIONS. QUESTIONS: 1. PLEASE TELL ME HOW YOU WERE INVOLVED IN THE USAF PROJECT ACTIVITIES IN SSA. IN-COUNTRY ASSISTANCE FROM INTEGRA________ ATTENDANCE AT A REGIONAL WORKSHOP OR SEMINAR___________ (IF SO, CAN YOU IDENTIFY THE APPROXIMATE DATE, LOCATION, AND FOCUS OF THE WORKSHOP? DATE__________________, LOCATION______________________, FOCUS__________________ Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 168 ATTENDED ONLINE WEBINAR_____________________________ IF SO, CAN YOU IDENTIFY THE SUBJECT OF THE WEBINAR?___________________________________________________________ OTHER – PLEASE BRIEFLY EXPLAIN___________________________________ 2. HOW DID YOU LEARN ABOUT THE USAF PROJECT?_____________________________________________________________________ _____________________________________________________________________________ _______ 3. WHAT DID YOU FIND TO BE THE MOST VALUABLE SUBJECT OR ACTIVITY THAT YOU WERE EXPOSED TO?_________________________________________________________________________ _____________________________________________________________________________ _________ 4. HOW HAVE YOU BEEN ABLE TO APPLY WHAT YOU LEARNED OR WERE EXPOSED TO IN THE USAF ACTIVITIES?___________________________________________________________________ _____________________________________________________________________________ ________ 5. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE INTEGRA TRAINING? _____________________________________________________________________________ _____________________________________________________________________________ __________ 6. WERE THE TOPICS RELEVANT COVERED TO YOUR AREA OF RESPONSIBILITY OR EXPERTISE?_________________ 7. WAS THERE INFORMATION OR TRAINING THAT YOU RECEIVED WHICH YOU HAVE NOT YET USED BUT PLAN TO USE IN THE FUTURE? IF SO, PLEASE BRIEFLY EXPLAIN______________________________________________________________________ _____________________________________________________________________________ ________ 8. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE USAF PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR TRAINING OBJECTIVES?__________________________________________________________________ _____________________________________________________________________________ ________ 9. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER FOR USAID CONSIDERATION IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING?__________________________________________________________________ _____________________________________________________________________________ Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 169 _____________________________________________________________________________ _____________ 10. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT?_____________________________________________________________________ _____________________________________________________________________________ _____________________________________ Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 170 SUB-ANNEX 4. DRAFT EVALUATION REPORT DESIGN Executive Summary I. Introduction– Purpose and Scope of Report II. Evaluation Purpose, Methodology, and Limitations III. Project Description and Context IV. Project Impacts A. USAF Usage B. Telecom access C. Accelerate telecom access D. Telecom take-up E. USAF capacity V. Cost-effectiveness of assistance approaches VI. Continuing Needs and Opportunities VI. Major Conclusions and Lessons Overall VII. Recommendations ANNEXES: (a) Country Reports; (b) Detailed Evaluation methodology & limitations; (c) Team qualifications; (d) Contacts Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 171 ANNEX 8. KEY INFORMANT INTERVIEW GUIDES INTERVIEW GUIDE – USAF PROJECT USAID PERSONNEL ID INFORMATION: NAME: POSITIION: ORGANIZATION USAID DATE OF INTERVIEW: INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I NEED TO INTERVIEW YOU. AS YOU KNOW, USAID HIRED DTS TO CONDUCT AN EVALUATION OF THE PERFORMANCE OF INTEGRA IN ASSISTING SSA COUNTRIES IN MOVING FORWARD IN UNIVERSAL ACCESS TO BROADBAND AND ICT SERVICES. TO BE AS THOROUGH AS POSSIBLE IN THE EVALUATION EFFORT, WE ARE INTERVIEWING AS MANY PEOPLE AS POSSIBLE WHO WERE INVOLVED IN THE PROJECT. SINCE OUR EVALUATION IS LIMITED TO INTEGRA’S WORK UNDER TASK ORDER 1 OF THE GBI PROJECT, WE ARE FOCUSING ON THE FOUR COUNTRIES WHICH INTEGRA SELECTED FOR DIRECT TECHNICAL ASSISTANCE. THE COUNTRIES WERE: KENYA, NIGERIA, GHANA AND TANZANIA. QUESTIONS: 1. WHAT WAS YOUR SPECIFIC ROLE IN THE USAF PROJECT (TASK ORDER 1)? 2. WHAT WERE THE SPECIFIC OBJECTIVES OF THE PROJECT? 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN EACH OF THE COUNTRIES? 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN EACH COUNTRY? 5. IF INTEGRA WAS NOT ABLE TO MEET THEIR OBJECTIVES OR USAID’S OBJECTIVES, WHY DO YOU BELIEVE THIS HAPPENED? WHAT COULD INTEGRA HAVE DONE TO OVERCOME THIS SITUATION? 6. CAN YOU TELL ME, IN THE FOUR COUNTRIES, WHAT ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 172 D. USAID E. OTHER 7. ARE YOU AWARE OF ANY SPECIFIC OBSTACLES ENCOUNTERED BY INTEGRA IN ANY OF THE FOUR COUNTRIES – OR OTHERWISE DURING THE TASK ORDER 1 TIME FRAME? 8. ARE THERE ANY OTHER COMMENTS OR THOUGHTS RELATING TO THE TASK ORDER 1 USAF PROJECT THAT YOU WOULD LIKE TO MENTION? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 173 INTERVIEW GUIDE – USAF PROJECT INTEGRA PERSONNEL ID INFORMATION: NAME: POSITIION: ORGANIZATION INTEGRA DATE OF INTERVIEW: INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIE IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA OR KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT IN EACH OF THE FOUR COUNTRIES – KENYA, TANZANIA, NIGERIA AND GHANA. 2. WHAT WERE THE SPECIFIC OBJECTIVES IN EACH COUNTRY? 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN EACH OF THE COUNTRIES? 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN EACH COUNTRY? 5. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 174 D. USAID E. OTHER 6. CAN YOU ELABORATE ABOUT THE OBSTACLES THAT YOU ENCOUNTERED IN EACH COUNTRY? AND, WHAT DID YOU OR OTHERS ON THE INTEGRA TEAM DO TO OVERCOME THOSE OBSTACLES? 5. IS THERE ANYTHING THAT YOU WOULD WANT TO RECOMMEND THAT USAID CONSIDER DOING DIFFERENTLY IN DESIGNING AND IMPLEMENTING PROGRAMS SUCH AS THE TASK ORDER 1 EFFORT? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 175 INTERVIEW GUIDE – USAF PROJECT NETHOPE PERSONNEL ID INFORMATION: NAME: POSITION: ORGANIZATION: DATE OF INTERVIEW: INTERVIEWER: W. KENT EDWARDS THANK YOU FOR TAKING THE TIME TO TALK WITH ME. MY FIRM, DTS, IS DOING AN EVALUATION FOR USAID OF THE SUB-SAHARAN AFRICA USAF COMPONENT OF THE GLOBAL BROADBAND AND INNOVATION (GBI) INITIATIVE. OUR EVALUATION IS ONLY FOCUSED ON THE USAF PROJECT; ALSO KNOWN AS “TASK ORDER 1.” AND, SINCE WE UNDERSTAND THAT NETHOPE WAS INVOLVED IN ASPECTS OF THIS PROJECT, WE NEED TO INTERVIEW YOU. QUESTIONS: 1. PLEASE DESCRIBE NETHOPE’S ROLE OR INVOLVEMENT IN THE USAID USAF GBI PROJECT IN SUB-SAHARAN AFRICA. 2. CAN YOU TELL ME WHEN THE PROJECT (OR FUNDING) STARTED AND ENDED? 3. (IF APPROPRIATE) CAN YOU EXPLAIN THE REASON FOR OR CAUSE OF THE DELAY IN FUNDS GETTING RELEASED TO THE PROJECT? 4. (IF APPROPRIATE) DID NETHOPE PLAY ANY ROLE IN THE USAF PROJECT OTHER THAN AS A CONDUIT FOR FUNDING? IF SO, PLEASE ELABORATE ON THE NETHOPE ROLE. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 176 5. IS THERE ANYTHING ELSE THAT YOU CAN THINK OF ABOUT THE USAF PROJECT THAT MIGHT BE PERTINENT TO OUR EVALUATION? THANKS FOR PROVIDING THIS INPUT. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 177 SITE VISIT INTERVIEW GUIDE – USAF PROJECT ID INFORMATION: NAME: _________________________________________________________________ POSITIION: _________________________________________________________________ ORGANIZATION_______________________________________________________________ __ DATE OF INTERVIEW: __________________________________________________________ INTERVIEWER: __________________________________________________________ INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIE IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA OR KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OFANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN _______________________________________________. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 178 B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 179 Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 180 ANNEX 9. KEY INFORMANT INTERVIEW WRITE-UPS NIGERIA SITE VISIT INTERVIEW– NIGERIA - USAF PROJECT ID INFORMATION: NAME: ABDULLAHI MAIKANO POSITIION: SECRETARY ORGANIZATION USPF DATE OF INTERVIEW: 4 FEBRUARY 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. NIGERIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: Mr. Maikano was joined for part of the interview by Engr. Oluwatoyin Yusuf Asaju, who is currently the Head of Connectivity and was previously Head of Infrastructure at USPF. 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN NIGERIA. When GBI (Integra) came, Mr. Maikano said that they first met with the Minister, who then said to get GBI fully involved in the strategy. Most of the time spent with Integra was with David Townsend and Parvez Iftikhar. Engr. Asaju was new at that time. At some point, an MOU was signed with Integra (not clear if the MOU was signed prior to or subsequent to the KPMG workshop). GBI spent a week at the KPMG retreat, where USPF was working on designing a Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 181 new five-year strategic plan. USPF decided to take a year to evaluate the situation and develop the new plan for 2013 through 2017. Prior to that retreat, he had travelled to India, Malaysia and Uganda to gain more in-depth understanding of their approaches. USPF looked at previous projects by hiring other outside consultants to help assess the impact of such previous projects as input for the strategy deliberations. He said that they learned that the old strategy was good, but that some objectives were too broad, and that the strategy was limited to the goal of providing telephony to every household, i.e., was outdated, as technology had changed. Also, there had been no effective measurement and evaluation (M&E) of the projects to determine if they were really achieving the objectives. With GBI’s assistance, they did another workshop in Abuja to focus on the M&E needs. David Townsend attended, and emphasized that they should not wait until the end of a project to start the M&E activities. Many of the lessons learned were then incorporated into the new strategic plan, which was more compact and concise than the old plan. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? See responses to Question 1, above. A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? Engr. Asaju stated that the success of the strategic plan was due to the assistance provided by David Townsend and Parvez Iftikhar. Also, GBI provided assistance in designing USPF projects. Parvez gave them examples from his “on-the-ground experiences. They then decided to create “clusters” of ICT gaps by geographic areas, recognizing that everywhere is not the same as far as digital gaps. This enabled them to better target possible project activities. Mr. Maikano stated that, under the old strategic plan, they supported 54 base stations over a five￾year period. Under the new plan, they have implemented twenty (20) in one year, and that any project will be implemented on an open, transparent basis with full stakeholder involvement. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. The USPF organization and the Ministry were involved. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 182 As a result of the assistance from GBI, they opened talks with the National University Commission, and agreed that there was a need to involve other stakeholders, such as schools and the communities. A World Bank supported project was mentioned which is creating a broadband backbone connecting all universities in Nigeria (the Nigeria Research and Education Network, known as NGREN). Initially, this network is for 28 public universities where the main campuses are on the broadband ring. Each university has branches, hospitals, medical colleges, etc. Other public and private organizations will be brought on-line to foster research. The goal is to share knowledge, which is a lesson learned as a result of the GBI capacity building. Sixteen branches of universities are currently being brought onto the rings, where the access will include video conferencing and one smart classroom in the medical colleges. Another example is the design of the cluster creation project where they established the clusters based on voice and broadband gaps. They have had discussions with the Minister of Agriculture about the Growth Enhance Scheme, which has had E-voucher problems. Via gap analysis, they determined that farmers in rural areas have little cellular service available. Thus, they decided to identify specific locations where subsidies could be provided for building cell stations. Via this approach, they then invite operators to bid in each identified cluster. Awards are to be based on the following: (1) level of services offered = 40%, with scores ranging from 10% for voice only up to 40% for broadband 4G, (2) power, with scores ranging from 10% if the public grid is the sole source up to 40% for a hybrid approach, and (3) level of desired subsidy by the bidder = 20%. It requires a minimum score of 60% for the project to move forward. The approach envisions that OPEX will be subsidized for three years maximum, and the CAPEX subsidy is limited to one time. 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? Mr. Maikano said that there had been no obstacles of which he is aware. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? Mr. Maikano cited the B-Train program where broadband transmission infrastructure via fibre backhaul will support telephony and LTE. He said that the B-Train effort will extend fibre to the unserved clusters, and that USPF will subsidize extension to the clusters. He now realizes that it will take at least eighteen (18) months to accomplish, and that 60 to 70% of the effort will be civil work. If GBI (Integra) is to be involved in a project like B-Train, where infrastructure rollout is involved, USAID would need to ensure a commitment of a minimum of 18 months, considering deployment issues such as right-of-way acquisition. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? This question was not asked. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 183 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? This question was not asked. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 184 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? This question was not asked. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 185 SITE VISIT INTERVIEW– NIGERIA – USAF PROJECT ID INFORMATION: NAME: ALEFIA MERCHANT & JAMES LYKOS POSITIION: MERCHANT = AGRICULTURE DEVELOPMENT OFFICER; NOT SURE ON LYKOS (ONLY AT POST TWO WEEKS) DATE OF INTERVIEW: 16 December 2013 INTERVIEWER: Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. NIGERIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT OR KNOWLEDGE OF THE USAF PROJECT ACTIVITIES IN NIGERIA. No direct involvement. Roland Oroh also said that he had no direct involvement, per A. Merchant. These two people were asked to meet with me due to Roland apparently not feeling well. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO: A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES No B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? No 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN NIGERIA? Unaware. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT: Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 186 No, to all of the following. A. USAID FUNCTIONS HERE IN NIGERIA B. OTHER USG ORGANIZATIONS C. NGO’S D. NIGERIAN AGENCIES, MINISTRIES, ETC. E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? Unaware 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER FOR USAID-W CONSIDERATION IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? No input 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? Given the complete lack of knowledge about the program being evaluated, this question was not asked. 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? No. However, Alefia would like to obtain the cost expended by USAID in Nigeria and the results of the project. James said that he would like to receive the final report numbers, and asked if the numbers contained in the brief provided by Patricia Flanagan are current. I indicated that the numbers in the brief are current. 9. ARE THERE ANY OTHER PEOPLE IN NIGERIA YOU WOULD RECOMMEND I INTERVIEW IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT FOR US? No. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. NOTE: THIS INTERVIEW WAS NOT USEFUL FOR USAID PEOPLE AND FOR THE EVALUATION TEAM. AS A RESULT, I CUT THE INTERVIEW OFF SHORT IN ORDER TO RESPECT TIME OF OTHERS. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 187 SITE VISIT INTERVIEW– NIGERIA – USAF PROJECT ID INFORMATION: NAME: ALF MONOGHAN POSITIION: COP – NEXTT PROJECT (NIGERIA EXPANDED TRADE AND TRANSPORT) DATE OF INTERVIEW: 18 December 2013 INTERVIEWER: Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. NIGERIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. THE USAID MISSION SUGGESTED THAT I CONTACT YOU TO LEARN ABOUT THE NEXTT PROJECT AND EXPLORE THE RELATIONSHIP WITH THE USAF INTEGRA PROJECT. Mr. Monoghan has been the COP for about four months. NEXTT is a USAID-funded project for which CARANA is the contractor. To brief me on the NEXTT Project, Mr. Monoghan used what appeared to be the standard NEXTT explanatory presentation, which he kindly later forwarded to me. This interview write-up will focus on points made during the presentation, some of which may be redundant to information contained in the slides of the presentation by Mr. Monoghan. The logistics of the project is to improve commerce along the LAKAJI Corridor, which is 1,150 Klm in distance (North to South – on the Eastern side of the country). The basic problem is congestion and blockages all the way along the existing road. The Nigerian government wants to move Agriculture to a commercial business posture, which – if successful – will only serve to increase the congestion on the roadway. Thus, the NEXTT project is focused on providing alternatives to improve the situation. What is envisioned is a multimodal corridor, which would include: improved roadway; moving the rail system from narrow gauge to standard gauge (which can handle much heavier loads); improved power (which has just been privatized); freight only airports in the Northern area; ICT – at least along the roadway and nodals; state junctions – where non-contiguous states would have storage facilities; and making the entire corridor a preferential zone in order to attract investment. Legislation is in process to create a PPP for the rail system, and would include Arik, AP Moller, Maersk (SP?) the shipping/container company, at around $3.5 billion. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 188 The goal of NEXTT is to have a single linear entity. There are eight jurisdictions along the corridor, so that it will be necessary to create a single entity to manage the corridor. The objective is to change the Nigerian “Brand” world-wide to make it an attractive brand, i.e., to change the country’s image abroad. Also, there is a large domestic and regional market to better serve. ICT involvement – there is a need for improving the use of the road, and Eric White has been working on this. NEXTT is proposing an advisory council. Structure would be: CARANA focus on agricultural/industrial aspects.  Integra – ICT  Crown Agents – Customs  Global Cold Chain Alliance  RBS – a Lagos company with local presence and very connected.  DA Associates – a Nigerian business development provider Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 189 SITE VISIT INTERVIEW– NIGERIA – USAF PROJECT ID INFORMATION: NAME: GORDON ONONIWU POSITIION: Director, External Relations & Capacity Building, Markets Project DATE OF INTERVIEW: 18 December 2013 INTERVIEWER: Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. NIGERIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. THE USAID MISSION SUGGESTED THAT I MEET WITH YOUR ORGANIZATION TO LEARN MORE ABOUT THE MARKETS PROJECT. NOTE: Since there was no involvement by the Markets Project with any of Integra’s activities in Nigeria, the interview guide was not used. Instead, the following description of the Markets Project was provided. Mr. Ononiwu explained that MARKETS II is the third iteration; that Markets I went from 2005-2010, and then there was a bridge project from 2011 – 2012, followed by MARKETS II, which runs 23012 – 2017. All were and are USAID-funded projects. The aim of the current project is to transform agriculture in selected sites from a sustenance operation to a commercial farming endeavor. The average farm in Nigeria is less than 2 hectares, and in some cases, as small as .5 hectare. The goal is to maximize revenues for the following commodities:  Rice  Sorghum  Cassava  Coco  Soya  Maize  Aquaculture The project follows from where the product is grown to where it is processed, and trains farmers in improved processes (capacity building) via cooperatives and linking to the processors. OCAM and other Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 190 companies provide seeds. The project has trained out growers in production. Capacity building and technical support is ¾ of the work – working on how to run groups and microenterprise operations. One objective is to make groups sustainable. A US company “Making Cents” is working with MARKETS II to develop training programs on the areas noted above. The project also seeks to target youths and other vulnerable groups. Guidelines are: as an example, for a group of 100 people, 50% must be female, 30% must be youth. An example of youth specialized training is to train a young person in handling chemicals, then equipping the person with some gear and sending him/her back to home area to start up a spraying business for farms. The idea is to improve economic opportunity and keep some young people on the farms. The project has encouraged millers to come on board. An example here is that a former minister took on Rice and now is getting ready to open the second mill. The project encourages millers to form a group to explore: what are the common challenges – and then the MARKETS project supplies a small amount of funding to help solve common problems. As a result, the millers now have a strong presence in the country and have presented challenges, such as the smuggling issue, to the ministry. The project did a Rice study, which was provided to the millers, who found it useful. Also, MARKETS II is working with the government more, and has gotten closer. It now attends a weekly meeting and learns about challenges, especially about Rice issues. Africa Rice (part of an international agency) is working closely with MARKETS II. I asked about the role of ICTs. He said Redemption. There has been much talk about the use of ICT in agriculture extension work, but it is very slow in arriving. One of the problems is that there is a low level of interest on the part of the farmers, some of which is attributable to the lack of power in many areas. I asked Mr. Ononiwu if he was familiar with the USAF project, or if he had heard of it. He indicated that he had never heard of it. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 191 SITE VISIT INTERVIEW– NIGERIA - USAF PROJECT ID INFORMATION: NAME: THE HONORABLE MADAME OMOBOLA JOHNSON POSITIION: MINISTER OF COMMUNICATIONS & TECHNOLOGY - NIGERIA ORGANIZATION: FEDERAL MINISTRY OF COMMUNICATIONS & TECHNOLOGY - NIGERIA DATE OF INTERVIEW: 4 FEBRUARY 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. NIGERIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN NIGERIA. MET WITH INTEGRA TEAM; AGREED TO CAPACITY BUILDING ACTIVITIES; PARTICIPATED IN KPMG RETREAT WHERE INTEGRA PLAYED A SIGNIFICANT ROLE. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES SHE BECAME AWARE OF THE INTEGRA TEAM AND PROPOSED SUPPORT VIA AN INTRODUCTION BY PROFESSOR RAYMOND AKWULE. INTEGRA CONTACTED HER AND DISCUSSED POTENTIAL ACTIVITIES TO SUPPORT USAF VIA SHOWIING HER EXAMPLES OF WHAT HAD BEEN DONE IN OTHER AFRICAN COUNTRIES. SHE STATED THAT IT HAD BEEN QUITE A WHILE AGO, SO THAT PRECISE MEMORY ON SOME OF THE DETAILS WAS UNCLEAR. THE MINISTRY WAS, AT THE TIME, Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 192 UPDATING THE NATIONAL BROADBAND STRATEGY – AND INTEGRA ASSISTED IN THAT EFFORT. INTEGRA ALSO PLAYED A BIG ROLE IN THE RETREAT HELD FOR USPF, MANAGED BY KPMG. INTEGRA ALSO PROVIDED ASSISTANCE SUBSEQUENT TO THE RETREAT. B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? PRESUMABLY VIA THE EXAMPLES PROVIDED, SHE INDICATED THAT INTEGRA DID DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS. SINCE IT APPEARED THAT SHE WAS NOT FEELING WELL DUE TO A COLD OR SOMETHING SIMILAR, I DID NOT PROBE FURTHER ON THIS ITEM. 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? THE GREATEST IMPACT OF INTEGRA’S WORK IN NIGERIA WAS IN THEIR HELP IN MAKING THE NEW NATIONAL STRATEGIC PLAN FOR BROADBAND MORE REALISTIC, AND ALSO TO SHOW USPF THAT THEY DID NOT NEED TO ALWAYS RELY ON OURSOURCING TO ACCOMPLISH ACTIVITIES, I.E., THAT, WITH TRAINING AND GOOD FOLLOW-UP PRACTICES, THEY COULD DO VARIOUS PROJECTS INTERNALLY. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER THERE WAS AN IMPACT ON KPMG, WHICH HAD DONE THE FIRST NATIONAL STRATEGIC PLAN AND WAS WORKING ON THE NEW ONE. INTEGRA SUPPORTED KPMG AND THE MINISTRY IN THAT EFFORT. 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? SHE STATED THAT, NO, THERE WERE NO OBSTACLES FOR INTEGRA IN NIGERIA, AS THEY WANTED THE ASSISTANCE. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 193 IT WOULD BE HELPFUL TO A COUNTRY IF USAID COULD TAKE A LONGER TERM VIEW IN DESIGNING PROJECTS. SPECIFICALLY, IN THE CASE OF THE PROJECT WORK DONE BY INTEGRA IN NIGERIA, IT WOULD BE A GOOD IDEA FOR THEM (INTEGRA) TO COME BACK IN A YEAR OR TWO – DO AN ASSESSMENT OF PROGRESS, THEN ASSIST IN DETERMINING GAPS AND WHERE TO GO AT THAT TIME. BUILDING EXPERTISE IN THESE TYPES OF FUNCTIONS TAKES TIME, AND A COUNTRY COULD USE ASSISTANCE FROM EXPERTS WHO ARE TRULY FAMILIAR WITH THE PROBLEMS AND CHALLENGES. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? YES, SYNERGIES WITH VARIOUS PROGRAMS OUGHT TO BE TAKEN INTO ACCOUNT, EVEN IF IT IS JUST BY PROVIDING EXAMPLES OF SYNERGIES DISCOVERED IN OTHER COUNTRIES. 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? SHE STATED THAT SHE BELIEVES THAT USAID’S FOCUS ON BROADBAND IS A VERY GOOD IDEA. ALSO, FOR NIGERIA, THE QUALITY OF THE PEOPLE ON THE INTEGRA TEAM WAS VERY HELPFUL, I.E., HAVING PEOPLE WHO HAVE SIGNIFICANT EXPERIENCE IN THE FIELD IS IMPORTANT. SHE SAID THAT SHE WOULD VERY MUCH LIKE TO SEE INTEGRA COME BACK TO FURTHER ASSIST NIGERIA. 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? THIS QUESTION WAS NOT ASKED. NOTE: IT APPEARED THAT THE MINISTER DID NOT FEEL WELL, AS SHE APPEARED TO HAVE THE “SNIFFLES” AND CONGESTION. IN CONSIDERATION OF THAT, I PURPOSELY KEPT THE INTERVIEW SHORT. ADDITIONALLY, SHE APPARENTLY WAS NOT TOTALLY AWARE OF THE PURPOSE OF THE INTERVIEW, SO THAT SHE HAD TWO OTHER OF HER STAFF ATTEND THE MEETING. NEITHER OF THE OTHER PEOPLE HAD ANY INVOLVEMENT WITH THE INTEGRA PROJECT. ONE PERSON WAS SEJI ONANBANJO, SPECIAL ASSISTANT (LOCAL CONTENT) TO THE HON. MINISTER, AND THE OTHER WAS DR. TUNJI OLAOPA, THE NEW PERMANENT SECRETARY TO THE MINISTRY, WHO HAS BEEN ON THE JOB ONLY ABOUT ONE AND ONE￾HALF MONTHS. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 194 SITE VISIT INTERVIEW– NIGERIA – USAF PROJECT ID INFORMATION: NAME: KELECHI NWANKWO POSITIION: Head of Strategy & Corporate Performance Monitoring, USPF DATE OF INTERVIEW: 12 December 2013 INTERVIEWER: Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. NIGERIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT OR KNOWLEDGE OF THE USAF PROJECT ACTIVITIES IN NIGERIA. He attended the Turkey, Dar es Salaam and Zanzibar regional workshops. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO: A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? These two questions were not directly asked, but his discussion made it evident that he had been contacted by someone to attend the workshops and for other training. He indicated that the workshops validated what USPF was doing and also provided learning. For specifics on what was learned, he related that previous to the training, USPF would tender USAF projects on a subsidized basis with no geographic targets. Companies such as MTN would bid and get the contracts. There was no directed follow-up by USPF. After the training, USPF puts the targets into the public domain in geographic clusters, so that all vendors will know what is being tendered and the results expected. This avoids the situation where a vendor might decide to start service in an area where USPF has awarded a bid and will be subsidizing the facilities. USPF subsidizes only facilities at the moment, but is considering subsidizing end-users in some number (TBD) for a limited time. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 195 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN NIGERIA? After the strategy sessions by KPMG (in which Integra participated), the total focus of the program changed. For schools, efforts became more targeted to where they believe the student body is. As previously mentioned, the USPF tenders became more targeted. And, the community center program was improved. USPF designed a prototype model for centers – where USPF subsidizes the building and has a contractual agreement with a person or vendor to operate the center. The operator has “skin in the game,” and the arrangement is to last for one year. The vendor must report on number of people using the center, what uses are made, what businesses has the operator created, etc. He said that the program is not going very well at the moment due to the lack of familiarity with reporting on the part of the operators. He indicated that more monitoring and evaluation is needed and will be taking place. Included in the community center package is broadband access and PC training (but it was not made clear if the PC training was for the operator or for customers of the center). 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. USAID FUNCTIONS HERE IN NIGERIA B. OTHER USG ORGANIZATIONS C. NGO’S D. NIGERIAN AGENCIES, MINISTRIES, ETC. The Ministry of Communications Technology is now working with the Ministry of Agriculture on a national program for farmers. There is now a study by the World Bank on E-Health which may be integrated into tri-agency efforts. E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? He believes that the challenge to Integra and to the USPF is to study and determine how best to utilize ICTs for institutions. Also, part of the problem in implementing universal access programs is that many individuals have multiple phones, and this is increasing with the urban migration. In the Eastern half of the country, more females are now attending schools, but in the northern areas, this is not the case. Thus there is currently an imbalance in education. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 196 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER FOR USAID-W CONSIDERATION IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? This question was not asked. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? This question was not asked. 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? This question was not asked. 9. ARE THERE ANY OTHER PEOPLE IN NIGERIA YOU WOULD RECOMMEND WE INTERVIEW IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST US BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT FOR US? HE RECOMMENDED THAT WE INTERVIEW THE MINISTER OF COMMUNICATIONS AND TECHNOLOGY, AND SAID TO USE HIS NAME IN GETTING THE APPOINTMENT WITH HER. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 197 SITE VISIT INTERVIEW– NIGERIA - USAF PROJECT ID INFORMATION: NAME: PROFESSOR RAYMOND U. AKWULE POSITIION: PRESIDENT – DIGITAL BRIDGE INSTITUTE ORGANIZATION: DIGITAL BRIDGE INSTITUTE, AN ORGAN OF THE NIGERIAN COMMUNICATIONS COMMISSION DATE OF INTERVIEW: 5 FEBRUARY 20124 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. NIGERIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN NIGEREIA. HE HEARD ABOUT THE GBI EFFORT AT AN ITU MEETING. HIS INVOLVEMENT WAS PRIMARILY TO ADVISE INTEGRA TEAM ABOUT WHO TO CONTACT IN NIGERIA AND HOW TO SIDE-STEP POTENTIAL RIVALRIES OR TENSIONS BETWEEN VARIOUS PLAYERS. IN ADDITION TO THE NETWORKING TYPE FUNCTION, HIS OFFICE MORE OR LESS ACTED AS A “WAR ROOM” FOR THE TEAM TO PREPARE CONTACTS AND LATER ANALYZE AND STRATEGIZE. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES INTEGRA FOUND HIM VIA THE ITU-US DELEGATION AND ASKED HIM TO ASSIST THEM BY PROVIDING INSIGHTS AND THE “LAY OF THE LAND” IN ICT AREA IN NIGERIA. GBI WANTED TO GET TO THE USPF ORGANIZATION VIA THE MINISTER. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 198 HE AGREED AND THEN INTRODUCED THEM TO THE MINISTER, USPF PLAYERS AND THE NCC. B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? HE MET WITH A. MAIKANO, SECRETARY USPF, AND WAS ADVISED THAT THEY HAD ALREADY ENGAGED KPMG TO DEVELOP THE BROADBAND PLAN, BUT THAT MAIKANO FELT THAT IT WOULD BE A GOOD IDEA TO ANOTHER GROUP TO PROVIDE INPUT TO THE PLAN. 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? USPF AND INTEGRA DEVELOPED AN MOU TO SPECIFY WHAT ASSISTANCE WOULD BE PROVIDED BY INTEGRA. PROF. AKWULE RECALLED THAT IT TOOK QUITE A WHILE TO GET THE MOU FINALIZED. HE SAID THAT HE WAS AWARE OF THE USPF RETREAT ATTENDED BY THE MINISTER AND USPF, BUT THAT HE DID NOT ATTEND. HE FURTHER STATED THAT HE DID NOT GET INVOLVED IN THE DETAILS OF THE ASSISTANCE PROVIDED BY INTEGRA TO USPF, BUT THAT HE BELIEVES THAT, AT MAIKANO’S REQUEST, INTEGRA PROVIDED SUPPORT ON THE DEVELOPMENT OF A DIGITAL DIVIDE ASSESSMENT. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER THIS QUESTION WAS NOT ASKED. 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? THIS QUESTION WAS NOT ASKED. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? THIS QUESTION WAS NOT ASKED. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 199 THIS QUESTION WAS NOT ASKED. 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? THIS QUESTION WAS NOT ASKED. 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? THIS QUESTION WAS NOT ASKED. ADDITIONAL INFORMATION: HE OFFERED THE FOLLOWING OVERVIEW OF ICT REGULATION IN NIGERIA. HE STATED THAT NIGERIA HAS A GOOD REGULATORY SYSTEM AND ORGANIZATION. THE COUNTRY WAS ABLE TO MOVE FROM NEARLY ZERO GSM (MOBILE) PENETRATION TO A SIGNIFICANT LEVEL IN A FEW YEARS – PARTIALLY OWING TO THE GOOD REGULATORY ENVIRONMENT. HE BELIEVES THAT THE SAME TYPE OF “RAMP UP” WILL TAKE PLACE WITH BROADBAND. IMPORTANTLY, SUCCESSIVE PRESIDENTS AND OTHERS HAVE KEPT THEIR HANDS OFF OF THE REGULATORY (THEREFORE, REAL INDEPENDENCE HAS EXISTED). HE ALSO RESPONDED TO MY QUESTION ABOUT CORRUPTION THAT THE GOVERNMENT WAS SMART IN SETTING UP THE REGULATORS (INCLUDING USPF), IN THAT THEY PROVIDED A HIGH ENOUGH LEVEL OF PAY TO THE STAFF TO ENSURE THAT THEY DO NOT HAVE TO PANDER TO ANYONE. THE RESULT HAS BEEN THAT THE STAFF REMAINS WITHOUT CORRUPTION, AND IN SOME CASES EVEN APPEAR A BIT ARROGANT, I.E., NOT AFFECTED BY PRESSURES FROM VENDORS, ETC. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 200 SITE VISIT INTERVIEW– NIGERIA - USAF PROJECT ID INFORMATION: NAME: ROLAND OROH POSITIION: USAID – EGAT ORGANIZATION USAID – NIGERIA MISSION DATE OF INTERVIEW: 4 FEBRUARY 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRISE IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. NIGERIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN NIGERIA. HE STATED THAT HE WAS NOT INVOLVED IN THE INTEGRA USAF PROJECT IN NIGERIA. HIS KNOWLEDGE OF INTEGRA CAME FROM HIS INVOLVEMENT IN THE NEXT PROJECT (USAID FUNDED). HE DESCRIBED THE NEXT PROJECT AS HAVING THREE COMPONENTS: 1. CORRIDOR IMPROVEMENT TO REDUCE TRAVEL TIME AND GET INVESTMENTS; 2. TRADE POLICY AND FACILLITATION VIA REDOING POLICY AND MODERNIZATION OF THE CUSTOMS FUNCTIONS; AND 3. EXPORT EXPANSION SUPPORT. HE INDICATED THAT USAID’S ROLE IN NEXT WAS AS A CATALYST – RATHER THAN FUNDING ANY PHYSICAL INFRASTRUCTURE. ALONG THE CORRIDOR, INTEGRA HAS BEEN SUPPORTING ICT, AND BROUGHT AWARENESS OF THE NEXT PROJECT TO THE ATTENTION OF THE MINISTER OF COMMUNICATIONS AND TECHNOLOGY. THE INTEREST BY USAID AND THE NIGERIAN GOVERNMENT IN THE CORRIDOR IS BECAUSE LAGOS (SOUTH END) AND KNAO (NORTHERN END) ARE THE TWO LARGEST MARKETS IN NIGERIA, HAVING BETWEEN THEM MORE COMMERCE THAT SOME OF THE ADJOINING Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 201 COUNTRIES. HE INDICATED THAT THE MINISTRY (NOT SURE WHICH) JUST LAUNCED THE “TECH PARK” INITIATIVE – INTENDED TO INTRODUCE ENTREPRENEUR FOSTERING LOCATIONS. PROJECT WAS STARTED IN THE SOUTH, WITH PLANS TO ADD LOCATIONS IN THE NORTH. HE SAID THAT HE WAS AWARE OF THE USAF PROJECT, BUT WAS NOT DIRECTLY INVOLVED. HE DID SEE A DEBRIEF SESSION FROM INTEGRA. NOTE: GIVEN HIS LACK OF DIRECT INVOLVEMENT IN THE USAF PROJECT, I DID NOT ASK ANY OF THE FOLLOWING QUESTIONS OTHER THAT NUMBER 6. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? HE BELIEVES THAT USAID-W NEEDS TO CONSIDER HOW TO INTEGRATE ANY PLANNED PROJECTS WITH MISSION ACTIVITIES OR PRIORITIES BEFORE LAUNCHING. THERE IS A NEED TO GET THE MISSION PROGRAM OFFICES INVOLVED IN ADVANCE. IF A COUNTRY IS TARGETED FOR A PROGRAM, THE MISSION NEEDS ADVANCE INVOLVEMENT AND COORDINATION. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 202 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 203 SITE VISIT INTERVIEW– NIGERIA – USAF PROJECT ID INFORMATION: NAME: SCOTT WALLACE POSITIION: IFDC/AFAP COUNTRY REPRESENTATIVE DATE OF INTERVIEW: 17 December 2013 INTERVIEWER: Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. NIGERIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. I WAS ADVISED BY THE USAID MISSION THAT IFDC HAS BEEN INVOLVED WITH THE MINISTRY OF AGRICULTURE PROGRAMS, WHICH, IN TURN ARE BEING COORDINATED TO SOME EXTENT WITH THE MINISTRY OF COMMUNICATIONS AND TECHNOLOGY. THUS, I FELT THAT IT WOULD BE WORTHWHILE TO VISIT WITH YOU TO LEARN MORE ABOUT YOUR PROGRAM. Mr. Wallace indicated that the GES (Growth Enhancement Support) program is the flagship activity for the Ministry of Agriculture. It is a five-year plan whereby the Ministry wants to get government out of the business and transform Nigerian agriculture to a commercial sector. There are 7.8 million farmers in the base, and this year the program reached 4.5 million of them. Basic Process: IFDC provides pelletized fertilizer for deep penetration. The advantage of the deep penetration process is similar to that of a time-release medicinal capsule, where the benefits are slowly released over time. This is contrasted to the traditional broadcasting method of fertilizing, esp. for rice, where the excess fertilizer ultimately contributes to environmentally undesirable results. Also, experience has shown about a 35% yield increase in rice crops via the deep penetration method. Project Process: Via a paper process, each redemption center gets a list of eligible farmers, and IFDC gets a list. The idea is to manage the process whereby farmers receive both seeds and fertilizer. Apparently, the paper process has not worked very well. DFID (UK) brought in AFAP and results criteria to examine the program, i.e., how enumeration and redemption were being done. Cellulant has a 3-year contract to develop and manage a mechanized platform for the program. IFDC has the same time frame to achieve changes on Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 204 the supply side of the program. Mr. Wallace indicated that Hyperion, one of the patent holders of the M￾Pesa application (from Kenya, the cellular banking application), has been consulted, and he believes that DFID is in final stages of funding pilots in two areas. For the pilots, there will be 250 enumerators in each area, each equipped with Nexus 7 tablets. They will take photos of the farmers and obtain key information to identify and classify the farmers. Basically, there will be two classifications: 1. Farmer has NIMC card (or passport, voter card, etc.), and will be given a NF Card. 2. Farmer has no Identification. In that case, a holder of an NF Card will be able to sponsor up to 10 other farmers, each of whom will receive an NFC Card, which is a black card – good for only one season. After one season, the holder of the NFC Card will have to get an NF Card to remain in the program. The idea of this approach is to clearly identify the farmers, and to avoid duplication and potential fraud. IFDC is a sub-contractor to Chemonics (Harvey Shartup [SP?]. Mr. Wallace indicated that the Minister of Agriculture holds a meeting every Monday with key people involved in the GES program. In 2013, the Minister wanted to get 10 million phones to rural farmers, but he did not think that the goal was achieved. Also, the lack of energy infrastructure (to recharge batteries for the phones) is something of an impediment to adoption of cell phones in some areas. He also said that IFDC is mapping agriculture dealers as part of the program. He said that I could contact PropCom – Julian Peach (GRM – I think = general research manager, but not totally certain) at 08077098110, jpeach@propcommaikarfi.org – DFID funded effort supporting agriculture and soap with a rural focus. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 205 SITE VISIT INTERVIEW– NIGERIA – USAF PROJECT ID INFORMATION: NAME: SHARON PAULING & TOKS ALUKO POSITIION: Director s(Acting Mission Head), and ICT Expert DATE OF INTERVIEW: 13 December 2013 INTERVIEWER: W. Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. NIGERIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT OR KNOWLEDGE OF THE USAF PROJECT ACTIVITIES IN NIGERIA. As explained in 2, below, Ms. Pauling was briefed on the USAF project, but has not been directly actively involved. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO: A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES Judith Payne always briefed them before any project, and did so in advance on this one. Later, either Bob Otto or Eric White briefed-in Sharon. She said that it was a good move versus some other centrally-funded programs, and looked for alignment with the Nigerian government and USAID. She indicated that there is lots of interest in ICT in the mission and that it has expertise in that area. The challenge is to determine how other areas can tie-in to ICTs – and how to strengthen relations with partners. B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN NIGERIA? Toks indicated that Integra identified in 2011 that there was no broadband policy, and assisted in developing one. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 206 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. USAID FUNCTIONS HERE IN NIGERIA B. OTHER USG ORGANIZATIONS C. NGO’S D. NIGERIAN AGENCIES, MINISTRIES, ETC. The Minister of Agriculture is very active in supporting agriculture transformation using the private sector; wants to turn the USAID fertilizer program into an E-Agriculture Growth Enhancement Scheme (GES). Thus, at least two ministries are working together. E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? This question was not directly asked during the interview. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER FOR USAID-W CONSIDERATION IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? Get the missions involved up front – before project launch. Ask the relevant questions: Should Nigeria be involved? Will it fit with current or planned programs? Also, what lessons have been learned from others? Have any redundancies been taken into account? Have any leverage areas been identified (or – has the question even been asked)? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? See the answer to 5, above. Also, Sharon indicated that the Ministries need to talk to each other, and that she thinks that the Minister of Agriculture and the Minister of Finance “get it.” This is an area that needs continued emphasis and work. 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? NO 9. ARE THERE ANY OTHER PEOPLE IN NIGERIA YOU WOULD RECOMMEND I INTERVIEW IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST US BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT FOR US? Toks suggested we interview the National IT Development Agency. Sharon suggested that we might want to examine the Alliance for Affordable Internet in Nigeria and Ghana. They also Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 207 suggested we interview Alf Monaghan (Carana) the COP for NEXTT Project and Harvey Schartup the COP of the Chemonics Markets II Project. Toks will provide coordinates. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 208 GHANA SITE VISIT INTERVIEW– GHANA – USAF PROJECT ID INFORMATION: NAME: ALFRED BOYAN POSITIION: DIRECTOR, FINANCE & ADMINISTRATION, GHANA INVESTMENT FUND FOR ELECTRONIC COMMUNICATIONS (GIFEC) DATE OF INTERVIEW: 19 DECEMBER 2013 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. GHANA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT OR KNOWLEDGE OF THE USAF PROJECT ACTIVITIES IN GHANA. He indicated that he was significantly involved, given his role at GIFEC. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO: A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES Yes. Eric White, David Townsend and KB (who is?) contacted GIFEC – having come through the Ministry of Communications. B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? GBI’s (Integra) initial work was to assist in development of the strategic plan, to assist in preparing to get a bid for development of the plan. Subsequently, their role was to help develop the TOR for the desired Digital Divide Study. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 209 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN GHANA? The GBI efforts were of value in helping GIFEC in the above activities. His understanding was that GBI worked through the Director of MOC. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. USAID FUNCTIONS HERE IN GHANA B. OTHER USG ORGANIZATIONS C. NGO’S D. OTHER GHANA AGENCIES, MINISTRIES, ETC. E. OTHER Other programs that have been implemented are: ICT for Livelihood for the fishing communities. The program is educating fishermen, providing fish finders to help eliminate illegal fishing, and the Government has set up nodes for safety purposes. As a result of the Digital Divide Report – GIFEC wants to do local stakeholder assessments with smaller groups than has been thus far done. He said that the community center managers need support (said managers being local government employees). 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? The Digital Divide Report was delayed. First, ISSER delayed signing the contract. Then ISSER sent a draft report to GIFEC, and apparently there was some discussion and/or tension between ISSER and MOC. The draft report was revised and ultimately sent to GBI. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER FOR USAID-W CONSIDERATION IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE IN GHANA YOU WOULD RECOMMEND I INTERVIEW IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST US BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT FOR US? NOTE: QUESTIONS 6 THROUGH 9 WERE NOT ASKED BECAUSE KOFI ATTOR (ADMINISTRATOR/CEO GIFEC) INTERRUPTED THE INTERVIEW AND Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 210 REQUESTED THAT THE INTERVIEWER ATTEND A GROUP PRESENTATION BY GIFEC PERSONNEL. THE INTERVIEWER THEN ATTENDED THE PRESENTATION, WHICH PROVIDED INFORMATION ABOUT THE VARIOUS GIFEC PROGRAMS, ACTIVITIES AND ACCOMPLISHMENTS. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 211 ADDITIONAL INFORMATION Mr. Boyan provided the following information during the interview – not directly tied to the Interview Questions, but explaining some of GIFEC’s activities. GIFEC hopes to complete all 500 secondary locations in the country by the end of 2014. There is a lack of awareness, particularly in the rural areas. Community engagement is now underway to educate communities about ICT and the community centers. Currently 93 of the community centers have broadband Internet. All players in the ICT market are required to pay 1% of turnover, net of VAT, to support universal service/access. There are, however, some issues with getting ISP and broadcasters to pay. The Government is using a three-pronged approach: 1. Health effects – educating about the electromagnetic effects using resources from the Telecom Association, NCA and others. 2. Benefits – explaining the benefits of ICT to educate people not to be against telephone or Internet services. Chiefs and some others are provided with lap top computers. He indicated that GIFEC has an M&E program to assess the effectiveness of these activities. 3. A Telecom Association was formed to assist in getting people to accept/understand Internet. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 212 SITE VISIT INTERVIEW – GHANA – USAF PROJECT ID INFORMATION: NAME: HONORABLE KOFI ATTOR POSITIION: ADMINISTRATOR/CEO, GHANA INVESTMENT FUND FOR ELECTRONIC COMMUNICATIONS (GIFEC) DATE OF INTERVIEW: 19 DECEMBER 2013 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. GHANA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: NOTE: THE INTERVIEW GUIDE WAS NOT USED WITH MR. ATTOR, AS HE LED THE DISCUSSION ACCOMPANYING A SLIDE PRESENTATION ABOUT GIFEC AND ITS ACTIVITIES. THE PRESENTATION WAS ATTENDED BY WHAT APPEARED TO BE MOST, IF NOT ALL, OF THE GIFEC STAFF. SUBSEQUENT TO THE PRESENTATION, A BRIEF FOLLOW￾UP INTERVIEW WITH MR. ATTOR WAS DONE, AND IS DESCRIBED BELOW. The interviewer asked Mr. Attor – “Do you have any further thoughts or comments about Integra’s involvement in Ghana?” He responded THAT GBI did not impose anything on GIFEC; rather, it showed the organization best practices around the globe on universal access/service. Integra helped revised the presentation (that had just been shown to the interviewer); provided proposals for improvement; helped expand the organization chart for GIFEC; and David Townsend also provided an organization chart for NCA (the regulator) which is responsible for M&E of the various universal access/service programs. He stated that the delay in the Digital Divide Study delivery was due to ISSER wanting more money. 1. PLEASE TELL ME OF YOUR INVOLVEMENT OR KNOWLEDGE OF THE USAF PROJECT ACTIVITIES IN GHANA. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO: A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 213 B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN GHANA? 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. USAID FUNCTIONS HERE IN GHANA B. OTHER USG ORGANIZATIONS C. NGO’S D. OTHER GHANA AGENCIES, MINISTRIES, ETC. E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER FOR USAID-W CONSIDERATION IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE IN GHANA YOU WOULD RECOMMEND I INTERVIEW IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST US BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT FOR US? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 214 SITE VISIT INTERVIEW – GHANA – USAF PROJECT ID INFORMATION: NAME: ISSAH YAHAYA POSITIION: DIRECTOR, POLICY PLANNINE, MONITORING & EVALUATION, MINISTRY OF COMMUNICATIONS DATE OF INTERVIEW: 20 DECEMBER 2013 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. GHANA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: NOTE: DIRECTOR YAHAYA EXPLAINED HIS ROLE IN THE MOC AND OFFERED THE FOLLOWING COMMENTS. Ghana has transitioned to a broadband strategy and policy for ICT. Broadband implementation is the responsibility of NITA (National Information Technical Agency), which directly supervised implementation of optic fibre and broadband development in Ghana. Current development in the country: minimum of 2 Mbps, there are 5 submarine cables. The terrestrial National Communications Backbone has been built in the Western area @ 12.3 Tbps, and the Eastern portion is currently being planned for 780 Klm (and funded by Danida). Further, an E￾Government Platform is being built to cover all of the centers. There are currently 30 WIMAX (LTE) sites, with plans to upgrade to 90 WIMAX sites. He was not aware of GBI or Integra’s involvement in Ghana, so that most of the questions in this guide were not pertinent to the interview. 1. PLEASE TELL ME OF YOUR INVOLVEMENT OR KNOWLEDGE OF THE USAF PROJECT ACTIVITIES IN GHANA. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO: A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 215 B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN GHANA? 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. USAID FUNCTIONS HERE IN GHANA B. OTHER USG ORGANIZATIONS C. NGO’S D. GHANA AGENCIES, MINISTRIES, ETC. E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER FOR USAID-W CONSIDERATION IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? He thinks there is a need for more collaboration between MOC and USAID. He would like to see more USAID contact. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE IN GHANA YOU WOULD RECOMMEND WE INTERVIEW IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT FOR US? He suggested interviewing the NITA General Director, William Tevie and also the NCA Chief Director (Tel: 0244-015660). Mr. Tevie was later interviewed, but the interviewer was not able to reach the NCA Director. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 216 SITE VISIT INTERVIEW – GHANA – USAF PROJECT ID INFORMATION: NAME: KIM KIM YEE POSITIION: PRIVATE ENTERPRISE OFFICER, OFFICE OF ECONOMIC GROWTH, USAID, GHANA DATE OF INTERVIEW: 20 DECEMBER 2013 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. GHANA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT OR KNOWLEDGE OF THE USAF PROJECT ACTIVITIES IN GHANA. MS. YEE HAD NO EXPOSURE TO THE INTEGRA PROGRAM. SHE PROVIDED AN EXPLANATION FOR WHY THERE WAS NO APPARENT INTEREST (INTEGRA’S DESCRIPTION) BY THE MISSION IN THE PROJECT. SHE WAS AWARE THAT INTEGRA HAD MET WITH NINO IN ECONOMIC GROWTH, SEEMING TO RECALL THAT THERE WERE THREE OR FOUR INTEGRA PEOPLE INVOLVED. AT THAT TIME, THE MISSION WAS IN AN “ALL HANDS ON DECK” EFFORT TO SUPPORT THE “FEED THE FUTURE” USAID INITIATIVE. SHE SAID THAT THE MISSION WAS STRUGGLING TO MEET THE REQUIREMENTS OF THAT PROGRAM; THAT THERE WERE NOT ENOUGH RESOURCES IN THE MISSION TO CONCURRENTLY SUPPORT THE GBI PROGRAM. SHE STATED THAT THE “TIMING” OF THE GBI PROGRAM WAS NOT “GREAT.” NOTE: SINCE THE CURRENT MISSION DIRECTOR WAS NOT IN PLACE DURING THE GBI EFFORTS IN GHANA, NO ATTEMPT WAS MADE TO INTERVIEW HIM. MS. YEE MET WITH THE INTERVIEWR AT MS. FLANAGAN’S REQUEST. MS. YEE WAS JOINED FOR A SHORT WHILE DURING THE INTERVIEW BY CYNTHIA (LAST NAME UNKNOWN), WHO IS IN THE PROGRAMS GROUP AT THE MISSION. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO: Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 217 A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN GHANA? 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. USAID FUNCTIONS HERE IN GHANA B. OTHER USG ORGANIZATIONS C. NGO’S D. GHANA AGENCIES, MINISTRIES, ETC. E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER FOR USAID/W CONSIDERATION IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? MS. YEE STATED THAT NEW USAID-W PROGRAMS SHOULD BE CONSIDERED IN TERMS OF THE COUNTRY DEVELOPMENT COOPERATION STRATEGY (CDCS), WHICH INCLUDES ALL MAJOR PLANS FOR THE COUNTRY. THE IDEA HERE WOULD BE TO ENSURE THAT USAID-W PROJECTS CAN BE INTEGRATED OR SUPPORT MISSION PROGRAMS/PROJECTS. SHE SAID THAT PETER (PRESUME THAT IS THE CURRENT MISSION DIRECTOR) WOULD BE INTERESTED IN LEARNING MORE ABOUT HOW ICT COULD BE A COMPONENT OF MOST PROGRAMS. 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE IN GHANA YOU WOULD RECOMMEND I INTERVIEW IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST US BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT FOR US? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 218 SITE VISIT INTERVIEW – GHANA – USAF PROJECT ID INFORMATION: NAME: PHILIP PREMPEH POSITIION: BUSINESS DEVELOPMENT MANAGER, GHANA INVESTMENT FUND FOR ELECTRONIC COMMUNICATIONS (GIFEC) DATE OF INTERVIEW: 19 DECEMBER 2013 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. GHANA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. NOTE: MR. PREMPEH WAS PRESENT FOR THE GIFEC SLIDE PRESENTATION. AFTER THE PRESENTATION, THERE WAS A VERY BRIEF SESSION WHERE HE RESPONDED TO SEVERAL OF THE INTERVIEW QUESTIONS, AS SHOWN BELOW. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT OR KNOWLEDGE OF THE USAF PROJECT ACTIVITIES IN GHANA. He went to the workshop in Dar es Salaam; he worked on the TOR for the Digital Divide Study; and also worked on developing foundation questions for the survey (Digital Divide). He also attended the second Integra webinar – on strategic planning. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO: A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN GHANA? 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 219 A. USAID FUNCTIONS HERE IN GHANA B. OTHER USG ORGANIZATIONS C. NGO’S D. GHANA AGENCIES, MINISTRIES, ETC. E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER FOR USAID-W CONSIDERATION IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? Regarding the webinar, he had expected the sessions to be two-way (interactive), but found that it was not. He said that it was “OK” as presented. He offered that people (meaning – in Ghana and other African countries) are not accustomed to that type of long-distance learning. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? He said that GIFEC initially thought that GBI was going to fund the Digital Divide Survey – but added that it was not a big issue. 9. ARE THERE ANY OTHER PEOPLE IN GHANA YOU WOULD RECOMMEND I INTERVIEW IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST US BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT FOR US? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 220 SITE VISIT INTERVIEW – GHANA – USAF PROJECT ID INFORMATION: NAME: WILLIAM TEVIE POSITIION: DIRECTOR GENERAL, NATIONAL INFORMATION TECHNOLOGY AGENCY (NITA) DATE OF INTERVIEW: 20 DECEMBER 2013 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. USAID FUNDED THE USAF PROJECT, WHICH SOUGHT TO SUPPORT SSA AFRICAN COUNTRIES IN CONTINUING TO DEVELOP AND IMPLEMENT EFFECTIVE UNIVERSAL ACCESS PROGRAMS AND FUNDS. GHANA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA (THE FIRM CHOSEN TO IMPLEMENT THE USAID PROGRAM) FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID HAS HIRED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO BOTH ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: NOTE: DIRECTOR TEVIE HAD NEVER HEARD OF EITHER GBI OR INTEGRA. HE STATED THAT HE IS RESPONSIBLE FOR PLANNING AND IMPLEMENTING ALL GOVERNMENT ICT PROJECTS. HE MENTIONED THAT THE CHINESE EXIM BANK IS PROVIDING 18 MIL (NOT SURE OF CURRENCY) SUPPORT AND THAT THE WORLD BANK IS SUPPORTING E-GOVERNMENT INITIATIVE. DANIDA IS PROVIDING 40 MILLION EUROS FOR THE BROADBAND BACKBONE. HE ALSO MENTIONED AN OFF-SHORE GHANA BROADBAND NETWORK. HE DECLINED ELABORATING REGARDING THE RELATIONSHIP BETWEEN GIFEC AND NITA. GIVEN HIS LACK OF AWARENESS OF GBI OR INTEGRA, THE INTERVIEW GUIDE QUESTIONS WERE NOT ASKED. 1. PLEASE TELL ME OF YOUR INVOLVEMENT OR KNOWLEDGE OF THE USAF PROJECT ACTIVITIES IN GHANA. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO: A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 221 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN GHANA? 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. USAID FUNCTIONS HERE IN GHANA B. OTHER USG ORGANIZATIONS C. NGO’S D. GHANA AGENCIES, MINISTRIES, ETC. E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER FOR USAID-W CONSIDERATION IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE IN GHANA YOU WOULD RECOMMEND I INTERVIEW IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST US BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT FOR US? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 222 TANZANIA SITE VISIT INTERVIEW – TANZANIA - USAF PROJECT ID INFORMATION: NAME: HON. MAKAME M. MBARAWA POSITIION: MINISTER ORGANIZATION MINISTRY OF COMMUNICATIONS, SCIENCE & TECHNOLOGY DATE OF INTERVIEW: 15 JANUARY 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN TANZANIA. Minister Mbarawa stated that he has never worked with USAID, and that there have been no contacts on a government to government basis. He further stated that he believed that USAID and the Ministry need to talk – to explore how “we can work together.” Although he was not familiar with the GBI/Integra project, the Minister provided information about the history and status of universal access in Tanzania. Notes from the conversation are at the end of this interview guide. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 223 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? This question was not asked, due to his lack of knowledge about GBI and Integra. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? This question was not asked, due to his lack of knowledge about GBI and Integra. A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? He described the delay in staffing the UCAF organization. He said that the problem was the vetting process to hire government employees was a lengthy process, but that in December 2013, clearance was given to hire personnel. He said that he had received no communications from Peter Ulanga or USAID regarding the delay, but emphasized that the regulatory authority and UCAF are both independent agencies. He further said that he believed that Peter Ulanga had also been tied up quite a bit in various tender projects, and had been under much pressure. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. COMMENTS OF MINISTER MBARAWA: In 2006, the universal service fund was created. In 2010, Tanzania started to implement universal service. The first tender for a pilot project failed. For the second tender, a vendor applied to provide 52 watts in Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 224 316 villages (UCAF – Peter) is in charge of this project. All carriers must bid on the tenders. He cited two methods for awarding: 1. Auctions like in the U.S., 2. a “beauty contest” where if commitments are not met, license would be taken away. He indicated that they plan to use the auction approach for LTE awards. Ten regions have been converted from analog to digital; with 30 total regions – all to be completed by July 2014. He said that the ITU guidelines will be used to handle the “white space” frequencies. A third tender will be funded. He said that the service providers did not want to bid. The Government had introduced a 1,000 TZHS Sim card fee (wanting to use the resulting funds for rural electricity). The plan did not go into effect, as the vendors fought it. He then removed the fee, but also required the vendors to build not less than 168 watts in rural areas by 15 February. The vendors agreed. He believes that, by the end of the year, there should be at least 1670 watts for about 900 villages. Four new regions were subdivided for the analog/digital conversion, and broadcasters are included. Sources of funding for universal access/service are: 1. World Bank, 2. The Universal Service fund (0.3% of turnover), and 3. Other Government monies. Both mobile carriers and TTCL pay into the fund, but the ISPs are not yet paying. He said that the small ISPs may not survive if they have to pay the 0.3%. A fourth tender is planned by June or July 2014. The government wants to use low cost technologies, and an Indian company has a small package that may be attractive. The requirements will be that it must be inexpensive and sustainable. He believes that the tender can be closed in three months. He indicated that there is a need for regulations to decide on how to utilize “white space” frequencies, and that the regulations are needed by the end of 2014. Minister Mbarawa also stated that COSTECH is important as an incubator for ICT and information technology development, citing some 295 graduate students (either already or in pipeline – which was not clear). COSTECH and UCAF are working closely together. He thinks that there will be many activities in the universal access/service areas. The national broadband infrastructure was built with Government money. In 2007, the President tried to get the private sector to build it, but they said that the country was not ready. 7,560 Klm is now operational with three self-healing rings (Southern, Western and Northern). All region headquarters offices are now connected, and all countries which are contiguous to Tanzania have also been connected. Vodacom, AirTel, SimbaNet, Zantel, TTCL and small ISPs are now using the broadband backbone. In 2009, Vodacom had built a broadband backbone from Dar es Salaam to Doma, some 450 Klm. Vodacom has now “jumped” to the national backbone because it is cheaper. Pricing – Before the national backbone, 2 Gbs cost 100K THS (in 2009). In 2010 – after Phase I, the price went down to 60K THS for 2 Gbs. In 2012, Phase II brought the price down to 30K THS for 2 Gbs. And, in 2013, the price for 5 Gbs was 27K THS. He does not believe that the national fibre backbone is expensive. He also mentioned that a “bundle” is available where 1Gb, 15K SMS, and 150 minutes of voice cost 5K TS. In 2013, the interconnection fee was small – at 150 THS per minute. In 2014, the fee is 34 THS for the same minutes. The five year plan for Phase III will bring data centers and potentially a mesh network. Part 1 is estimated to cost $93 Million USD (from now until April 2015). Part 2 will provide E-Government and “E-Everything” for somewhere around $300 Million USD. The national backbone is operated by an agency under TTCL (with separate financial accounts). Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 225 He said that a revised national broadband strategy is currently under development, with a white paper due by July 2014. Peter Ulanga is involved, along with others. Visits are planned for Nigeria, Kenya and Ghana as part of the effort. Minister Mbarawa again emphasized that he believes that Tanzania and USAID need to work together in these areas, and he would welcome contacts from USAID in that regard. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 226 SITE VISIT INTERVIEW – TANZANIA - USAF PROJECT ID INFORMATION: NAME: JAMES M. KILABA (PREng) POSITIION: Deputy Director, ICTs Development ORGANIZATION Tanzania Communications Regulatory Authority DATE OF INTERVIEW: 20 January 2014 INTERVIEWER: W. Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN TANZANIA. In May of 2012, TCRA organized “ICT Week,” which was attended by Peter Ulanga. He heard about Integra’s work and the TOR at that time, but thought the TOR was universal rather than just for Tanzania. He also attended the workshop held in Dar es Salaam, and recognized the need to review the ICT policy to include broadband. He said that TCRA also recognized the need to be ready, to address spectrum availability in particular. He said that TCRA could use assistance in determining how best to handle the spectrum that results from the analog to digital conversion of TV broadcasting, and that the need will be by the end-of-year 2014. The 694 to 862 MHz spectrum will need to be “reformed” (he could have meant or said “reformed), and there is a need to determine its economic value. If Tanzania wants to use that spectrum for wireless broadband, a policy will be needed for that. TCRA will need to evaluate whether or not to have a frequency auction, and if so, how and what to prescribe in that regard. He mentioned at this point that the UCAF organization would be the implementer of whatever is ultimately decided. He also said that TCRA recognized the need for spectrum efficiency in broadband. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 227 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? No. 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? Note: Given his lack of direct involvement, Questions 3 through 9 were not asked. Additional comments that he offered during the interview were: Price of the fibre backbone – He said that the network is not finished, so that he believes that the current prices are transitional rather than final. He said that he was not, however, intimately aware of the current prices and tariffs as there is no regulatory approval yet. That will take place when the backbone has been completed. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 228 He stated that he believes that the Ministry is receiving assistance in developing the broadband strategy and policy, but was not aware of the source of the assistance. From a regulatory perspective, he stated that the following questions need to be addressed:  What are optimum technological methods for rural broadband?  How should the CPE aspects be addressed, such as learning, affordability, etc.?  How to maintain “technology neutrality” versus preferential treatment? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 229 SITE VISIT INTERVIEW – TANZANIA - USAF PROJECT ID INFORMATION: NAME: Kevin McGown POSITIION: Policy & Research Advisor ORGANIZATION USAID - Tanzania DATE OF INTERVIEW: 17 January 2014 INTERVIEWER: W. Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: Mr. McGown indicated that he has been on post in Tanzania for 2.5 years involved in Policy & Research for the Feed the Future program. 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN TANZANIA. He became aware of the program about a year ago, first about NetHope, and then through Jeff Cochrane and Bob Otto, who visited (when they came to Tanzania for the Zanzibar workshop). He also mentioned that Bob Otto had been there prior to a year ago and Judith Payne had also briefed the Mission. He said that Jeff Cochrane had been keeping him up to date on the status of the project. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES His only involvement was to be briefed in on the project – and to be kept up to date. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 230 B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? He thought that there was not much impact, that it was not a big focus, perhaps due to learning that the work with Peter Ulanga would not take place. He put Integra in touch with potential implementation partners (IAGRI of Ohio State), Sukawini (Sp?) University (COP working on agriculture + ICT, and had some ideas for sites for broadband). COP’s name was David Kraybil. There were a couple of others he apparently had mentioned, but he did not name them specifically. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER He mentioned UCAF and COSTECH as partners, and a visit to two (2) research stations. 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? He cited Peter Ulanga’s lack of staff and mentioned the underutilization of the fibre backbone as obstacles. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? He appreciated not having been overloaded about the project, and that Jeff Cochrane had provided just enough information for him to stay generally up to date. He said that he would appreciate continued guidance on the project if it moves forward – what it is and how it will be implemented. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? This question was not asked. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 231 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? This question was not asked. 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? This question was not asked. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 232 SITE VISIT INTERVIEW – TANZANIA - USAF PROJECT ID INFORMATION: NAME: KRISTIINA LAHDE POSITIION: CHIEF TECHNICAL ADVISER ORGANIZATION TANZICT DATE OF INTERVIEW: 16 JANUARY 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN TANZANIA. Ms. Lahde was unaware of Integra’s involvement in Tanzania. For that reason, this interview guide was not used during the session. Instead, the interview became one of her imparting information about her work and COSTECH (and other ICT related items) activities in the country. See the end of the interview questions for the write-up of this “free-flowing” interview. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 233 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Interview with Ms. Lahde: She indicated awareness of Peter Ulanga and the Universal Access organization. Her work is funded by Finland via the Ministry of Information and Ministry of Foreign Affairs. The objective is to develop an information society in Tanzania, and the project has three components: 1. Renewal of ICT policy, as the existing policy is too general. Effort being led by a Tanzanian in the Ministry, and the UN Agency for Africa is helping. Eight pillars have been identified and published, and currently the policy is being edited and put into proper format. It should go out for public comment/input soon. She mentioned here that COSTECH will prepare a “tool Kit” for organizations to use to share with their constituents – to enable them to see/learn what is important. 2. Capacity building in the Ministry – educating on how ICT works, identifying what can be changed to make ICT the driver. I asked where the current Minister fits in the political “pecking order?” She said that the president is interested in ICT; that the former permanent secretary to the Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 234 Ministry is now in the president’s office; and that the Deputy Minister appears to be a “rising star” and that he is ICT savvy. 3. Innovation Component – K. Lahde and her office mate are responsible. They are looking at the local ICT sector – how to strengthen it. For example, the ISP’s and companies do not have R&D today, as they are mostly small companies. Living laboratories are being developed where non￾paid volunteers assist people (two to three times per week). COSTECH is now committed to this activity, and has hired two people who will become permanent employees. She indicated that there is one and one-half years remaining on the Tanzania ICT project. The question was asked if there are any defined deliverables, such as target numbers, etc. Ms. Lahde replied yes, but not very early on in the project. They now want to work towards COSTECH taking over the project. Universities are big partners, especially the ones with interest in ICT. The team has attempted to socialize their efforts to let organizations know about the project and advise them about what Tanzania ICT project can do for them. There is some amount of funding to send people to Finland. She said that the universities have not received much guidance about ICT. Some universities came back for assistance and some did not. In Dar es Salaam, Zanzibar, Iringa, Arusha and a new one in Mbeya (Sp?) apparently got involved or expressed further interest. She mentioned that the Nelson Mandela African Institute of Science near Arusha wants to be the “MIT’ of Africa. She also mentioned that different programs are available for innovation, such as curriculum inputs or incubators. On the community side, she offered the following information. Living laboratories – where they learned that innovations came from the users – not from R&D. They learned that it was better to take the efforts to where the people are, rather than trying to draw people to a different location. They have done workshops on how to provide living laboratories, and in Iringa they got the community to define the problems. This led to a changed mindset, where the community learned that it can do something, i.e., it does not have to wait for the government to do something. She said that R-Labs from South Africa is their partner in the Living Labs effort, and that it has tools including social media and other tools showing how to connect with each other and the outside world (for example: Google, Facebook, etc.). Thus, ICT is seen as a “tool” to enable people to get noticed. She mentioned a global outreach aspect which is youth oriented. She mentioned an “E-Paper” commercial experiment which was a joint project involving the Finnish Company Leia Media and Suntrica. Apparently, Leia Media supplied the e-paper and Suntrica supplied the solar powered devices. The project was completed during November/December 2013, and the results are now being analyzed. Living Labs involvement was mainly in working with companies. On the subject of Universal Service in Tanzania, her assessment was that connectivity is good in the towns, but not far outside of town the connectivity is poor. Connectivity also is unreliable. She mentioned a “Girls in ICT and engineering effort that was “very visible.” But, she then indicated her opinion that in Tanzania there is a tendency to “check the boxes” versus taking real action – and that she is not sure about any follow-up on this effort. Ms. Lahde thinks that universal service should be linked to electricity. She stated that she believes that DFID is looking to do something in Tanzania, and is considering several modalities. The project manager for DFID is Charles Gore (in Kenya @ +254-72-776-7537), and also named Liz Taylor as part of the effort. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 235 She explained the Ministry organization/functioning as follows: Under the Minister is the Permanent Secretary who has three reporting Directors. These are: Director ICT, Director Communications, and Director Science & Technology. The ICT Director has all broadband responsibility, but because the Tanzania ICT role is not considered infrastructure, Tanzania ICT responsibility was transferred to the Director Communications. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 236 SITE VISIT INTERVIEW– TANZANIA - USAF PROJECT ID INFORMATION: NAME: MIHAYO WILMORE POSITIION: MANAGING DIRECTOR ORGANIZATION UHURUONE DATE OF INTERVIEW: 15 JANUARY 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN TANZANIA. His only engagement was that he met “someone” (could not recall who) in Singapore where white space was being discussed. Also, Bill Garrison of NetHope and Mihayo had been talking for about a year regarding how to get broadband into schools in Tanzania. He provided a significant amount of information relating to various initiatives and issues in Tanzania, which is contained in the Comments at the end of this interview guide. Because of his lack of contact and awareness of Integra’s proposed activities in Tanzania, Questions 3 through 9 were not asked during the interview. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES He did not make clear how any other contacts with Integra happened. He did mention that he had presented a talk about ICT and Education at the Zanzibar workshop. He Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 237 indicated that Integra was “in and out” of that conference – monitoring the workshop progress. B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? He thinks that Integra only acts as consultants rather than implementers – versus NetHope. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. FURTHER COMMENTS/INFORMATION PROVIDED BY MR. WILMORE: Regarding getting broadband into the schools, he was already doing TZ21 in Tanzania, and said that it is “tough” to start as an ISP in the rural areas. Language translations are problematic – due to academic versus the “real” Swahili (that actually spoken by the people). Also, smartphones have limitations so that everything that is necessary cannot be handled, especially for business applications. VSat @ 512 Mbps is good for the education centers, especially where KA band is used. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 238 Uhuruone advised TZ21 that 2G/3G will not work for the needed applications, but that KA band would work. The only problem now is the budget issue. He believes that the subject needs more “education” at the national parliament level. He stated that every employer (any business with a TIN) pays 3% of gross turnover to the Ministry of Education. Vocational training gets the 3%. He explained how the educational system works in Tanzania regarding where students are placed as they progress through the system. He mentioned USAID in connection with the Tawezwa Program, and then described where and how students are placed in the educational system. His objective is to find ways to embed ICT throughout the entire educational process – in all of the different types or modalities of training. He said that NetHope is working with SOKOINE, an agriculture university some 400 Klm from Dar es Salaam. Service is being provided by radio from Dar es Salaam, then via a WI-FI mesh network on campus – using white space frequencies. Uhuruone did the design for the system, but it fell apart, he believes due to funding issues. MOU between Uhuruone, Microsoft and COSTECH. Objective is to provide broadband for the whole of Tanzania education system. Microsoft has provided grant money of $300k USD for equipment for four schools. Now, a feasibility study is needed for how to get money to provide equipment for 72K students. For three years, 200 Mbps will be provided, with COSTECH as custodian. After three years, Uhuruone will take over. He is working with the Tanzania Investment Bank to get funds to purchase lap top computers for the four schools. The schools with then add technical fees, charging $15USD per month. The target is to bring 63 universities on line in five years, with more than 500k students. By 2025, 5 Million Tanzanians will be trained. Microsoft will provide an IT Academy for universities – to train people on Office. This item is planned under the MOU and will include a full service contract (one per 5K students). He mentioned that neither CISCO no Microsoft have offices in Tanzania, so that he is trying to encourage Microsoft, at least, to put someone in the country on a permanent basis. Current GAP – He believes there is a need to charge the broadband “pipes” on a reverse basis. Whereas the Government today charges $1k USD per STM-1, he believes, for example, the charge should be $100 USD at the beginning, with the price increased later on. He is now negotiating with TTCL for a revenue sharing arrangement, but it has been “on the table” for nine months. He believes that TTCL “does not get it.” He stated the opinion that COSTECH reports directly to the President of Tanzania; that the Minister is not in the loop. He questions why TTCL operates the national fibre backbone, and believes that there is a need to a totally separate entity to operate the network. He said that he cannot force TTCL to speed up. He has seen only one tender, but it was only for voice. He thinks that USAF is more motivated to get voice services to areas due to political pressures. He does not know how to contact or connect with UCAF, but stated that he pays.8% to the regulator, who then provides .3% to UCAF. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 239 SITE VISIT INTERVIEW– TANZANIA - USAF PROJECT ID INFORMATION: NAME: Eng. Peter Ulanga POSITIION: Chief Executive Officer ORGANIZATION Universal Communications Service Access Fund - Tanzania DATE OF INTERVIEW: 17 January 2014 INTERVIEWER: W. Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: Mr. Ulanga had the following people join the interview:  Justin Mashiba – Board Secretary and Legal Counsel  Albert Richard – Head of Operations  Pius Joseph – Head of Administration  John Munkondwa – Head of Procurement Management  Abaham Mangesbo – Head Audits and M&E 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN TANZANIA. He met David Townsend in May of 2010 at an Intel initiative in Nairobi, Kenya. He said that he had been meeting with the Integra team over time, and mentioned meeting in both Istanbul, Turkey and then in Dar es Salaam. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 240 B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? He tried to work with Integra via networking, and then signed a TOR, but no work took place under the TOR because he did not have the needed staff. TOR was signed in January 2012, but his first staff person did not come on board until August 2012, followed after that by other staff. I asked why this happened, what was the delay. He said it was due to government bureaucratic processed. He said that he has been using Ministry staff on a loan basis, but no ICT skills on￾board. He is still forcing up. It was mentioned to him that there were other people needs, such as: operations, accounting, legal and administration/logistics. 2. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? Since no work took place under the TOR, this question was not asked. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER Since no work took place under the TOR, this question was not asked. 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? There were no obstacles other than the lack of staff to benefit from the TOR planned efforts. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? Question asked: “What would you want to see from USAID?” His responses were: For any infrastructure project, at least five (5) years is needed for support. He also said he appreciates that each country is different, so that countries can learn from each other, so that more regional or mixed type workshops would be beneficial (and, the workshops should not be limited to African countries). He said that funds are looking at extending infrastructure, but he thinks that they need to move toward application and training roles, i.e., how to use ICT to improve Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 241 social benefits. Along those lines, he indicated that he is currently working in Tanzania with the Ministry of Health on Telemedicine. Albert Richard said that he would like to see more time for learning, that there is a need to look at the entire project before launching. There is a need to find out what the country and stakeholders in that country need before starting a project. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? This question was not directly asked, but see A. Richard’s comments on Q6, above. 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? I asked if Peter’s organization would like to “retrigger” the Integra involvement. Peter responded yes, and that specifically they could use assistance in the Market Assessment (coverage gaps and similar aspects). He indicated that his organization is now doing the analysis, but does not have the same level of skills as Integra. 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? This question was not asked. Peter said that he would like to receive a copy of the evaluation report when completed. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 242 KENYA SITE VISIT INTERVIEW – KENYA - USAF PROJECT ID INFORMATION: NAME: Bitange Ndemo, PhD. POSITIION: University Professor and Informal Advisor to President, formerly was Permanent Secretary ORGANIZATION Ministry of Communications (during Integra’s project) DATE OF INTERVIEW: 21 January 2014 INTERVIEWER: W. Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: Dr. Ndemo provided the following information during the introductory part of the interview. He is currently teaching entrepreneurship at university in Nairobi, and is acting as an informal strategic advisor to President Kenyatta. I asked about current status of the major business park endeavor which he had developed. His response was that he was somewhat disappointed, as the new administration appeared to be leaning toward the Chinese more than the Western countries. He indicated that he has been helping U.S. companies in doing business in Kenya, and mentioned IBM and Mercy Corp. He said that he is the Chairman of the Alliance for Affordable Internet. He stated that he believes that spectrum should be open, and that Kenya needs a spectrum policy. He has been working with McKinsey (Bill Russo), and mentioned their report “Lions Go Digital.” His objective is to get McKinsey in front of the Kenya government people in order to have an impact, as he thinks that the firm has changed the western discourse and awareness of the emerging African middle class. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 243 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN KENYA. His awareness on Integra’s activities came about as a result of his daily contact with the Minister and ministerial personnel. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? He was not directly contacted by Integra. A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? He said that Integra’s output went to CCK, which then provided information to him (as the Permanent Secretary), then from him to the Minister for policy considerations. He then stated that it will be critical for Kenya to deal with the so-called “white space” spectrum, and that such spectrum will need to be returned after a one-year period if not used. Also, he said that the process must be handled in a public and transparent manner, and that CCK will have the responsibility for regulating spectrum subsequent to completion of the policy. He said that Kenya has an ICT strategy and an ICT Board. This strategy was done after the NBS, with the NBS as the foundation policy. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? This question was not asked. A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? He said that the previous president took an interest in the ICT area and was easier to do business with, whereas the current administration is somewhat “sensitive.” 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 244 He said that the IBM Company sends senior executives to the Ministry to ask what they want. The Ministry then identifies what it believes is a problem where assistance could be of value. He also said that it is valuable in Kenya to get a local point person who understands the industry in question and the local politics, with the point person being invaluable in getting appointments with the necessary players. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? This question was not asked. 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? He said that the operators frustrated the universal service access fund activities in Kenya. 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? This question was not asked. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 245 SITE VISIT INTERVIEW – KENYA - USAF PROJECT ID INFORMATION: NAME: Francis W. Wangusi POSITIION: Director General ORGANIZATION: Communications Commission of Kenya (CCK) DATE OF INTERVIEW: 22 January 2014 INTERVIEWER: W. Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN KENYA. His initial awareness came from within CCK, likely Susan Mochache. He became interested for two reasons: (1) Because the Universal Service Fund was underway, and (2) they were in the process of developing the National Broadband Strategy (NBS). 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? See response to Question 1, above. A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 246 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? He said that the USF was a new concept, and that the general issues were how to do it and what the gaps in access were. Integra enabled CCK to address some of the issues. In addressing how to implement USF, Integra helped them look at the experiences of other countries. Thus CCK felt it needed to broaden the scope of the program and to develop more information. Integra’s involvement helped CCK and the Board better understand the issues, and Integra also assisted and guided them in the development of the NBS. He said that Integra’s assistance to CCK had been very successful. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. He said that the project had had an impact on policy makers and experts from other key ministries, such as roads and infrastructure, finance and planning. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER He said that universities and lecturers experienced the impact, as well as representatives from industry (he appeared to be referring to the NBS stakeholder process here.). He further said that the question was “Who should be prime in socializing ICT as the needed link in development?” But, nothing happened, so that CCK took the ball and was most instrumental in socializing the ICT role. 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? He felt that CCK did not get full support; that the Permanent Secretary and Minister could have helped move it faster. He said that, in the end, “they came.” I asked why he thought that the Minister did not do more. He said that the Universal Service Advisory Council issue seemed to be the hold up (but he did not elaborate on this). He also said that the Minister’s “awareness” may not have been complete due to the various priorities of his advisors. He said that the Minister was not an ICT type, but that he was a political appointee. And lastly, he stated that CCK kept getting “empty promises.” He also said that he did have much time with Integra, but thinks that sometimes the unavailability of CCK people to Integra may have presented some obstacles. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 247 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? He stated that there is a need to build in adequate time to ensure sustainability of the capacity that has been built. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? This question was not asked. 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? He said he just wanted to thank USAID for its support, and that CCK could use continued support. 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? This question was not asked. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 248 SITE VISIT INTERVIEW – KENYA - USAF PROJECT ID INFORMATION: NAME: Mervyn Farroe & Albert Waudo POSITIION: MF = Supervisory Program Officer & AW = Program Management Assistant ORGANIZATION: USAID Kenya DATE OF INTERVIEW: 24 January 2014 INTERVIEWER: W. Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN KENYA. Mervyn said that he was working on Agriculture and received an email from Integra providing background on the USAF GBI project. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES See response to Question 1, above. Also, Mervyn said that what Integra needed was help in facilitating the MOU with the Kenya government. He handed the request off to Michael Jones, and recalls that it took about a year to get the MOU signed/completed. Albert came in after the MOU, and his role was limited mainly to administrative assistance to Integra. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 249 It was asked – Did any USAID person attend any meetings with Integra? Mervyn responded that possibly Tom Mooga (Sp?) attended something, as he was the East Africa IT person. Also, M. Jones may have attended a meeting. Mervyn stated that the project was not a “heavy lift” for the mission, as it did not need much effort. But, he always thought that it was an important endeavor. He said that Integra never asked for anything. Question asked – Was the USAID Mission OK with Integra running without USAID technical involvement? Mervyn said “No problem.” Was there any advance input from USAID-W (J. Payne)? Mervyn said that he was not aware of any input prior to the MOU request. Mervyn stated that he had to brief-in the front office at the Mission, but that ICT was not a primary focus, and was a cross-sector item. B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? Albert said the impact was good. Getting the National Broadband Strategy (NBS) was valuable, and that CCK is now getting the tender out. The current Minister, Dr. Fred Matiangi used to be the Chief of Party of the USAID project – parliamentary strengthening. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER Albert said that there was impact on the ICT Board, telecommunications providers and ISPs. 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? Mervyn thinks that there was some in-fighting within CCK which resulted in delays. He mentioned Beverly Hadley, who was an attorney who worked on the MOU. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 250 Mervyn: From the country’s perspective USAID-W needs to reach out early and understand what the local focus is. It is not completely necessary to follow the Mission’s priorities, but early-on coordination and links are critical – even where the Mission is not engaged due to its current focus. Albert: There is a need for more sharing of documents and reports, as sometimes the Mission does not receive status reports. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? They indicated appreciation for the project. 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? This question was not asked. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 251 SITE VISIT INTERVIEW – USAF PROJECT ID INFORMATION: NAME: Muriuki Mureithi POSITIION: C.E.O. (Interviewed in his role as an expert consultant for Integra) ORGANIZATION: Summit Strategies, Ltd. DATE OF INTERVIEW: 21 January 2014 INTERVIEWER: W. Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIE IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: Background provided by Mr. Muereithi He previously worked for Telecom Kenya Post & Telecom; he left that company in 1994 via voluntary retirement and became a consultant. As a consultant, he has worked with governments, Telcos, and investors. He worked with UNEC of Africa on the Information Society Initiative in 1996 to develop a framework to open markets. In Botswana he worked with the National Information Communications Infrastructure (NICI) which developed tools to scan ICT adoption to measure progress in Kenya. Along with David Townsend, he supported a World Bank paper. He won the bid for the Community Learning and Information Centers (CLIC) project. He also worked with USAID East Africa an Inter-Country effort to explore how ICT could be used to help support peace. 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OFANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN KENYA. He was recruited by David Townsend. He and David worked together to assess how Kenya’s USAF situation could be best supported. This involved determining where the country was currently in Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 252 USAF and identifying any gaps needing addressing to move forward. Later, he and David worked together supporting Kenya’s development of the National Broadband Strategy (NBS). 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? This question was not asked, as it was not relevant to Mr. Mureithi’s involvement in the project. A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? He stated that there were two aspects of Integra’s work in Kenya: First – The NBS launched in May 2013 – with a schedule to implement the first phase of the strategy in five (5) years. The major impact has been on the mindset of stakeholders, i.e., recognition of the need to work together to accomplish the goals. The NBS achieved “mainstreaming” of ICT – bringing it into the national awareness. The launch was done by the Deputy President, and the NBS also was able to establish the linkage of ICT to other national programs. Second – The universal service program had challenges, primarily the delay in establishing the Universal Service Advisory Council. The telecom operators wanted to be included in the advisory council. From 2007 (when the structure was created) until 2013, CCK was supposed to be collecting universal service assessments from the operators, but that did not happen due to the challenges to the authority and structure of the regulatory system. 3. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? This question was not asked. A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? The bureaucracy caused delay by CCK in getting the MOU done. This was attributed to the fact that the MOU was to be an agreement between federal governments (Kenya and the U.S.), which raised the process to a higher political level. When asked why a MOU was necessary, he did not provide a direct answer. He then stated that it took about a year to get the MOU approved. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 253 He said that CCK has now put lots of resources on universal access; has been getting outside expertise, and that the approach of working together has been meaningful. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? He stated that consideration needs to be given to the need to manage the relationships between USAID and recipient organizations at all levels. The rotation of people in various positions can cause problems unless the relationships are “refreshed” on an ongoing basis. In some cases, the top levels involved sign off on an item, but lower level relationships do no always get established. 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? Questions 7, 8 and 9 were not asked. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 254 SITE VISIT INTERVIEW –KENYA - USAF PROJECT ID INFORMATION: NAME: Nicholas Nesbitt, OGW POSITIION: Country General Manager (formerly CEO of KenCall) ORGANIZATION: IBM DATE OF INTERVIEW: 23 January 2014 INTERVIEWER: W. Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN￾DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: Mr. Nesbitt was not contacted or involved in any way with the Integra project. Therefore, this interview guide was not used. Given his background in ICT, he did offer the following information: When he was CEO of KenCall, he helped Kenya to do outsourcing and convinced Kenya to do broadband access out to the country-side. He was the first proponent for fibre in Kenya, and was visited by many of the stakeholders in connection with that interest. In the last two or three years, he has not had any contact with CCK. He is currently working with others on how to get users on-board in using ICTs. He is the Chairman of the American Chamber of Commerce in Nairobi (and is a native Kenyan). He suggested contacting the following two people in connection with universal access and ICT: Victor Kyalo – email: vkyalo@ict.go.ke, cell: 0722-316-216. Victor in an ICT expert and is now the CEO of the ICT Authority. The authority brings together different agencies in the various ministries pursue ICT strategies. The effort is funded by the World Bank, and has brought fibre to the shore and other efforts. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 255 Dr. Kate Getao – email: kgetao@ict.go.ke, cell: 0722-376-659. She was formerly the Head of E￾government group in the Ministry, and is now still focusing on E-government, but at the ICT Authority. 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN KENYA. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 256 SITE VISIT INTERVIEW – KENYA - USAF PROJECT ID INFORMATION: NAME: Susan N. Mochache POSITIION: Assistant Director Universal Service Obligation ORGANIZATION : Communications Commission of Kenya (CCK) DATE OF INTERVIEW: 22 January 2014 INTERVIEWER: W. Kent Edwards INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIES IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: Ms. Mochache was joined for the interview by Mr. Godfrey Muhatia – an Assistant Manager in her organization. She said that Paul Kiage would have been present, but was on leave. She also mentioned that she had worked with the local USAID people closely, and mentioned M. Jones and Tom Muga (whom she thinks was transferred out of Kenya). She said that later on she also had worked with Albert Waudo. 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT, OR OF ANY KNOWLEDGE YOU MAY HAVE OF THE USAF PROJECT ACTIVITIES IN KENYA. She met with Integra in 2010 at the Intel sponsored workshop in Nairobi. She indicated that she made the “first move” by asking Eric White of what Integra’s project could do for Kenya. 2. WERE YOU CONTACTED BY THE IMPLEMENTATION TEAM TO? A. INFORM YOU OF THEIR PRESENCE AND PLANNED ACTIVITIES See her comments in response to Question 1, above. B. DISCUSS POTENTIAL SYNERGIES WITH OTHER PROGRAMS THAT YOU MAY HAVE HAD GOING OR PLANNED? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 257 She saw that Integra’s expertise could help in Kenya. Kenya had started work on the National Broadband Strategy (NBS), but the efforts were “lethargic.” David Townsend came in and provided a “grounded” perspective and brought up areas that they had not previously considered. 3. WHAT IS YOUR OPINION OF THE IMPACT OF THE USAF PROGRAM THUS FAR IN YOUR COUNTRY? One of the “pluses” was that the government learned that it could get acceptance by involving stakeholders – rather than just trying to impose a program or policy. The consultation efforts resulted in making the NBS a document of the people of Kenya. She said that they were very fortunate as a country to have received the assistance from Integra. Integra provided them with the foundational framework and an organized workplan. She also thinks the NBS success was due in part to having gotten the industry on board, and said that the launch meeting for NBS had been “totally full.” She indicated that CCK has been able to share its experience with other countries. She cited the Eastern Africa Communications Organization (Kenya, Uganda, South Sudan, Tanzania, and Burundi) in that regard. She also mentioned sharing with Mr. Mfugameha from TCRA (Tanzania) who is responsible for competition and economics at TCRA. When asked why the MOU process took so long, she responded that the “government-to￾government” dimension was the reason for the delay. 4. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. She said that mostly other ministries have been involved – and she said that she was thinking about the NBS in that statement. The NBS has raised ICT on the agenda of people in Kenya. With the governmental devolution into counties, she has met with county government representatives. She said that, afterwards, some county people went back to their counties and announced the elimination of fees on infrastructure. NBS has a ten-member (10) steering committee and has created a technical working group which has weekly workshops. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER Intel has invited them to India to share their experiences (she mentioned Ralph Corry (Sp?) and John Roman). The Universal Service Fund – They have now been able to organize the fund; having the operational framework in place has helped, and Integra was very involved in developing the framework. The two to three days spent in stakeholder sharing with the industry (by David Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 258 Townsend) was valuable. With the framework in place, her organization has been able to adequately respond to all of the Board’s questions, and having Parvez’s support really helped in that area. Further, David Townsend helped them identify gaps, and now CCK is doing another gap study – and this time, they know where to start. 5. TO THE EXTENT OF YOUR AWARENESS OR INVOLVEMENT IN THE PROJECT, CAN YOU HIGHLIGHT FOR ME ANY OBSTACLES YOU BELIEVE WERE ENCOUNTERED BY THE IMPLEMENTERS IN ACHIEVING THEIR PROJECT OBJECTIVES? She felt that the delays had been the major obstacles, and stated that there was a need to extend the project (Integra’s involvement). I asked when CCK will start collecting money from the operators. She said that they got the “OK” from the Board in December 2013 to start collecting, and that they have now issued invoices to the telecom, cellular and cable TV companies for the period through July 2014. The fees will be billed on an annual basis. CCK is holding off on billing the broadcasters due to the ongoing legal challenges that have been brought by the broadcasters. She also indicated that the current pilot project was funded by CCK. 6. WHAT REACTIONS OR ADVICE WOULD YOU LIKE TO OFFER TO USAID IN DESIGNING AND IMPLEMENTING PROGRAMS/PROJECTS SUCH AS THE ONE WE HAVE BEEN DISCUSSING? She thought that USAID needs to gain better understanding that things move slowly in Africa, i.e., to be more aware of the cultural differences in Africa. She felt that Integra’s project was a success, but expressed uncertainty about whether or not it would be extended by USAID. The result of the uncertainty has been “a lot of unnecessary anxiety.” 7. DO YOU THINK THAT THERE ARE SYNERGIES THAT SHOULD BE TAKEN INTO ACCOUNT PRIOR TO LAUNCHING SUCH PROGRAMS? IF SO, CAN YOU EXPLAIN IN MORE DETAIL? This question was not asked. 8. IS THERE ANYTHING ELSE THAT YOU WOULD LIKE TO SHARE ABOUT YOUR ASSESSMENT OF THE USAF PROJECT? She stated that involvement with Integra has been a good experience, and that their having had a good local consultant really helped, as it was valuable in enabling them to engage the various local stakeholders. 9. ARE THERE ANY OTHER PEOPLE YOU WOULD RECOMMEND INTERVIEWING IN CONNECTION WITH THIS PROJECT? IF SO, CAN YOU ASSIST ME BY PROVIDING THEIR COORDINATES AND POSSIBLY MAKING AN ADVANCE CONTACT? This question was not asked. THANK YOU FOR GIVING ME YOUR TIME FOR THIS INTERVIEW. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 259 UNITED STATES – USAID INTERVIEW– USAID USAF EVALUATION PROJECT USAID ID INFORMATION: NAME: DARRELL OWEN POSITIION: CONSULTANT ORGANIZATION USAID DATE OF INTERVIEW: 22 APRIL 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I NEED TO INTERVIEW YOU. AS YOU KNOW, USAID HIRED DTS TO CONDUCT AN EVALUATION OF THE PERFORMANCE OF INTEGRA IN ASSISTING SSA COUNTRIES IN MOVING FORWARD IN UNIVERSAL ACCESS TO BROADBAND AND ICT SERVICES. TO BE AS THOROUGH AS POSSIBLE IN THE EVALUATION EFFORT, WE ARE INTERVIEWING AS MANY PEOPLE AS POSSIBLE WHO WERE INVOLVED IN THE PROJECT. SINCE OUR EVALUATION IS LIMITED TO INTEGRA’S WORK UNDER TASK ORDER 1 OF THE GBI PROJECT, WE ARE FOCUSING ON THE FOUR COUNTRIES WHICH INTEGRA SELECTED FOR DIRECT TECHNICAL ASSISTANCE. THE COUNTRIES WERE: KENYA, NIGERIA, GHANA AND TANZANIA. QUESTIONS: 1. WHAT WAS YOUR SPECIFIC ROLE IN THE USAF PROJECT (TASK ORDER 1)? Judith Payne arranged the task order funding with the USAID African Bureau. Darrell arranged with Intel for USAID to participate in regional workshop. History of his involvement is that Joe Duncan asked him about four (4) years ago to help design the GBI initiative. Joe wanted a “field designed” project. Based on the Viet Nam successful experience, Joe wanted to help countries get USAF up to speed to catch up with the mobile phone explosion worldwide. Per Darrell, some things were not factored into the design of GBI:  In Viet Nam the timing was propitious relative to their stage of development (He mentioned that he had recruited David Townsend there.).  Getting the government and private sectors together  Using USF as capital = national backbone: Who is going to invest in 3G & LTE in rural areas, and same for Internet. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 260 He mentioned the National Broadband Strategy (NBS) in Kenya was time consuming, but was important as a part of the country’s development, and per David Townsend, the NBS should have a “long tail,” in that other countries will potentially adopt similar efforts. He stated that he thought that at least three (3) of the target countries advanced in USAF methods as result of the GBI. 2. WHAT WERE THE SPECIFIC OBJECTIVES OF THE PROJECT? Mr. Owen said that the objective was to provide technical assistance to enable countries to “get up and running” or to improve their USAF functions. He recalled that there was some element included to help in SSA on a regional basis. He thought that Judith Payne allowed other funds to supplement the funding by the African Bureau. Also, he said that he got some sense that other commercial organizations may have been involved in funding, especially in Nigeria. 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN EACH OF THE COUNTRIES? This question was not specifically asked. See responses to 4, following. 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN EACH COUNTRY? He thinks that Integra’s being able to get top notch resources such as David Townsend and Parvez Iftikhar contributed significantly to the project’s effectiveness. He also said that he thinks that David Townsend believes that there is a need to keep involved with the countries to maintain the progress. 5. IF INTEGRA WAS NOT ABLE TO MEET THEIR OBJECTIVES OR USAID’S OBJECTIVES, WHY DO YOU BELIEVE THIS HAPPENED? WHAT COULD INTEGRA HAVE DONE TO OVERCOME THIS SITUATION? Shifting politics, especially in Ghana, presented obstacles. 6. CAN YOU TELL ME, IN THE FOUR COUNTRIES, WHAT ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? This question was not asked, as Mr. Owen was not directly involved in the in-country execution of the project. A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 7. ARE YOU AWARE OF ANY SPECIFIC OBSTACLES ENCOUNTERED BY INTEGRA IN ANY OF THE FOUR COUNTRIES – OR OTHERWISE DURING THE TASK ORDER 1 TIME FRAME? See response to 5, above. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 261 8. ARE THERE ANY OTHER COMMENTS OR THOUGHTS RELATING TO THE TASK ORDER 1 USAF PROJECT THAT YOU WOULD LIKE TO MENTION? He thinks the timing is critical now to support continued efforts in the countries to improve USAF execution. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 262 INTERVIEW– USAID USAF EVALUATION PROJECT USAID ID INFORMATION: NAME: JEFFREY A. COCHRANE, PhD POSITIION: DIVISION CHIEF – ICT ORGANIZATION USAID DATE OF INTERVIEW: 24 APRIL 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I NEED TO INTERVIEW YOU. AS YOU KNOW, USAID HIRED DTS TO CONDUCT AN EVALUATION OF THE PERFORMANCE OF INTEGRA IN ASSISTING SSA COUNTRIES IN MOVING FORWARD IN UNIVERSAL ACCESS TO BROADBAND AND ICT SERVICES. TO BE AS THOROUGH AS POSSIBLE IN THE EVALUATION EFFORT, WE ARE INTERVIEWING AS MANY PEOPLE AS POSSIBLE WHO WERE INVOLVED IN THE PROJECT. SINCE OUR EVALUATION IS LIMITED TO INTEGRA’S WORK UNDER TASK ORDER 1 OF THE GBI PROJECT, WE ARE FOCUSING ON THE FOUR COUNTRIES WHICH INTEGRA SELECTED FOR DIRECT TECHNICAL ASSISTANCE. THE COUNTRIES WERE: KENYA, NIGERIA, GHANA AND TANZANIA. QUESTIONS: Dr. Cochrane made it clear at the beginning of the interview that, formally, the task order being evaluated belongs to the African Bureau, not the E3-ICT management. He said that the report will administratively go to the African Bureau. 1. WHAT WAS YOUR SPECIFIC ROLE IN THE USAF PROJECT (TASK ORDER 1)? He indicated that he joined the E3 organization about a year and one-half ago, and that his role has been the overall coordination of his division’s work. In that regard, the GBI initiative includes four organizations (Integra, NetHope, Technology Leadership Program @ U.S. State Department, and World Learning). He had observed that the program was not very well coordinated among these four groups, so that he mandated that Integra talk to the other program players. Since that time, he has required once weekly meetings to ensure that all parties are kept up-to-date, although he indicated that State does not always attend, given its broad policy role. He also said that he had encouraged Judith Payne, Dan Kingsley and Bob Otto to figure out how to get the country governments to use the universal service/access funds to “help poor people.” He also indicated that he observed that the COR seemed not to be involved in Integra’s work closely enough, that the project was managed from “a distance.” As a result, he made a personnel change. 2. WHAT WERE THE SPECIFIC OBJECTIVES OF THE PROJECT? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 263 He said that he had “no idea, other than the TOR.” He indicated that he had read the TOR and also “hears what other people say” about the project, i.e., that some are disappointed in the overall results of the program. 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN EACH OF THE COUNTRIES? He stated that he believed Integra did good work in Kenya and Nigeria. 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN EACH COUNTRY? This question was not asked. 5. IF INTEGRA WAS NOT ABLE TO MEET THEIR OBJECTIVES OR USAID’S OBJECTIVES, WHY DO YOU BELIEVE THIS HAPPENED? WHAT COULD INTEGRA HAVE DONE TO OVERCOME THIS SITUATION? This question was not asked. 6. CAN YOU TELL ME, IN THE FOUR COUNTRIES, WHAT ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? This question was not asked. A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 7. ARE YOU AWARE OF ANY SPECIFIC OBSTACLES ENCOUNTERED BY INTEGRA IN ANY OF THE FOUR COUNTRIES – OR OTHERWISE DURING THE TASK ORDER 1 TIME FRAME? This question was not asked. 8. ARE THERE ANY OTHER COMMENTS OR THOUGHTS RELATING TO THE TASK ORDER 1 USAF PROJECT THAT YOU WOULD LIKE TO MENTION? This question was not asked. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 264 INTERVIEW– USAID USAF EVALUATION PROJECT USAID ID INFORMATION: NAME: JOE DUNCAN POSITIION: ACTING TEAM LEADER E3-ICT (DURING PROJECT INCEPTION) & ALTERNATE COR FOR THE PROJECT ORGANIZATION USAID DATE OF INTERVIEW: 24 APRIL 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I NEED TO INTERVIEW YOU. AS YOU KNOW, USAID HIRED DTS TO CONDUCT AN EVALUATION OF THE PERFORMANCE OF INTEGRA IN ASSISTING SSA COUNTRIES IN MOVING FORWARD IN UNIVERSAL ACCESS TO BROADBAND AND ICT SERVICES. TO BE AS THOROUGH AS POSSIBLE IN THE EVALUATION EFFORT, WE ARE INTERVIEWING AS MANY PEOPLE AS POSSIBLE WHO WERE INVOLVED IN THE PROJECT. SINCE OUR EVALUATION IS LIMITED TO INTEGRA’S WORK UNDER TASK ORDER 1 OF THE GBI PROJECT, WE ARE FOCUSING ON THE FOUR COUNTRIES WHICH INTEGRA SELECTED FOR DIRECT TECHNICAL ASSISTANCE. THE COUNTRIES WERE: KENYA, NIGERIA, GHANA AND TANZANIA. QUESTIONS: 1. WHAT WAS YOUR SPECIFIC ROLE IN THE USAF PROJECT (TASK ORDER 1)? Mr. Duncan was the COR for the GBI contract with Integra, whereas Ms. Payne was the COR for the Task Order 1 (USAF) project. 2. WHAT WERE THE SPECIFIC OBJECTIVES OF THE PROJECT? Mr. Duncan said that the objective was to work with regulators in SSA countries to improve existing universal service/access funds. He further said that Judith Payne had a component for the project to build a tool kit for use by countries. 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN EACH OF THE COUNTRIES? He said that Integra has done a good job overall, especially in Kenya. Neither USAID nor Integra knew that Integra would be doing the National Broadband Strategy (NBS) in Kenya, but he thinks that it was a good idea to support that effort. 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN EACH COUNTRY? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 265 He said that Integra did a good job meeting the objectives. He elaborated that the genesis of the entire GBI idea was to try to replicate the success experienced in the Viet Nam project, but that the situation in the SSA countries turned out to be different from Viet Nam. 5. IF INTEGRA WAS NOT ABLE TO MEET THEIR OBJECTIVES OR USAID’S OBJECTIVES, WHY DO YOU BELIEVE THIS HAPPENED? WHAT COULD INTEGRA HAVE DONE TO OVERCOME THIS SITUATION? He said that the objectives need to be flexible, so that USAID should not “close the door” because the situation and/or needs in a country turned out not to fit the terms of the project TOR. Reprioritizing efforts in Kenya to support the NBS development was an example of this type of needed flexibility. 6. CAN YOU TELL ME, IN THE FOUR COUNTRIES, WHAT ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? This question was not asked. A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 7. ARE YOU AWARE OF ANY SPECIFIC OBSTACLES ENCOUNTERED BY INTEGRA IN ANY OF THE FOUR COUNTRIES – OR OTHERWISE DURING THE TASK ORDER 1 TIME FRAME? Mr. Duncan indicated that his focus had been primarily on monitoring the project activities in Kenya, so that he was not in a position to address the situations encountered in the other three countries. 8. ARE THERE ANY OTHER COMMENTS OR THOUGHTS RELATING TO THE TASK ORDER 1 USAF PROJECT THAT YOU WOULD LIKE TO MENTION? This question was not asked. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 266 INTERVIEW– USAID USAF EVALUATION PROJECT USAID ID INFORMATION: NAME: JUDITH PAYNE POSITIION: CONTRACT OFFICER REPRESENTATIVE (FORMER) USAF PROJECT ORGANIZATION USAID DATE OF INTERVIEW: 23 APRIL 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I NEED TO INTERVIEW YOU. AS YOU KNOW, USAID HIRED DTS TO CONDUCT AN EVALUATION OF THE PERFORMANCE OF INTEGRA IN ASSISTING SSA COUNTRIES IN MOVING FORWARD IN UNIVERSAL ACCESS TO BROADBAND AND ICT SERVICES. TO BE AS THOROUGH AS POSSIBLE IN THE EVALUATION EFFORT, WE ARE INTERVIEWING AS MANY PEOPLE AS POSSIBLE WHO WERE INVOLVED IN THE PROJECT. SINCE OUR EVALUATION IS LIMITED TO INTEGRA’S WORK UNDER TASK ORDER 1 OF THE GBI PROJECT, WE ARE FOCUSING ON THE FOUR COUNTRIES WHICH INTEGRA SELECTED FOR DIRECT TECHNICAL ASSISTANCE. THE COUNTRIES WERE: KENYA, NIGERIA, GHANA AND TANZANIA. QUESTIONS: The interview with Ms. Payne was relatively brief, and the following interview guide questions were not specifically asked. Following is a summary of her comments about the project’s execution: She had wanted more of a regional impact from the project, and included such requirement in the TOR she prepared. She indicated that she encouraged Integra to expand the project beyond the four specifically targeted countries (Nigeria, Kenya, Tanzania and Ghana). With regard to budget, she indicated that she had required Integra to stop funding the “learning curve” to bring Mr. White up to speed on the project. In sum, Ms. Payne thought that Integra had not been innovative enough in finding new approaches that would have more of a regional impact. She specifically mentioned as an example – “How to get countries to pay for some of David Townsend’s time,” etc. She said that she would give Integra a “C￾plus or B-minus” for performance on the project. 1. WHAT WAS YOUR SPECIFIC ROLE IN THE USAF PROJECT (TASK ORDER 1)? 2. WHAT WERE THE SPECIFIC OBJECTIVES OF THE PROJECT? 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN EACH OF THE COUNTRIES? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 267 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN EACH COUNTRY? 5. IF INTEGRA WAS NOT ABLE TO MEET THEIR OBJECTIVES OR USAID’S OBJECTIVES, WHY DO YOU BELIEVE THIS HAPPENED? WHAT COULD INTEGRA HAVE DONE TO OVERCOME THIS SITUATION? 6. CAN YOU TELL ME, IN THE FOUR COUNTRIES, WHAT ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 7. ARE YOU AWARE OF ANY SPECIFIC OBSTACLES ENCOUNTERED BY INTEGRA IN ANY OF THE FOUR COUNTRIES – OR OTHERWISE DURING THE TASK ORDER 1 TIME FRAME? 8. ARE THERE ANY OTHER COMMENTS OR THOUGHTS RELATING TO THE TASK ORDER 1 USAF PROJECT THAT YOU WOULD LIKE TO MENTION? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 268 INTERVIEW– USAID USAF EVALUATION PROJECT USAID ID INFORMATION: NAME: MCDONALD HOMER POSITIION: DIRECTOR – AFRICAN BUREAU ORGANIZATION USAID DATE OF INTERVIEW: 30 MAY 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I NEED TO INTERVIEW YOU. AS YOU KNOW, USAID HIRED DTS TO CONDUCT AN EVALUATION OF THE PERFORMANCE OF INTEGRA IN ASSISTING SSA COUNTRIES IN MOVING FORWARD IN UNIVERSAL ACCESS TO BROADBAND AND ICT SERVICES. TO BE AS THOROUGH AS POSSIBLE IN THE EVALUATION EFFORT, WE ARE INTERVIEWING AS MANY PEOPLE AS POSSIBLE WHO WERE INVOLVED IN THE PROJECT. SINCE OUR EVALUATION IS LIMITED TO INTEGRA’S WORK UNDER TASK ORDER 1 OF THE GBI PROJECT, WE ARE FOCUSING ON THE FOUR COUNTRIES WHICH INTEGRA SELECTED FOR DIRECT TECHNICAL ASSISTANCE. THE COUNTRIES WERE: KENYA, NIGERIA, GHANA AND TANZANIA. QUESTIONS: 1. WHAT WAS YOUR SPECIFIC ROLE IN THE USAF PROJECT (TASK ORDER 1)? The USAF project was funded out of his office and he took on responsibilities for monitoring and overall supervision. The project had been started under his predecessor’s (Bruno Cornelius) tenure. 2. WHAT WERE THE SPECIFIC OBJECTIVES OF THE PROJECT? His bureau wanted to look at ways to unlock economic growth, especially in Africa. They were interested in seeing how broadband could be expanded to benefit large and small businesses to stimulate economic growth. It appeared that there was a need to change telecom policies, to get reforms. Unlocking access to broadband appeared to need improved universal service functioning. The overarching goal behind the project was to stimulate economic development. 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN EACH OF THE COUNTRIES? He stated that the project had a great impact on the countries, especially in Kenya where working with the regulator helped open up the universal service fund operations. He also Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 269 stated that funds had been underutilized in the past. Given his past experiences at USAID, he recognizes that policy reform takes significant amounts of time, but he was impressed at how quickly Integra was able to achieve policy changes during the project. He said that it was perhaps due to the urgency of demand for telecom services that things could be moved more quickly than in some other sectors. He summarized his view of the impact of the USAF project as having been effective in getting policy reform and universal funds moving forward. He also indicated that USAID is interested in considering expanding this type of effort or program to other countries. 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN EACH COUNTRY? This question was not specifically asked, as the response to 3, above, seemed to cover it. 5. IF INTEGRA WAS NOT ABLE TO MEET THEIR OBJECTIVES OR USAID’S OBJECTIVES, WHY DO YOU BELIEVE THIS HAPPENED? WHAT COULD INTEGRA HAVE DONE TO OVERCOME THIS SITUATION? He did not cite any specific obstacles encountered in the four countries. He did, however, mention that there is a more broad issue that has an impact on many centrally funded projects (USAID –W funded projects). That issue is that the particular project in question may not be a current priority in the missions, so that it can be difficult to either get the missions attention or active involvement. This is an area where USAID need to figure out how to improve the processes. 6. CAN YOU TELL ME, IN THE FOUR COUNTRIES, WHAT ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? This question was not asked. A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 7. ARE YOU AWARE OF ANY SPECIFIC OBSTACLES ENCOUNTERED BY INTEGRA IN ANY OF THE FOUR COUNTRIES – OR OTHERWISE DURING THE TASK ORDER 1 TIME FRAME? He did not recall any specific obstacles that had been brought to his attention. 8. ARE THERE ANY OTHER COMMENTS OR THOUGHTS RELATING TO THE TASK ORDER 1 USAF PROJECT THAT YOU WOULD LIKE TO MENTION? He said that his overall rating of Integra’s performance on this project is Outstanding. He further stated that he would welcome having this same team work on future projects. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 270 He indicated that his comments during this interview were based on recall, and that he would try to take a look at the Integra report to see if anything else comes to mind. If so, he indicated that he would follow up via an email to me. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 271 UNITED STATES – INTEGRA INTERVIEW– USAID USAF EVALUATION PROJECT INTEGRA ID INFORMATION: NAME: DAVID TOWNSEND POSITIION: CONSULTANT ORGANIZATION INTEGRA DATE OF INTERVIEW: 13 MAY 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIE IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA OR KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: Because of Mr. Townsend’s involvement in the technical assistance provided to the entire USAF Project, his involvement in each of the four target countries is described separately, generally following the interview questions below. KENYA 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT IN KENYA. Kenya was the first SSA engagement for the GBI, so that it was a “bit of trial and error.” At an Intel workshop, GBI was approached by Susan Mochache, who asked Eric White for help. The GBI plan was to go into Kenya and help them set up the universal service plan. However, he thinks that the Permanent Secretary of the Ministry had decided that the country need a national broadband strategy, and that the task of developing that strategy was given to CCK, where it was subsequently assigned to Susan Mochache. She then asked Integra for help in Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 272 developing what was to become the National Broadband Strategy (NBS). Another consultant had already done a GAP analysis for CCK. Someone involved in the negotiations came up with the idea that a memorandum of understanding (MOU) would be necessary, as the work was considered as “government to government.” It took a significant amount of time to get the MOU approved, but the actual substance never really changed during the process. The period of time taken up in the approval negotiations/process was from March or April of 2011 until June of 2012, almost an entire year. Although he did a lot of the work before the actual MOU was approved, after the signing, the NBS effort became the total focus for quite a while. In order to get the NBS “off the ground.” He had some ideas, and provided CCK with a framework for a participatory process. This included creating an NBS steering committee, which Susan Mochache then organized. Having Mr. Muruiki on board was a real advantage, as he acted as the liaison and helped to keep the process moving forward. As part of the participatory process, CCK organized a retreat which was attended by more than fifty (50) stakeholders. A local professor and students were hired to help run the retreat, which lasted for five (5) days. After the NBS was approved, Parvez Iftikhar was brought in and did more stakeholder work. He also provided the fund framework and the design for a pilot project. 2. WHAT WERE THE SPECIFIC OBJECTIVES IN KENYA? Integra came up with an initial agreement with CCK that had three components:  To help with the NBS  To help with universal access fund planning  To help with implementation of the USAF plan 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN KENYA? Mr. Townsend noted that after the new president came into office, a project was developed to buy lots of computers for schools (part of a campaign promise). The RFP totally ignored the whole NBS process, and both iterations of the RFP failed. 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN KENYA? This question was not asked. 5. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 273 D. USAID E. OTHER – As mentioned above, the participatory process adopted for the NBS development included 50 attendees at the 5-day retreat. We are aware from other interviews that this group was comprised on representatives from varying stakeholder interest groups. 6. CAN YOU ELABORATE ABOUT THE OBSTACLES THAT YOU ENCOUNTERED IN KENYA? AND, WHAT DID YOU OR OTHERS ON THE INTEGRA TEAM DO TO OVERCOME THOSE OBSTACLES? His view of the obstacles encountered in Kenya included:  The MOU process – delay.  Delay in appointing the Advisory Council  The National elections  National devolution, which raised such questions as whether or not CCK could spend different amounts in the counties.  Creation of the ICT Authority, which caused some amount of friction between CCK and the Authority TANZANIA 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT IN TANZANIA. Tanzania was targeted for potential assistance via the selection criteria developed by Integra. He had met Peter Ulanga, who appeared to be responsive and enthusiastic – and was also the chairperson of AUSAFA. Integra sent Kojo Boyak to Dar es Salaam to get the “lay of the land.” The World Bank had funded a project in Tanzania which also ran into problems. It was just Peter and some consultants trying to complete the project. 2. WHAT WERE THE SPECIFIC OBJECTIVES IN TANZANIA? A TOR was agreed, including:  To help them design a strategic plan (similar to GBI work in other countries)  To help them develop a market assessment  To provide implementation support 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN TANZANIA? Integra produced some documents to support the strategic plan development. It also developed a template for a market assessment. 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN TANZANIA? Because of the bureaucracy in the country, things did not go anywhere. Peter never got any staff during the life of task order 1. Eventually people were brought on board, but it was too late for the project. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 274 5. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? This question was not asked. A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 6. CAN YOU ELABORATE ABOUT THE OBSTACLES THAT YOU ENCOUNTERED IN TANZANIA? AND, WHAT DID YOU OR OTHERS ON THE INTEGRA TEAM DO TO OVERCOME THOSE OBSTACLES? See response to 4, above. GHANA 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT IN GHANA. Mr. Townsend had done lots of work in Ghana and was working with GIFEC to develop a strategic plan. At the same time, he was trying to convince the Ghanaians to consider receiving technical assistance from GBI. The Chief Director (Kwaku Ofusu Adakwa) was contacted by Kojo Boyak (again, David had worked with the Chief Director previously). They asked him what would be of value to Ghana, and he came back with ideas which GBI then adopted. 2. WHAT WERE THE SPECIFIC OBJECTIVES IN GHANA? The objectives included:  To help answer the question of how to stimulate demand for broadband after the infrastructure has been put in place; what can be done about that, and what would be policy recommendations.  To support a digital divide analysis. The TOR distinguished between the market analysis and the GAP analysis. Ultimately, what was done was only the demand side. 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN GHANA? This question was not asked. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 275 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN GHANA? He said that the demand study, which used the questionnaire developed by Integra, was good, and that it had received some 5,000 responses. He and Eric White took the survey output (which had somehow been obtained) and provided GIFEC with an improved survey results report. Mr. Townsend did the survey analysis and report work on a pro bono basis due to the project having met its terminus date. 5. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? This question was not asked. A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID E. OTHER 6. CAN YOU ELABORATE ABOUT THE OBSTACLES THAT YOU ENCOUNTERED IN GHANA? AND, WHAT DID YOU OR OTHERS ON THE INTEGRA TEAM DO TO OVERCOME THOSE OBSTACLES? Completion of the survey took about a year. GBI had developed the questionnaire for the survey and GIFEC paid for the actual survey work. Getting additional data was a problem. A GIFEC person was transferred to the NCA and was supposed to obtain the data, but never delivered. David Townsend learned later during a different (non￾GBI project) that NCA does not collect the information that was supposed to have been received by GIFEC. ISSER (which was the organization hired by GIFEC to implement the survey) did not was to give GBI the results of the survey and produced its own report, which was of low quality. NIGERIA 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT IN NIGERIA. Integra first met with Dr. Akwule, who led them to the Minister, then to NCC and ultimately to USPF. Mr. Maikano had just been appointed to head USPF. The former head of USPF had been to some of the regional workshops, but was gone when the GBI work in Nigeria started. The NCC Executive VP (Director General) was new and did not initially “warm” to GBI, but later agreed to the work. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 276 2. WHAT WERE THE SPECIFIC OBJECTIVES IN NIGERIA? Integra had to adjust the objectives in Nigeria a couple of times. Originally, the objective was to just be there and provide input as needed. Also, Integra was supposed to do a gap analysis, but USPF had already hired a firm to do that study, so that GBI was to review when completed. At the time, USPF was in the process of developing their Strategic Management Plan (SMP), and GBI offered to help in that effort. USPF had scheduled a retreat to work on the SMP. He and Parvez Iftikhar attended and took part in the retreat. The sessions went from 0900 until 2000 hours for three days, and were attended by twenty (20) or so people. Local KPMG and some Indian KPMG people ran the retreat, and there was another company studying what had been done in the past. 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN NIGERIA? During the retreat, it became clear to him that lots of people at the retreat did not know what was going on. With his and Parvez’s contributions, the three-day retreat produced the SMP, 70% of which was GBI inputs. Integra also did some GAP analysis work in Nigeria. The final piece of work was assistance in developing the M&E process. At that time, only 50% of the USPF projects were still in operation. 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN NIGERIA? See response to 3, above. 5. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID - When asked about USAID’s role, he said that he had found it a bit confusing. A monthly report on Integra’s work went to Judith Payne, but they received little feedback and a few questions. Only when the Tanzania and Ghana problems arose was there much involvement. E. OTHER 6. CAN YOU ELABORATE ABOUT THE OBSTACLES THAT YOU ENCOUNTERED IN KENYA? AND, WHAT DID YOU OR OTHERS ON THE INTEGRA TEAM DO TO OVERCOME THOSE OBSTACLES? Not really, except for the quality of in-country “experts.” For example, a data project had been done and had lots of maps, but zero broadband coverage information. ALL FOUR COUNTRIES Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 277 7. IS THERE ANYTHING THAT YOU WOULD WANT TO RECOMMEND THAT USAID CONSIDER DOING DIFFERENTLY IN DESIGNING AND IMPLEMENTING PROGRAMS SUCH AS THE TASK ORDER 1 EFFORT?  Given the Kenya experience, avoid MOU arrangements that could lead to significant delays.  There is a need to be both flexible and to have a “long view.”  Projects need to include steps recognizing how to keep things going in countries receiving assistance. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 278 INTERVIEW– USAID USAF EVALUATION PROJECT INTEGRA ID INFORMATION: NAME: PARVEZ IFTIKHAR POSITIION: CONSULTANT ORGANIZATION INTEGRA DATE OF INTERVIEW: 10 APRIL 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIE IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA OR KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: Mr. Iftikhar was interviewed in Pakistan via Skype. 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT IN EACH OF THE FOUR COUNTRIES – KENYA, TANZANIA, NIGERIA AND GHANA. Mr. Iftikhar said that he only worked in Kenya and Nigeria. In Kenya, he joined the effort after the National Broadband Strategy was completed. In Nigeria, he joined the project at the KPMG led retreat. The rest of the interview addressed the two countries separately, as follows: NIGERIA 2. WHAT WERE THE SPECIFIC OBJECTIVES IN NIGERIA? He said that he was quite impressed by the Minister being at the retreat for the entire time. The objective for him was to show them examples of best practices in universal access. He was able to help establish credibility for the Integra team by sharing his knowledge and personal experiences in the Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 279 universal access work in Pakistan. The work on the strategic plan came out of the retreat, and Integra supported that effort. On his second visit, the first objective was to address monitoring and evaluation (M&E). Nigeria wanted to establish an M&E group and needed support in deciding what specifically to do in that regard. They had received complaints from Parliament representatives about the telecenters, so that it was clear that there was a need for an improved M&E process. Parvez and David Townsend provided M&E training and solutions for the M&E efforts. The second focus of the visit was the nationwide fibre rollout (B-Train project), which is still in progress. Also, USPF had received a gap analysis study and asked Integra to comment on it. The study indicated where coverage did and did not exist in the country. Parvez said that he advised them that there were aspects which needed more clarification, and that they should go “step-by-step” geographically to understand the national situation. He felt that the gap analysis lacked some things that could have improved the analysis. 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN NIGERIA? This question was not specifically asked. 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN NIGERIA? Mr. Iftikhar said that the objectives in which he was involved were achieved; i.e., that Integra was able to meet its objectives very well in that regard. Since his time in country, he was not sure what Nigeria has done, but recalled that a pilot program was started and that they had adopted the process of bringing the telecom operators on board. 5. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. – No other ministries were involved, to his knowledge. The impact was primarily to USPF, the Ministry of Communications, the regulator and representatives from the incumbent telecom operator. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID – He was not aware of any impact or involvement by USAID in country. E. OTHER 6. CAN YOU ELABORATE ABOUT THE OBSTACLES THAT YOU ENCOUNTERED IN NIGERIA? AND, WHAT DID YOU OR OTHERS ON THE INTEGRA TEAM DO TO OVERCOME THOSE OBSTACLES? He stated that he was not aware of any obstacles, since the Nigerian had requested the assistance and were very keen on actually adopting improved management practices for the universal access programs. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 280 7. IS THERE ANYTHING THAT YOU WOULD WANT TO RECOMMEND THAT USAID CONSIDER DOING DIFFERENTLY IN DESIGNING AND IMPLEMENTING PROGRAMS SUCH AS THE TASK ORDER 1 EFFORT? Mr. Iftikhar stated that he did not think that USAID had seen the program through – that it needs follow-up. Integra should have gone back to Nigeria to determine what obstacles USPF had encountered, what lessons they had learned, etc. Some additional amount of reinforcement and “handholding” would be of significant value in cementing the improved practices. KENYA 2. WHAT WERE THE SPECIFIC OBJECTIVES IN KENYA? The Kenyans wanted to set up the universal service organization functions. Part of the objective was to support them in designing a pilot project. He thinks that CCK had received some money for that purpose from some donor. Regardless of other existing issues, CCK wanted to do a pilot project. CCK wanted and needed support in addressing the details of operating the fund including relationships and involvement of the vendors. Integra visited each of the vendors and took some ideas from them. Integra then prepared a paper which was shared with the vendors, who provided comments which were included in revisions to the paper. Integra prepared a pilot project concept plan wherein the connections would be via fibre to a telecenter. The last mile would be technology neutral. Parameters for services were stated, but not the specific technologies to be used. A general description of the telecenter (what should be in a telecenter) was prepared, including specifying the need for back-up power. It was recommended that the bid should include three (3) years free service to the telecenter, but that schools and government offices would need to pay for services (leaving it up to the local area to decide how payments would be done). The telecenter operator would also be allowed to sell services to residential customers. Criteria were agreed with CCK for selection of a pilot location using the same incremental approach that Integra had recommended in Nigeria. In Kenya, this included the requirement that the county government needed to be interested, meaning that it would contribute to the effort. CCK pointed out that any pilot project would be “political.” Integra then recommended having multiple pilots using different approaches. In developing the pilot recommendations, Integra talked to the Kenyan telecom incumbent (Orange), which operates the national optic fibre backbone. Integra also recommended that there be equal access by other ICT players, and met with other metropolitan type operators (no broadcasters were included). He said that the Postal people were very much interested in participating in a pilot. He said that they also met with courier companies – who did not express any interest, although Integra pointed out that the potential for developing “E-Addresses” could be of benefit to the courier companies. The concept paper was sent from the U.S. to CCK, after which there was some “back and forth” between the parties via email comments. Then, the mall bombing in Nairobi took place, so that things got delayed. He indicated that he had been aware of some changes in CCK, something about CCK being changed to be an “Authority.” Also, there was lots of discussion and debate with the head of CCK about including operators on the Advisory Council. The operators had complained about their lack of representation, Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 281 but CCK felt that it would be a conflict of interest to include them. Integra recommended that the operators have some level of representation. 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN KENYA? He said that CCK now has the structure and framework in place, including organization, rules, bidding process and selection criteria. He also provided them with specimen bidding documents. In his view, CCK now has a complete roadmap for a functioning USAF program. He thinks that it would have made a difference if Integra went back to Kenya to provide further follow-up support. 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN KENYA? Mr. Iftikhar stated that Integra achieved its objectives in Kenya “very well.” CCK had “zero” at the start of the assistance, and now has the complete functional roadmap. 5. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID – He said that Integra went to USAID office and briefed them, but saw little active interest. E. OTHER 6. CAN YOU ELABORATE ABOUT THE OBSTACLES THAT YOU ENCOUNTERED IN NIGERIA? AND, WHAT DID YOU OR OTHERS ON THE INTEGRA TEAM DO TO OVERCOME THOSE OBSTACLES? He felt that the delay of the pilot was due to some obstacles, but did not state what specific obstacles he meant, just that the pilot appears to be an issue at this time. He also said that the operators at first expressed some resistance, but that, when their impressions were corrected, were not as unhappy. He said that Safaricom in particular had some very “sharp” questions. 7. IS THERE ANYTHING THAT YOU WOULD WANT TO RECOMMEND THAT USAID CONSIDER DOING DIFFERENTLY IN DESIGNING AND IMPLEMENTING PROGRAMS SUCH AS THE TASK ORDER 1 EFFORT? As in Nigeria, Mr. Iftikhar stated that follow-up assistance needs to be provided to Kenya. Areas such as how to evaluate bids for pilot projects, contract negotiations, etc., need to be further supported. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 282 INTERVIEW– USAID USAF EVALUATION PROJECT INTEGRA ID INFORMATION: NAME: ERIC WHITE POSITIION: MANAGING ASSOCIATE ORGANIZATION INTEGRA DATE OF INTERVIEW: 21 MARCH 2014 INTERVIEWER: W. KENT EDWARDS INTRODUCTION: THANK YOU FOR TAKING TIME TO MEET WITH ME. LET ME BRIEFLY DESCRIBE THE REASON THAT I AM HERE AND WANT TO INTERVIEW YOU. THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAS CREATED THE GLOBAL BROADBAND & INNOVATION PROJECT TO SUPPORT AND STRENGTHEN COUNTRIE IN THEIR UNIVERSAL ACCESS PROGRAMS AND FUNDS. THE FIRM SELECTED TO IMPLEMENT THE PROGRAM IS INTEGRA, LLC. TANZANIA OR KENYA WAS ONE OF THE FOUR COUNTRIES SELECTED BY INTEGRA FOR IN-DEPTH TECHNICAL ASSISTANCE. USAID SELECTED MY COMPANY, DTS, TO DO A PROJECT EVALUATION OF THE INTEGRA’S WORK ON THE USAF PROJECT. THE PURPOSE OF THE EVALUATION IS TO ASSESS THE PERFORMANCE OF THE IMPLEMENTER AND TO DERIVE “LESSONS LEARNED” FOR POTENTIAL USE IN FUTURE USAID ACTIVITIES TO SUPPORT UNIVERSAL SERVICE/ACCESS PROGRAMS. QUESTIONS: The interview of Mr. White took place at Integra’s Washington, D.C. offices. Given his involvement in all four of the target countries, we agreed that we would review his activities country by country, using the interview guide questions, as appropriate. KENYA 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT IN KENYA. He attended the first Intel forum in Nairobi in 2010i, where universal service was the topic. David Townsend was the primary presenter in the program, providing the attendees with the basic framework for thinking about USF and facilitating discussions. Eric’s role was to network with the attendees, and in doing so he made them aware of the GBI program. He met Susan Mochache at the forum, and she expressed an interest in working with Integra. They agreed that some work would be done for Kenya, and B. Otto, D. Townsend and Eric went to Kenya in April 2011 to discuss potential assistance. CCK asked for support on the National Broadband Strategy (NBS). After the NBS work was completed, Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 283 Eric’s role was monitoring the project’s progress and preparing monthly status reports for USAID. He subsequently handed off this role to Kimberly Hamilton. 2. WHAT WERE THE SPECIFIC OBJECTIVES IN KENYA? Three areas of assistance were agreed. These were: (1) Assist CCK in developing the NBS (Phase 1), ( 2) Review of the GAP analysis, support development of a strategic plan and operations manual (Phase 2), and (3) Define programs/projects for universal access implementation and see them through to pilots. Because a Memorandum of Understanding (MOU) approach was taken (because it was viewed as a “country to country agreement), approximately a year’s time was required to reach a signed agreement (the MOU was signed in March 2012). Integra recruited Mr. Muriuki Mureithi, whose ICT knowledge and local contacts was invaluable to the project. The major technical assistance work was provided by David Townsend and M. Muruiki initially, followed by Parvez Iftikhar after the NBS phase was completed. 3. WHAT IS YOUR ASSESSMENT OF THE IMPACT OF THE USAF PROGRAM IN KENYA? The majority of assistance time and budget was spent on the NBS development, so that, with CCK’s agreement, other deliverables got less attention. 4. HOW WELL WAS INTEGRA ABLE TO MEET ITS OBJECTIVES IN KENYA? The NBS was a major event in Kenya, and Integra met its objective on NBS. The GAP Analysis review was also completed. The other deliverables were continued under a different task order. 5. CAN YOU TELL ME WHICH ORGANIZATIONS AND PEOPLE WERE INVOLVED OR SAW IMPACT FROM THE PROJECT? He stated that there was a significant impact on most stakeholders in the ICT arena through the adoption of the NBS as a national, cabinet level commitment. A. MINISTRIES, AGENCIES, ETC. B. NGO’S C. OTHER USG ORGANIZATIONS D. USAID - Eric thought that USAID got credit for the NBS, but had minimal involvement. USAID was looking for immediate impact on USAID projects. He indicated that Mervyn Farroe definitely understood the value of the GBI initiative, but he was not sure that Mervyn was able to get USAID management “on board.” E. OTHER - 6. CAN YOU ELABORATE ABOUT THE OBSTACLES THAT YOU ENCOUNTERED IN KENYA? AND, WHAT DID YOU OR OTHERS ON THE INTEGRA TEAM DO TO OVERCOME THOSE OBSTACLES? The MOU delay was the only real obstacle encountered in Kenya. 5. IS THERE ANYTHING THAT YOU WOULD WANT TO RECOMMEND THAT USAID CONSIDER DOING DIFFERENTLY IN DESIGNING AND IMPLEMENTING PROGRAMS SUCH AS THE TASK ORDER 1 EFFORT? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 284 This question was not asked. TANZANIA 1. PLEASE TELL ME OF YOUR INVOLVEMENT IN THE USAF PROJECT IN TANZANIA. CTO held a conference in Dar es Salaam where Integra was a Silver Sponsor, which gave Integra a half day on the program. Integra’s program time was on universal service access funds. Eric said that there were representatives attending the conference from eleven countries. The Minister of Communications was there and asked Bob Otto for assistance to Tanzania. It was then agreed that Integra would work with Peter Ulanga to develop a program of support. Over several months and multiple discussions, Integra and UCAF agreed a TOR to provide assistance to Tanzania in three areas: (1) Market assessment, (2) Strategic plan, and (3) Implementation of the Strategic plan. The Minister then directed Integra to work with Peter Ulanga (whom Eric had met earlier at the Nairobi forum). At the CTO event, Peter said that he planned to have sixteen (16) staff members on board by November. In November, Peter said that procedural hurdles were stopping the hiring of staff, and he and Integra agreed to meet in February. After the signing of the TOR In February, Eric and David Townsend went to Dar es Salaam to develop a work plan and begin implementation. Eric was the “point” person for six (6) days, and David Townsend did the technical work (three (3) days. A work plan was developed, including deliverables, and Integra prepared the steps for completing a market assessment. A formal Terms of Reference (TOR) had been signed between Integra and Peter’s organization, and the February trip was intended to be a TOR implementation visit. Since Peter did not have any staff members yet in his organization, the February trip was the final involvement of Integra in Tanzania, due to the Task Order timeline constraints. Integra did, however, have occasional follow-up contacts with Peter, but no work activities. Given that it was not possible for Integra to move forward on providing technical assistance to Tanzania, the remaining questions in the interview guideline were not asked. GHANA The connection with Ghana came out of the CTO conference, as the Head of GIFEC – Kofi Ator – was there. CTO was currently working with Kofi to develop a strategic plan, and Integra wanted to leverage off of that activity. At GIFEC’s request a TOR was developed focusing on a Digital Divide Study. David Townsend and Kojo Boakye worked on the strategic plan, but not under GBI (rather, they worked on the CTO Contract). In February 2011, Eric went to Accra where he met with David Townsend and Kojo. They prepared a work plan for the TOR as follows: (1) GIFEC was to gather secondary data on demand for Internet in Ghana, (2) GBI was to design the survey instrument, which GIFEC was to then contract on an outsource basis for implementing the survey. GBI was then to work with GIFEC on interpreting the results of the completed survey. Eric and Kojo developed the survey over two months. GIFEC never came up with the secondary data; just offered lots of excuses. GIFEC also ran into problems getting a contract for the survey, but eventually got ISSER to handle the survey. There was some disagreement between ISSER and GIFEC about the cost of the survey, and ISSER also changed the Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 285 survey questions – and did not give the results to GBI. Kojo somehow managed to obtain the survey results (Eric did not know how he had been able to do so). GBI followed up by notifying the Ministry of Communications of the situation. Once GBI got the survey results, and after some “back and forth” with GIFEC, GBI agreed to prepare the best report it could, based on the draft survey results – and without any secondary data. GBI then prepared such report for GIFEC. Eric indicated that GBI did not spend much money on its activities in Ghana. In responding to the question about obstacles encountered in Ghana, Eric listed the following:  Lack of secondary data  Difficulty in getting a contractor to do the survey  ISSER not wanting to work with GBI NIGERIA Eric indicated that he had minimal involvement in Nigeria. The assistance activities were set up by Bob Otto, Kojo and David Townsend. Eric got involved because of his work on the Nigeria Expanded Trade and Transport (NEXTT) project. He also met with Mr. Makano three or four times to collect the cost numbers Nigeria had incurred in supporting universal service. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 286 UNITED STATES – NETHOPE INTERVIEW– USAID USAF EVALUATION PROJECT NETHOPE ID INFORMATION: NAME: WILLIAM (BILL) B. GARRISON POSITION: FORMER SENIOR DCOP & PROJECT SUPPORT FOR TASK ORDER 1 ORGANIZATION: NETHOPE DATE OF INTERVIEW: 28 MAY 2014 INTERVIEWER: W. KENT EDWARDS THANK YOU FOR TAKING THE TIME TO TALK WITH ME. MY FIRM, DTS, IS DOING AN EVALUATION FOR USAID OF THE SUB-SAHARAN AFRICA USAF COMPONENT OF THE GLOBAL BROADBAND AND INNOVATION (GBI) INITIATIVE. OUR EVALUATION IS ONLY FOCUSED ON THE USAF PROJECT; ALSO KNOWN AS “TASK ORDER 1.” AND, SINCE WE UNDERSTAND THAT NETHOPE WAS INVOLVED IN ASPECTS OF THIS PROJECT, WE NEED TO INTERVIEW YOU. QUESTIONS: 1. PLEASE DESCRIBE NETHOPE’S ROLE OR INVOLVEMENT IN THE USAID USAF GBI PROJECT IN SUB-SAHARAN AFRICA. When the Global Broadband and Innovation (GBI) program began, NetHope was inadequately staffed for purposes of overseeing activities under Task Order 1. Prior to his joining NetHope, there was no one assigned to the USAF (TO1) project. He ended up being the person at NetHope responsible for review and oversight of Integra’s performance of TO1. NetHope’s role in TO1 was that of general oversight, where he oversaw the budget spending. He said that Integra stayed within budget throughout his tenure and never required any corrective action. He believes that TO1 was started around the same time as the larger GBI effort, and was more or less appended to the GBI effort for operational rather than funding purposes. When he joined NetHope, Integra had already obtained staff for the USAF project, had selected the four target countries, and had already established contacts in those countries. He stated that Integra had brought on board the best possible people for the work, particularly David Townsend. 2. CAN YOU TELL ME WHEN THE PROJECT (OR FUNDING) STARTED AND ENDED? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 287 This question was not asked. 3. CAN YOU EXPLAIN THE REASON FOR OR CAUSE OF ANY DELAY IN FUNDS GETTING RELEASED TO THE PROJECT? He does not recall any delayed funding issues; nothing that caused any performance problem during his tenure. However, given the sometimes complex USAID funding approaches, there could have been delays prior to his time. 4. DID NETHOPE PLAY ANY ROLE IN THE USAF PROJECT OTHER THAN AS A CONDUIT FOR FUNDING? IF SO, PLEASE ELABORATE ON THE NETHOPE ROLE. No, there was no active role played by NetHope in TO1, especially since Integra had the right people and approaches in place. 5. IS THERE ANYTHING ELSE THAT YOU CAN THINK OF ABOUT THE USAF PROJECT THAT MIGHT BE PERTINENT TO OUR EVALUATION? He said that it appeared that some of the USAID people involved with the GBI program did not really understand the challenges involved in universal service/access fund operations, especially the necessity to involve multiple stakeholders in establishing the terms of a fund as well as improving existing fund operations. Mr. Garrison stated that Integra’s performance, in his opinion, was one of the best things he saw in the GBI efforts – in terms of political acumen, in terms of establishing the needed rapport with counterparts, and in terms of delivering assistance that will have long-term positive benefits for the countries. He also indicated that he thought that USAID’s desire for the project to have more regional impact did not adequately take into account the realities on-the-ground in Sub￾Saharan African countries. THANKS FOR PROVIDING THIS INPUT. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 288 ANNEX 10. NON-CORE SSA COUNTRY SURVEY ONLINE SURVEY QUESTIONNAIRE The purpose of this survey is twofold: First – to determine the extent of any regional impact resulting from technical USAF assistance funded by USAID, and Second – to develop a current picture of the progress and status of universal access/universal service fund activities in Sub-Saharan countries. 1. Please provide the following information about the person completing this survey: Name Title Organization & Ministry Telephone Email USAF Knowledge/Background 2. When did you first learn about Universal Services Access Fund as a concept? 3. How long have you been working on Universal Services/Universal Access? Less than one year One year Two –five years More than five years 4. What role does your organization play in the USAF management and/or implementation? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 289 What level of Universal Access/Universal Service has been achieved in your country? 5. Does your country have a law mandating universal access to broadband? Yes No 6. When was the legislation enacted? mm/year 7. Did this legislation create an independent government organization to implement and manage universal access? Yes No 8. Has your country developed a national broadband strategy or policy to link universal access to other national priorities? Yes No 9. Did your country examine strategies implemented by other countries – as “best practice” models to consider in developing your national strategy? Yes No 10. Please identify the country or countries you viewed as illustrative models. 11. Is your country currently collecting universal service/access fees from companies providing ICT services? Yes Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 290 15. If no, please describe the constraints that have kept your organization from supporting programs or projects Participation in Universal Service/Universal Access Training 16. Have you participated in a training or a workshop related to universal service/universal access funds? Yes No 17. What type of training or workshop did you attend? Regional workshops and conferences Regional webinars No 12. Please check each type of company from which you collect universal service/access assessments: Telephone companies – wireline Cellular companies Cable TV companies Satellite companies Internet Service Providers (ISPs) Broadcast companies 13. If your country has collected universal service/access fees, have you disbursed funds to support specific universal service/universal access programs or projects? Yes No 14. If yes, please list and briefly describe the supported programs and/or projects Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 291 Newsletters Internet blogs I haven’t participated in any activities Other (please specify) 18. Who sponsored the training? A company (please specify below) A specific government (please specify below) A regional organization African Union SADC International donors (please list below) United Nations Agencies USAID Other (please specify below) Please specify answers 19. Do you recall the name of any of the presenters or instructors at workshops or other training events you attended? If so, please list as many names as you recall. 20. If the training or workshop was a regional or international event, please indicate which other SSA countries were represented (as best you can recall). 21. Please describe how you have used the knowledge/tools received at the training? 22. Have you shared the lessons learned from your trainings or workshops with others in your organization? Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 292 Yes No 23. Please briefly describe what and how you shared with your colleagues. 24. Have you collaborated with any organizations in other countries to solve universal service/universal access issues? Yes No 25. Please list the country or countries and briefly describe the issues involved. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 293 LIST OF ONLINE SURVEY ADDRESSES33 # Country Email Description of Agency 1 Angola geral@mtti.gov.ao MTTI 2 Angola Sec_CA@angolatelecom.com Angola Telecom 3 Benin contacts@atrpt.bj ATRPT 4 Botswana info@bta.org.bw info@bocra.org.bw BTA 5 Burundi info@arct.gov.bi ARCT 6 Cameroon art@art.cm ART 7 Cape Verde info@anacweb.net ANAC WEB 8 Central African Republic art-rca@art-rca.org ART-RCA 9 Chad info@otrt.org OTRT 10 Comoros contact@anrtic.km ANRTIC 11 Congo (Brazzaville) contact@arpce.cg arnault.dicka@arpce.cg ARPCE 12 Congo (Democratic Republic) minptt@ic.cd MINPTT 13 Eritrea eritel@tse.com.er Eritel 14 Ethiopia public_relation@eta.gov.et ETA 15 Ethiopia dg@eta.gov.et ETA 16 Gabon serge_essongue@yahoo.fr ARCEP Gabon 17 The Gambia info@pura.gm PURA 18 Lesotho registrar@lca.org.ls admin@lca.org.ls or lca@lca.org.ls. Lesotho Communications Authority 19 Liberia info@lta.gov.lr Liberia Telecommunications Authority 20 Madagascar drec@omert.org OMERT 21 Madagascar dg@omert.org OMERT 33 Subsequent to distribution of the electronic survey by dTS, the African Union Commission (AU) sent the survey to all of its members, requesting that members complete and return the survey. Four countries have thus far responded to the AU (Burundi, Cote d’Ivoire, Seychelles, and The Gambia). Two of these countries had responded directly to dTS. Results from the other two (Burundi and Cote d’Ivoire) have been included in Tables 1 and 2, following. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 294 22 Malawi info@macra.org.mw The Malawi Communications Regulatory Authority 23 Mali amrtp@amrtp.ml AMRTP 24 Mauritania admin@are.mr Autorité de Régulation 25 Mauritius oolun@icta.mu ICTA 26 Mozambique info@incm.gov.mz INCM 27 Namibia cran@cran.na CRAN 28 Niger arm@arm.ne ARTP 29 Rwanda info@rura.rw RURA 30 Senegal contact@artp.sn ARTP 31 Seychelles benchoppy@ict.gov.sc Department of Information Communications Technology 32 Sierra Leone chairman@natcom.gov.sl NATCOM 33 Sudan customer@ntc.gov.sd NTC 34 Swaziland ps_mict@gov.sz Ministry of Information, Communications & Technology 35 Togo dg@artp.tg ART&P 36 Uganda ucc@ucc.co.ug UCC 37 Zambia info@zicta.zm ZICTA 38 Zimbabwe the.regulator@potraz.gov.zw PORTAZ Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 295 NARRATIVE DISCUSSION OF RESULTS OF SURVEY OF NON-CORE COUNTRIES Survey Description The countries responding to the survey include the Gambia, Rwanda, Lesotho, Namibia, and the Seychelles.34 The responses collected, tabulated, and analyzed here came from individuals holding high level positions in the Universal Service arena, such as Chief Operations Officer of the Regulator, Principal Secretary for ICT, Executive Secretary of the Universal Service Fund, and the Director of the Universal Service Fund. Exposure to Universal Access Ideas The respondents learned about the concept of a “Universal Service Access Fund” in a variety of ways: some learned about USAFs in a domestic context, during the process of drafting their nation’s communications bill, whereas others learned about the idea at international fora, such as ITU conferences. The average respondent had between two to five years experience working on universal services and universal access. The individual with the longest experience had worked on these issues for more than five years, whereas the person with the least experience had worked on universal access matters for less than one year. Role of Agencies in Implementing Universal Access The agencies surveyed had a broad range of duties, spanning from managing the USAF, to appointing licensees, to bidding for the project implementer. In some of the surveyed countries the USF had not yet been established, so respondents were unclear about what role their agency would play. Further, there was a broad range of responses in terms of when respondents learned about the concept of a USAF ranging from more than five years ago, to about one decade ago (2002), to one year ago. Laws and Policies Relating to Universal Access in Non-Core Countries Surveyed Of the countries surveyed, the majority did not have a national law mandating universal access to broadband. To put this in perspective, the US also does not have a national law mandating universal access to broadband or information technology services such as the Internet. Only one country (unknown) has a national law mandating universal access to broadband, which was enacted in 2001. Another country (Rwanda) has a presidential decree governing the Universal Service Fund. About half of the countries surveyed have a national broadband strategy, or some strategy linking universal service to other national priorities. All respondents but one indicated that they did, in fact, consider the policies of other nations with regard to “best practices” when developing their national broadband policies and universal access funds. The surveyed nations mentioned various African nations as models, including Kenya, South Africa, Rwanda, Tanzania, Mauritiusand Uganda, as well as non-African nations, including Singapore, the UK, and Brazil. Collection and Disbursement of USAF Funds Out of the five countries that completed the survey, two of the countries— 34 In addition, one respondent filled out part of a survey, but the country was not identified. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 296 Lesotho and Rwanda—had actually started collecting funds from companies providing ICT services for their USAFs. Both of those countries had also started disbursing funds, allowing them to fund projects related to education, health and local administration in Rwanda, and to establish internet exchange points, and fund infrastructure projects in Lesotho. Interestingly, the two countries that were collecting funds had both begun disbursing them, whereas there was no country surveyed that had begun funds collection, but had not begun disbursement. The survey responses show the wide variety of universal access projects that can be funded by a USAF fund, ranging from projects that are much more focused on expanding social services, to those focused on specific technical goals. Participation in Universal Access Training Based on the survey results, it appears that those participants who received training in issues related to universal access found it was valuable, applied what they learned, and shared their knowledge with their colleagues. This suggests that such training is cost effective and useful, and should be supported in the future. Participants from Rwanda, the Gambia and Lesotho received such training. Two of the respondents noted that their training was sponsored by Intel Corporation (in all likelihood, Integra participated in these training events/workshops). The three respondents who received training all noted that other Sub-Saharan African countries were represented at the training including South Africa, Ghana, Nigeria, and Kenya. Participants noted a variety of benefits they received from the training which they were then able to apply, including 1) having a sense of where their countries stood compared to others; 2) learning which kinds of projects can be financed; 3) mapping access gaps in ICT coverage; 4) developing guidelines for operators about installation; 5) developing a new strategy for the USAF fund; and 6) revising the operations manual. In addition to being able to apply their knowledge from the training, participants appear to have been successful in sharing their newly acquired information with colleagues. One respondent wrote a report putting the lessons into the context of universal service fund operations in his country. Another respondent made a presentation to professional colleagues of the lessons he had learned upon his return. Finally, a third said he discussed what he learned with colleagues. All who had received training said they shared documents from the training with their colleagues. Summary of Findings from the Survey of Non-Core Countries Six countries filled out the survey on universal access/universal service, and five of them completed the questionnaire. Most of the respondents had been working on universal access issues for about two to five years. Most surveyed countries had agencies working on implementing the USAF. Only one of the surveyed countries had a law mandating universal access. All of the non-core countries surveyed wee aware of the efforts of other African nations in implementing universal service projects, and were interested in looking at other countries’ best practice efforts in implementing their own programs. Two of the five countries that completed the survey had begun collecting USAF funds and disbursing them (Rwanda and Lesotho). Training regarding universal access was well regarded, and successfully used on the job, as well as shared with professional colleagues. Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 297 TABULATION OF SURVEY RESULTS OF SURVEY OF NON-CORE COUNTRIES Q2: When did you first learn about Universal Services Access Fund as a concept? *One year (SC) *Years ago when the Communications Bill for Namibia was drafted. (NA) *2002 (LS) *From ITU and CTO forums, (no year stated) *[A] long time ago. Q3: How long have you been working on Universal Services/Universal Access? *More than five years (RW) *Two to five years (GM) *Less than one year (LS) *Two to five years (SC) *Two to five years (NA) Q4: What role does your organization play in the USAF management and/or implementation? *As regulator, CRAN will have to manage the USF and will have to identify and appoint licensees to provide USA in to under serviced rural area's after having done a gap analysis. (NA) *Telecom Regulator (SC) *Management and Implementation (LO) *We are yet to have a USAF (GM) * Manage the fund Identify needs and Establish Programs and action plan for identified projects Call for bid and selection of project implementer Monitor implementation and evaluation (RW) Q5: Does your country have a law mandating universal access to broadband? *No. (NA) *No (SC) *No (LS) Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 298 *Yes (unknown) *No (GM) *No answer (RW) Q6: When was the legislation enacted? mm/year *2001 (unknown) * We have a Presidential decree that govern the USF and [there] is [a] new Broadband policy [that helps as a] reference to broadband services. (RW) Q8: Has your country developed a national broadband strategy or policy to link universal access to other national priorities? *No (NA) *No (SC) *Yes (LS) *No answer (unknown) *Yes (GM) *Yes (RW) Q9: Did your country examine strategies implemented by other countries – as “best practice” models to consider in developing your national strategy? Q10: Please identify the country or countries you viewed as illustrative models. *Yes, South Africa, HIPPSA - SADC Toolkit, GSR 13 ITU study, Brazil, Tanzania, Uganda, Peru and many others (NA) *Yes, Singapore, Mauritius, UK, South Africa (SC) *Yes, Kenya (LS) *Yes, ECOWAS member countries, Rwanda (GM) *Yes, The are many models e.g. South Africa, COMESA, also we have Rwanda NICI Plan that is the basis for Rwanda ICT development (RW) Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 299 Table 1: Universal Access Funds Collection and Disbursement Countries Collecting Universal Access Fees from Companies Providing ICT Services? Types of companies taxed for USAF Has government disbursed funds to support specific universal access programs or projects? What projects are supported by Disbursed Funds? Rwanda Yes  Broadcast Companies  Wireline Telephone  Cellular Telephone  Satellite Companies  ISPs Yes  Education  Health  Local Administration in remote areas. Gambia No NA NA NA Lesotho Yes  Wireline Telephone  Cellular Telephone Yes  Mobile networks expansion  establishment of internet exchange point  capital for infrastructure  for ccTLD operations. Seychelles No NA NA NA Namibia No NA NA NA Burundi No NA NA NA Cote d'Ivoire Yes  Wireline Telephone Yes  National Broadband Network  eGov Project  ICT Poularization  eEducation  eHealth  eAgriculture Source: Data Collected and Analyzed by DTS Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 300 Table 2: Participation in Universal Service/Universal Access Training List of Countries Surveyed Participated in a training related to universal service funds? Type of training attended? Entity which sponsored the training? Other SSA Countries Represented ? Have you used the knowledge you received at the training? Rwanda Yes Regional Workshops & Conferences Financed by RURA. Intel conducted part of the training Yes. Other African countries. “Shar[ing] experience[s] always help[s] know where you are compared to other[s] and […] how to improve services. [It] helps [to know] what kind of projects can be financed. Gambia Yes Regional Trainings & conferences My employer South Africa, Ghana, Gambia, Nigeria, Canada We have mapped out the access gaps of ICT and developed guidelines that obligate operators to install at least one BTS in rural and underserved [areas] per annum Lesotho Yes Regional Workshops & Conferences Intel Kenya We have developed the new strategy for the fund, covering the period: 204/15 - 2016/17; we have also drafted the revised version of the manual of operations. Seychelles No NA NA NA NA Namibia No NA NA NA NA Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 301 Table 2: Participation in Universal Service/Universal Access Training (continued) Source: Data collected by dTS and African Union Commission and analyzed by dTS Burundi Yes Regional Workshops & Conferences ITU/EU through HIPSSA project. All SSA Countries We report what we learn to our administration. This report still being useful in undergoing discussion between the stakeholders on establishment of USAF in Burundi Cote d'Ivoire Yes Regional Workshops & Conferences Intel Tanzania, Kenya, Senegal, Ghana Establishment of the Agency & Framing Projects Final Report, Universal Service/Universal Access Fund (USAF) Performance Evaluation 302 U.S. Agency for International Development 1300 Pennsylvania Ave., NW Washington, DC 20004 (202) 712-0500 www.usaid.gov