MANAGEMENT REVIEW OF IMPLEMENTATION OF SOUTHERN SUDAN INTERACTIVE RADIO INSTRUCTION (SSIRI) PROGRAM MID-TERM EVALUATION RECOMMENDATION & IG EDUCATION AUDIT FINDINGS NOVEMBER - DECEMBER 2008 This publication was produced for review by the United States Agency for International Development. It was prepared by Stuart Leigh, Management Systems International. MANAGEMENT REVIEW OF THE SOUTHERN SUDAN INTERACTIVE RADIO INSTRUCTION (SSIRI) PROGRAM MID-TERM EVALUATION RECOMMENDATIONS AND IG EDUCATION AUDIT FINDINGS A subsidiary of Coffey International, Ltd. Management Systems International Corporate Offices 600 Water Street, SW Washington, DC 20024 Contracted under DFD-I-00-05-00251-00, Task Order #2 USAID/Sudan SUPPORT Project Rahaf Complex Soba, Khartoum, Sudan SSIRI Implementer: Education Development Center (EDC) SSIRI Evaluation: Management Systems International (MSI) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i CONTENTS PROJECT SUMMARY ............................................................................................................................................ iii ACRONYMS...............................................................................................................................................................iv EXECUTIVE SUMMARY ........................................................................................................................................vi I. INTRODUCTION..............................................................................................................................................1 II. BACKGROUND – DEVELOPMENT OF THE PROBLEM & USAID’S RESPONSE .............................1 III. PURPOSE OF THIS REVIEW ........................................................................................................................1 VI. RESEARCH DESIGN AND METHODOLOGY............................................................................................2 ANNEX 1: SCOPE OF WORK...............................................................................................................................45 ANNEX 2: WORK PLAN - SSIRI EVALUATION REVIEW ACTIVITY.........................................................56 ANNEX 3: INTRODUCTION AND BACKGROUND STATEMENT TO THE MID-TERM SSIRI EVALUATION ..........................................................................................................................................................61 ANNEX 4: PEOPLE INTERVIEWED OR MET DURING 11/08 SSIRI EVALUATION REVIEW AND CONSULTANT’S TRANSCRIPTION OF INTERVIEWS ..................................................................................71 ANNEX 5: LIST OF CLASSES & LEARNING GROUPS OBSERVED ............................................................75 ANNEX 6: ONE TEACHER AT A TIME ..............................................................................................................76 ANNEX 7: EDC/SSIRI - RADIO BROADCASTING CONSULTANCY ..........................................................106 ANNEX 8: FREEPLAY RESPONSE ...................................................................................................................123 ANNEX 9: LIST OF SCHOOLS / LEARNING GROUPS OFFERED FOR OBSERVATION ......................125 ANNEX 10: IMPLEMENTATION OF THE SSIRI COMMUNICATION PLAN ...........................................126 ANNEX 11: SSIRI OUTREACH, M&E, AND IT STAFF DEADLINES FOR KEY ACTIVITIES...............129 ANNEX 12: VIDEO CONSULTANT’S REPORT - END OF TOUR REPORT FOR 14 JUNE TO 6 JULY 2008...........................................................................................................................................................................133 ii Table of Figures and/or Tables Table Name/Content Table 1 Mid-term Evaluation Recommendations / Education Audit Findings Table 2 Areas addressed by “One Teacher at a Time” document Table 3 Adjustments Related to Staff and Accountability Table 4 Evaluation Recommendations / Status iii PROJECT SUMMARY Project Name: Southern Sudan Interactive Radio Instruction (SSIRI) Project Implementing Partner: Education Development Center (EDC) Mechanism: Cooperative Agreement CTO: Inez Andrews Start Date: June 22, 2004 Planned End Date: June 21, 2009 Total SSIRI Est. Cost: $15.85 million, including: • DCOF: $2.86 million (a part of the $15.85 million that targets expansion of The Learning Village in The Three Areas). • TERBIA/PS101: $7.62 million (a part of the $15.85 million that aims at broadcasting English language educational programming to populations in Southern Sudan and the Three Areas). Geographic Focus: Southern Sudan with special focus on the three urban centers and the Three Areas. The Southern Sudan Interactive Radio Instruction (SSIRI) project was initially funded in 2004. It was designed to address USAID’s then Strategic Objective (SO) 6: “Improved Equitable Access to Quality Education,” which formed part of the USAID Sudan Interim Strategic Plan (ISP). The ISP goal was “Foundation established for a just and durable peace with broad participation of the Sudanese people.” Under the Fragile State Strategy, the USAID Southern Sudan Mission later shifted its strategic objectives and SSIRI then worked to address Strategic Objectives No. 9 “Avert and Resolve Conflict”, and No. 10 “Promote Stability, Recovery and Democratic Reform”. Currently it is focused on “Investing in People”, “Supporting the Comprehensive Peace Agreement (CPA)” and bringing peace dividends to the citizens of Southern Sudan through improved educational opportunities. The project lifespan is 5 years - June 22, 2004 - June 21, 2009. It is being implemented by the Education Development Center under Contract # 623-A-00- 04-00054-00. The total funding level is $15,852,781 ($5,000,000 through the original contract plus a subsequent funding of $500,000 through Modification 1 on July 11, 2005, $9,969,303 through Modification 04 on August 7, 2006, and $383,478 through Modification 09 on March 13, 2008.) iv ACRONYMS AES Alternative Education System (of the MoEST) ALP Accelerated Learning Program (of the AES) CDC Curriculum Development Center CEC County Education Center CMoE County Ministry of Education Office COP Chief of Party CPA Comprehensive Peace Agreement DG Director General EDC Education Development Center FE Formative Evaluation GoSS Government of Southern Sudan HEAR Health, Education and Reconstruction IG Inspector General IRI Interactive Radio Instruction ISP Interim Strategic Plan LV Learning Village (SSIRI Interactive Radio Programs and Teacher’s Guides) M&E Monitoring & Evaluation MDTF Multi-Donor Trust Fund MoE Ministry of Education MoEST Ministry of Education, Science and Technology MoU Memorandum of Understanding MSI Management Systems International NDI National Democratic Institute NGOs Non-Governmental Organizations OA Outreach Advisor OC Outreach Coordinator OTT One Teacher at a Time P1…P4 Primary 1, 2, 3, or 4 PMP Performance Monitoring Plan PS101 Professional Studies 101 SMoE State Ministry of Education v SSIRI Southern Sudan Interactive Radio Instruction Project TERBIA Teaching English through Radio-Based Instruction for All TTI Teacher Training Institute USAID United States Agency for International Development VSAT Very Small Aperture Terminals (used for data broadcasts) WBEG Western Bahr El Ghazal WES Western Equatoria State vi EXECUTIVE SUMMARY In May, 2008 a mid-term evaluation of the Southern Sudan Interactive Radio Instruction (SSIRI) project was conducted. The evaluation identified issues needing attention and it made certain recommendations, which were largely accepted by the project implementer, Education Development Center (EDC). A subsequent audit by the office of the Inspector General (IG) identified other issues to be addressed. This review was ordered to assess the extent to which the recommendations of the mid-term evaluation and the IG’s audit have been implemented. FINDINGS AND CONCLUSIONS According to the findings of this review, the SSIRI project has implemented or begun to implement 90% of the recommendations of the mid-term evaluation and the IG’s audit. This has significantly improved project management and operations. Evidence of improvement includes: • A revamped monitoring and evaluation system with regularized reporting from the field to head office and back resulting in more effective responses to beneficiary/project needs; • A model class/model teacher support system, resulting in improved observed lessons; • A radio engineering consultant hired to identify issues with Miraya-FM transmitters and begin to assess AM radio options, aiming to improve the classroom audio experience; • Assessment of shortwave reception in multiple project areas resulting in EDC deciding to cease daytime SW radio broadcasts for SSIRI basic education programming; • Contracts made to broadcast SSIRI on 4 new FM stations (Kajo Keji, Yei, Torit, Wau), and for broadcasts to begin on AM radio in Malakal, and likely soon on AM in Wau; • Use of only digital playback devices in Southern Kordofan as a practical, higher quality audio alternative to radio, facilitated by the effective use of solar rechargeable batteries; • Internal assessment of current SSIRI programming and, based on these findings, steps being taken to improve the audio programs now in production (Learning Village – P4); • Increased cooperative activities between SSIRI and state and county MoEST staff, including co-monitoring and co-training; • State and county education officials strongly endorsing SSIRI instructional programs, speaking of SSIRI’s “hand in hand” approach with MoEST counterparts and offices; • Improved public awareness of SSIRI through a communications and marketing strategy; • Initiation of an in-service teacher training short course with 19 teachers in Yambio; and • Further training and support provided to Teacher Training Institutions (esp. Maridi) resulting in tutors producing new course elements assisted by internet technologies. While there have been major improvements, both the mid-term evaluation and the current review findings suggest that the project is faced with continuing technical and other challenges: • Problems with FM reception on Lifeline (wind-up) radios, (i.e., reception of spurious Miraya-FM signals, or “crosstalk”, obscuring SSIRI programs on other FM stations); • Low audio output of Lifeline radios in large classes, and low Miraya-FM audio signals; • Miraya sometimes failing to broadcast SSIRI programs as scheduled; • Significant numbers of teachers ceasing to use SSIRI programs possibly due to broadcast and radio transmission or audio issues (though other factors also contribute to drop-out); • Possible disinclination of a very high Miraya manager to continuing SSIRI broadcasting. vii In addition, the project is hampered by GoSS education system problems, including: • Failure of GoSS to pay teachers on time leading to teachers dropping out of the program; • Lack of clarity about how to timetable for SSIRI to articulate with other daily teaching; and • Absence of means of transportation for local MoE staff to do independent monitoring. And EDC has not yet implemented, or not yet had time to implement certain recommendations: • Revision of Learning Village programs for P1-P3 now awaits completion of LV for P4; • Revision of teacher training and training-of-trainers materials and programs for staff and for counterpart county and state MoE trainers remains undone; and • No study of comparative economics of radio and digital playback devices has been done. We observed that the best SSIRI pupil/learner educational experiences involved large digital playback devices, proving that louder, clearer sound is important. There are pedagogical and practical reasons to prefer digital playback devices for use in schools. Clarity and sound volume able to serve large classes are compelling benefits; as are the benefits of teachers being able to schedule locally, and to pause or back up and review parts of programs. There are, however, significant unknowns in Southern Sudan with the suitability of digital playback devices and their power sources. Issues of long-term cost, maintenance and durability of equipment, etc., all must be assessed further, attending equally to digital devices and wind-up radios. And there are good countervailing arguments for radio. A study is needed to provide adequate basis for smart decisions about the alternatives and/or degree of mix of technologies to be employed should the SSIRI project be extended, or should another similar project seek to apply lessons learned here. RECOMMENDATIONS • We recommend a study of the scalability and sustainability of (1) digital audio playback devices, (2) wind-up radios, and (3) a context-sensitive combination of the two, examining both financial and implementation issues. • The MoEST and USAID should very soon secure a Memorandum of Understanding with the United Nations and Miraya to continue using its air time for Learning Village (LV). • Assess fully and resolve current technical issues with both Miraya and Lifeline, exploring the possibility of buying louder wind-up radios with better high frequency response; • Expand the use of appropriately sized digital devices where they are needed and where it is economical. This mode of delivery is currently providing the most value to users. • Conduct rigorous summative assessments of LV at a number of grade levels; • Evaluate the effectiveness of this year’s pilot version of the PS101 course; • Increase the number of model classes by fading in current schools and moving to others; • Work with the local MoEST offices to integrate Learning Village in timetables with explicit understandings to relate LV teaching to the other daily English and math periods; • Complete revising teacher and trainer training designs, publishing them and providing these materials to the counties and states with training so they can offer more support; • Arrange for AM transmission in Wau and continue to expand over new FM stations; • Maintain audio lesson revision criteria (pauses, songs, longer teacher led activities); and • Continue to do formative assessment of P4 and subsequent programs in Southern Sudan. 1 I. INTRODUCTION The mid-term evaluation, conducted in May 2008, reported extensively in the Introduction and Background sections about the nature of the project and the conditions and circumstances within which it is functioning. The Introductory discussion included: Background and Development of the Problem and USAID’s Response • Southern Sudan • Primary Education in Southern Sudan • Problem Stated • History of USAID Intervention in Southern Sudan • SSIRI Program Description Since the background conditions previously described have not changed materially, that information is included as Annex 3. II. BACKGROUND – DEVELOPMENT OF THE PROBLEM & USAID’S RESPONSE Please see Annex 3: Introduction and Background Statement to the Mid-term SSIRI Evaluation for a full discussion. III. PURPOSE OF THIS REVIEW In May 2008, a mid-term evaluation of the Southern Sudan Interactive Radio Instruction (SSIRI) project identified a number of serious issues that were impeding the project in meeting its objectives. An audit by the office of the Inspector General found that additional data needed to be gathered to substantiate certain PMP indicators. USAID will soon need to decide whether or not to extend the SSIRI, currently scheduled to end June 2009. To provide data to support that decision, USAID-Sudan requested this review to assess to what degree the SSIRI project has addressed the concerns raised in the mid-term evaluation and in the IG audit – which were also endorsed by EDC and USAID. The Scope of Work (Annex 1) for this review called for responses to four key questions related to the mid-term evaluation’s recommendations: Major Questions: 1. To what extent has EDC implemented recommendations from the May 2008 evaluation and what has been the success of these interventions in addressing issues identified? 2. Have these actions achieved the changes envisioned in the May 2008 evaluation recommendations? If so, to date, what specific changes/or outcomes have been achieved? 3. From classroom observations, to what extent do the SSIRI Learning Village programs seem to be working as a teaching and learning tool in the classroom? And to what extent do the English Terbia for Beginners programs seem to be an effective approach for teaching English? 2 4. Are the learning technologies including radios, digital devices, and computer labs/VSAT equipment scalable and sustainable? The SOW for the review also included three requirements related to the findings of the program audit by the Inspector General’s office: 1. Review the current documentation required to support the data required for EDC PMP indicators. 2. Give special attention to the validity and completeness of the following data sources: i. Teachers/facilitators trained ii. Number of pupils/learners utilizing the SSIRI programs iii. Radios and other equipment/materials distributed iv. Number of education officials trained 3. Make recommendations on how learner outcomes can be measured. Since this review was conducted just six months after the mid-term evaluation, the period in which project adjustments could begin to have an effect has been brief. However, USAID and EDC both agreed that in light of the the closing of the school year for the long break in December, looming funding decisions, and EDC’s responsive changes immediately after the mid-term evaluation and program audit, the review should be scheduled for completion in December 2008. Assessment of activities at project sites in Southern Sudan, and at EDC’s Nairobi office lasted for two weeks. There were nine days of observations at schools and learning group sites. While the timeframe was limited, the review yielded significant new primary and secondary data and sufficient opportunities for direct observations. Consequently, the Consultant has been able to assess the performance of EDC as specified, and is confident that his findings are sufficiently valid to warrant this report’s conclusions and recommendations. VI. RESEARCH DESIGN AND METHODOLOGY After an initial debriefing with the USAID CTO and the EDC SSIRI Project Director, a Review Work Plan was submitted to USAID. That document is attached as Annex 2. Findings for the review were drawn from three sources: 1) Desk review of project documents; 2) Interviews with over 100 key stakeholders and other interested parties, including GoSS MoEST (Government of Southner Sudan, Ministry of Education, Science and Technology personnel, SSIRI project staff, USAID staff, teachers, administrators, radio station staff, and ad hoc informants (see Annex 4: People Interviewed or Contacted During SSIRI Review); 3) Primary data gathered through school and training site visits and observations in seven locations (17 Learning Village classes, seven Terbia B, one Terbia A classroom, and one PS 101 training). 3 The Consultant was accompanied to all meetings by a representative of USAID, one from the MoEST, and some from EDC, on the provision that they act as observers and not become active participants in interviews. The review group traveled (itinerary included as Annex 1) to project sites in Juba, Kajo Keji, Yei (Central Equatoria), Maridi, Yambio (Western Equatoria), Kauda (Southern Kordofan, one of the Three Areas), and Wau (Western Bahr El Ghazal) between November 17 and November 25, 2008. The Consultant presented the study’s draft findings, conclusions and recommendations at the MoEST prior to departing Juba, Sudan. In addition to the four major questions to be answered by the review in relation to the evaluation recommendations, and the three review requirements related to the program audit, eleven activities related to the four major questions were required by the Scope of Work. These were: a. Assess the extent to which EDC has adjusted its strategy and program activities to take into account the findings and recommendations of both the mid-term evaluation and the education audit. (Related to Major Question #1 above) b. Assess the success of EDC’s efforts to (a) evaluate the reception of Miraya Radio in project sites and, in collaboration with Miraya Radio, improve the reception of Miraya Radio and (b) contract with additional radio stations to broadcast SSIRI programs. c. At each location, choose schools and listening groups to visit. These classes will represent teachers and facilitators who are using the SSIRI lessons on a regular basis. The Consultant will also visit some other schools or groups where teachers and facilitators have been trained, but where there have not been regular follow-up visits by EDC staff. The Consultant will also visit some different sites than those visited in May 2008 in order to assess EDC’s capacity to implement evaluation and audit recommendations in all project areas. (Related to Major Question #3) d. Assess the extent to which the classrooms are receiving good quality sound from radio broadcasts for the Learning Village and Terbia classes and that the sound appears to be adequate for the size of the class. (Related to Major Questions #1-3) e. In classes that are using digital devices, the Consultant will assess the adequacy of the devices in terms of the quality and loudness of the sound, the use of the devices by the teacher, and the apparent success of recharging the batteries by the teacher. (Related to Major Questions #3 and 4) f. Assess the quality of the lessons: the apparent preparation and use of the teacher's guide by the teacher, the extent to which the children or listeners can hear a good signal, the level of involvement and participation of children or listeners, etc. (Major Question 3) g. Assess the quality of the EDC monitoring process, including the extent to which there is regular, informative feedback from the field sites, i.e., detailed descriptions of ongoing class visits and reporting of basic data. (Major Questions #1-2) h. Comment on factors, both within and outside of EDC’s control, that may impede the implementation of the SSIRI programs. i. Assess the progress of implementing the teacher training program, PS101. j. Assess the use of learning technologies in support of the SSIRI activities, including the Lifeline radios and digital devices, such as MP3 players. At the Maridi TTI the Consultant will assess the extent to which the computer center with VSAT is being 4 used effectively by the staff of the TTI for their own development and for benefit of their classes. (Related to Question #3 - 4) k. Assess the extent to which the design, scripts and recordings of the new P4 Learning Village series take into account the findings and recommendations of the mid-term evaluation regarding the quality of the IRI programs. (Related to Major Question #3) The review mission encountered no significant barriers to the proposed work plan other than the fact that Miraya radio did not broadcast the SSIRI programs on schedule and on a daily basis when we were supposed to assess the quality of radio reception in schools. The Consultant visited 22 schools and adult learning sites in three States and in one of the Three Areas, as well as one PS 101 teacher training site, and two TTIs (see Annex 5: List of Classroom and Learning Group Observations). Interviews were held with key stakeholders in the MoEST; education officials in three states and one of the Three Areas, including County Education Directors and Inspectors, and Payam Supervisors; approximately 36 teachers and school directors; 28 EDC staff members; and other representatives of the radio broadcasting and education communities (see Annex 4: People Interviewed). The Consultant is confident that the methodology meets expectations for objectivity, reliability and validity. A. ANALYSIS AND FINDINGS As presented in the Scope of Work (Annex 1) the Consultant was asked to respond to, 1. Four Major Review Questions; 2. Eleven required Review Activities (see Item 2 (a-k) – “Major activities of the Consultant” – of the SOW); and 3. Three additional Audit Findings related to the findings of the program audit by the Inspector General’s office. Where considerable overlap among exists among these SOW questions, findings for them are clustered in the following discussion. Moreover, to ensure thoroughness the following discussions also reference key Mid-term Evaluation recommendations, which were also highlighted in the Scope of Work. A final cross-reference is to the document (One Teacher at at Time) that EDC generated to respond to the mid-term evaluation recommendations. Re: Major Review Question 1: Assess to what extent EDC has implemented recommendations from the May 2008 evaluation and the success of these interventions in addressing issues identified. Review Activity (a).: Assess the extent to which EDC has adjusted its strategy and program activities to take into account the findings and recommendations of both the mid-term evaluation and the education audit. Re: Major Review Question 2: Determine if these actions have achieved the changes envisioned in the May 2008 evaluation recommendations. If so, to date, what specific changes/or outcomes have been achieved? The mid-term evaluation recommendations were in four main areas of concern (shaded gray in Table 1 below). Some recommendations were more essential to the original SSIRI project design and to fulfilling the terms of the USAID-EDC SSIRI agreement than were others. Thus, 5 conforming to each and every specific recommendation should not be the final measure of the review. An important related framework for assessing SSIRI adjustments can be found in a document called One Teacher at a Time (see Annex 6). This was written in response to the evaluation by SSIRI’s Chief of Party and sent on June 7, 2008 to USAID, the evaluators, and to all SSIRI staff. The One Teacher at a Time (OTT) document incorporated many of the findings and recommendations of the mid-term evaluation. It outlined new and adjusted practices to be followed, setting high expectations for SSIRI staff. Because it became a touchstone for SSIRI management and staff, we include reference to its main points and indicate in Tables 1 and 2 below their relationship to the mid-term evaluation’s recommendations and to the findings of the Inspector General’s education audit. Table 1: Mid-term Evaluation Recommendations Relate to OTT # 1. Assuring clear, strong sound in all SSIRI learning environments 6, 7 1.a. MP3 for TERBIA B Groups 6 1.b. Study of Economics of IRI by Radio and MP3 (or other codec) For Formal and Informal Settings 6 1.c. Marketing and P/R: Rename “Low-Cost Technologies” 6 1.d. Medium Wave (MW) Radio Stations 6 Audio Programs 1.e. Get the Pauses Right 7 1.f. Consider Longer Teacher Led Activity Sections 7 1.g. Examine all Songs for Sing-ability 7 2. Working in Communities 1 2.a. Model Teacher/Classes 1 2.b. Choosing Project Sites Carefully 1 2.c. Continue to Focus on Relations with the States and Counties 1 2.d. Counterpart Identification, Incorporation and Training 1 2.e. Relationship of the Directorates of Basic Education and AES 1 2.f. Promotional and Communication Strategies - 2.g. Partnering with NGOs for pre- and in-service teacher training 5 3. Monitoring and Evaluation Systems 2 4. Investing in teacher training, especially in-service training 8, 9 4.a. Re-examine Strategies to Enrich IRI Training for New Teachers: 8, 9 4.b. Additional School-based Support for Teachers - 4.c. School-based SSIRI Training - 4.d. English Resource Corners for Teachers’ Self-paced English Language Development - Education Audit Findings OTT # EDC reported results were not supported by available data 2, 3 The nine areas of adjustment to SSIRI project implementation outlined in One Teacher at a Time decidedly shifted the emphasis of the project to “getting it right” in some places in each county rather than spreading the project thinly. 6 Table 2: Areas addressed by “One Teacher at a Time” document Relate to Eval. Rec. 1 Focus on quality in a few schools and do not undertake the level of scale up anticipated for 2008 - Demonstrate that the system works 2, 2.a., 2.b., 2.c., 2.d., 2,e. 2 Provide appropriate staffing, training, monitoring and accountability 3 3 EDC staff – capacity building and accountability 3 4 Senior level monitoring 3 5 English ability of teachers 2.g., 4 6 Radios and digital players 1, 1.a., 1.b., 1.c., 1.d. 7 Improvement of the lessons 1, 1.e., 1.f., 1.g., 1.h. 8 Implementation of Professional Studies for Teachers – PS 101 4, 4.a. 9 Technologies for strengthening teacher education at TTIs 4 The degree to which each recommendation has been followed is guaged according to the following scale: • “Not addressed”; EDC has not made significant progress in implementing the recommendation; • “Being addressed”: EDC has at least made some progress in implementing the recommendation – even if it is only to conduct analysis to inform future actions. • “Implemented”: EDC has substantially implemented the recommendation. Mid-term Evaluation Recommendation 1: Assuring clear, strong sound in all SSIRI learning environments (and OTT #6 - Radios and digital players) Review Activity (b): Assess the success of EDC’s efforts to (a) evaluate the reception of Miraya Radio in project sites and, in collaboration with Miraya Radio, improve the reception of Miraya Radio and (b) contract with additional radio stations to broadcast SSIRI programs. Review Activity (d): Assess the extent to which the classrooms are receiving good quality sound from radio broadcasts for the Learning Village and Terbia classes and that the sound appears to be adequate for the size of the class. Review Activity (e): In classes that are using digital devices, assess the adequacy of the devices in terms of the quality and loudness of the sound, the use of the devices by the teacher, and the apparent success of recharging the batteries by the teacher. Mid-term Evaluation Recommendation 1 (d): Medium Wave (MW or AM) Radio Stations Recommendations are being addressed. EDC has taken steps to implement these two related recommendations (Recommendations 1 and 1 d) in order to improve the SSIRI audio experience. While this has been successful in certain ways and locations, the quality of the audio reaching the ears of the learners and teachers still varies greatly. While some children are receiving fine audio, others still must struggle to hear clearly. SSIRI sound is dependent on a large number of factors, including the choice of radio or other playback device, the selectivity of the radio receiver, the quality of the speaker hardware, the power output of the playback equipment, the strength of the source signal, the use of FM or AM or SW frequencies, the time of day, the harmonic (band pass) filters on the transmitters, the normalization level of the digital audio program files, weather, etc. 7 Radio reception reports are now a normal part of the weekly reporting from SSIRI Outreach Coordinators and Outreach Advisors. Largely for this reason, EDC now knows more about the quality and reliability of Miraya FM’s broadcasting and reception. Miraya radio repeater transmitters are not attended, so the first notice that Miraya may get about one of its transmitters being off the air is often from SSIRI staff calling or emailing SSIRI’s Juba office so that SSIRI Juba staff can follow up with Miraya. Huub Gales, Miraya Station Manager, confirmed one of SSIRI’s problems with radio reception: that Miraya’s signal may bleed over onto other frequencies, “Yes there is a spurious signal (FM 101.5 bleeding onto other frequencies.) Miraya does not have band pass filters on their equipment.” This problem was observed in May and again during this review and more will be said about it below. In recent months a radio engineering consultant, Nicholas Mwangi, was hired by SSIRI. He reported on FM radio transmission equipment configurations and on radio reception in Juba, Torit, Wau, Yambio, and Malakal, and to a degree on AM (medium wave) signals in Wau and Malakal. His report is provided as Annex 7. Miraya’s Director Gales confirmed the findings of his report that Miraya is transmitting at approximately half power on a number of its transmitters. Gales attributed this to parts of the transmitters being damaged as a result of the absence of surge protectors at the transmitter sites and to the absence of other equipment. Some replacement parts and other equipment are said to be on order, but the time of delivery is unknown and dependent on the responsiveness of the UN’s procurement system. Beyond providing the results of the consultant’s evaluation to Miraya management, there is no indication that EDC has worked with Miraya to improve reception, as such. On the other hand, SSIRI has frequently notified Miraya that the wrong program is being broadcast or that a local Miraya station is off the air. Other cooperative work is about to begin. SSIRI has contracted a further study of Miraya listenership that will give SSIRI and Miraya a better idea of the listenership to Learning Village, Terbia B, and (esp.) Terbia A and other of Miraya’s programs. Shortwave: With regard to radio bands (SW, FM, AM), the May evaluation questioned the utility of conventional shortwave for basic education in South Sudan. Subsequently, EDC conducted an assessment of shortwave throughout its operational areas. Except in Pochalla, where SSIRI M&E Assistant, Ochan Bongo, reported finding consistently fine SW reception, SSIRI staff have agreed the quality of shortwave signals is inconsistent and too often insufficient for either Learning Village or Terbia classes at the times that they normally meet. Based on this survey, EDC decided to cease daytime shortwave transmissions for basic education programming as of December 10, 2008. The project is moving to FM and AM radio transmission and to solar rechargeable digital players. This has been made possible by the rapid growth in the number of FM stations, and by the project seeking relations with AM (or medium wave) stations. Additional Radio Stations: Since the mid-term evaluation, new stations in Kajo Keji (Grace￾FM), Yei (Spirit-FM), Torit (Voice of Eastern Equatoria) have begun to broadcast SSIRI programs. Broadcasts are now set to begin on FM (and possibly soon on AM) radio in Wau, and on AM radio in Malakal. Other FM stations are likely to appear soon (e.g. Yambio, Bor) Contracts for carriage of SSIRI programs have been issued as follows: 8 • Grace FM (Kajo Keji, Central Equatoria) - Local Consultant Service Agreement signed by Steven Sokiri Samuel, Station Manager, 12 October 2008, and by Tom Tilson, 14 October, 2008. • Spirit-FM (Yei, Central Equatoria) - Local Consultant Service Agreement signed by Alex Loburi, Station General Manager, 7 October, 2008, and by Tom Tilson, 10 October 2008. • Voice of Eastern Equatoria, (Torit, Eastern Equatoria) - Local Consultant Service Agreement signed by Alex Locor, Director General, Ministry of Information and Communication, and Francis Oliha Abalu, Director of Radio and Broadcasting, Voice of Eastern Equatoria, 23/10/2008, and by Tom Tilson, 3 November, 2008. • Radio Wau-FM (Wau, Northern Bahr al Gazal) Local Consultant Service Agreement signed by Luis Pasqual, Managing Director of Radio Wau, 7/11/2008 and by Tom Tilson, 30 October 2008. • Radio Malakal – As noted above, Radio Malakal will be broadcasting the programs for a fee equal to that paid to other stations. A letter was written by MoE Undersecretary William Ater to the Minister of Education in Upper Nile State to urge him to work with the Radio Malakal to provide airtime at EDC’s proposed rate of SP43 per half hour, if not for free. The station has agreed to provide additional fuel to power SSIRI transmissions. EDC is in discussions with others about further radio options: • The Deputy Governor of Western Equatoria State, Yambio (who is also the Minister of Information) informed the review team through USAID’s Emmanuel Walla that the government plans to build an FM station and network of repeaters to bring service to most areas of Western Equatoria. This will further reduce dependence on Miraya-FM and broaden the reach of SSIRI by radio. SSIRI Outreach Coordinator, Edward Kasran, confirms that he has been told of transmitters already being delivered to Yambio. • Internews – EDC is investigating the possibilities for transmission of SSIRI programs over their four stations in the Three Areas next year. And SSIRI remains on the air over • Bakhita Radio in Juba; and • Still most importantly, Miraya-FM which continues to add locations. As FM becomes the new preferred band, there are issues that must be addressed concerning the FM behavior of Lifeline radios since these have been the mainstay technology of the project. Especially at issue is “crosstalk”, the reception of spurious Miraya-FM signals at additional FM frequencies, often obscuring SSIRI broadcasts on SSIRI other host stations. The crosstalk issue may be partially or solely due to the Lifeline receivers, but Miraya itself may be contributing to problems of audibility. The user issues are: 1. The low volume of Miraya-FM signals overall and relative to other SSIRI FM stations; 2. Crosstalk between Miraya (101.5) and other stations/frequencies - a problem that appears most serious in Wau, possibly due to the absence or quality of transmitter filters, yet also possibly due solely to the specifications (esp. filters) of the Lifeline receivers; and 3. Local Miraya stations being off the air for days or weeks. 9 There have been other issues with Miraya related to human error, namely in station personnel playing incorrect programs. At one point SSIRI was partially at fault for this problem by naming/labeling programs in a way that confused Miraya staff. SSIRI discussed this with Miraya and corrected the naming/labeling, but occasional human errors continue, as the Review Team experienced in Maridi during this review. There is no single SSIRI staff person regularly responsible for monitoring and following up with Miraya on their broadcasting performance. According to the Acting Director General of the SMoE in Wau, some teachers have become discouraged or frustrated and dropped out of the SSIRI program – possibly due to problems with SW and FM transmissions. SSIRI’s local Outreach Advisor and Outreach Coordinator estimated that perhaps 40% of trained IRI teachers in Wau and Jor River counties have dropped out. While there are many possible contributing factors, including failure by government to pay salaries, some teachers likely cease using SSIRI programs due to poor radio reception and erratic program transmission. This was the case in the May mid-term evaluation and remains an issue needing attention. Assessing Classroom Radio Reception: The review noted that the presence and quality of SSIRI radio broadcasts for classroom use was affected by various factors. Due to a strike by public school teachers in Juba on Day 1 of this Review, only one Miraya-FM radio class was observed. This was a P1 class in a private school. While more audio volume from the Lifeline radio would have been very helpful, the class did participate actively and the teacher was able to take directions from the radio and apply them. The SSIRI Outreach Coordinator brought a second radio to the back of the classroom in the middle of the lesson, which improved audibility. On Day 2 in Maridi there was no SSIRI broadcast from Miraya. SSIRI called the station and Miraya played the right program number (#92) but from the wrong grade level series. The SSIRI Coordinator helped the teacher to use a digital player (Sanyo) and the class was then successful. On Day 3 in Kajo Keji the broadcast signal picked up by a single Lifeline radio from Grace-FM was adequate for a P1 class with around 50 pupils in a small space. This was a “non-model” class. We noted that the high frequency response of the speaker in the Lifeline radio was not very good. High frequencies are important in audio-assisted language learning. The Grace-FM signal was also fine for a P3 class of about 50 students at another school with two radios in a concrete block classroom. Kajo Keji: Leikor P/S P1 classroom and P1 (non-model) Learning Village teacher, Poni Joice, with pupil 10 On Day 4 in Yei the aforementioned crosstalk problem from Miraya FM on the Spirit FM frequency prevented the P1 teacher from using the Spirit frequency, though the school was in the town near the Spirit FM transmitter (and possibly Miraya’s transmitter as well). At a location farther from town, the signals from Spirit FM were unaffected by crosstalk and were adequate for P2 and P3 classes of 29 and 27 students, respectively, using dual Lifeline radios. On Day 5 in Yambio Miraya was again off the air. Here a Sanyo digital device was used in two Learning Village classes (one model and one non-model). In each case the sound was excellent, as is to be expected with MP3 playback on a “boombox”. A small Nextar device plus speakers was used in a Terbia B group indoors with good sound. On Days 6 and 7 in Kauda large Sanyo digital devices were used in three Learning Village classes and one Terbia B group. The sound was uniformly excellent. There is currently no radio option in Kauda. Terbia B group in Kauda using Sanyo “digital device” On Day 8 in Jor River County the Miraya-FM signal was barely adequate for the first LV class (P1) using two Lifelines outdoors. This class was taught by a non-model teacher. In the second school, while a Lifeline radio was available, the model teacher of the P3 Learning Village class chose to use the Head Teacher’s personal radio because it had a louder output than the Lifeline. The sound from this alternate radio was adequate for this class of 41 students. The head teacher said that he bought a set of batteries each week to keep that radio running. Later that day, a Terbia B facilitator in Jor River County chose to use the shortwave band on a single Lifeline. The volume was louder than Miraya-FM and it was clear with only occasional distortion. On Day 9, a P1 teacher attempted to use Wau FM outdoors but the crosstalk with Miraya was so bad that this was impossible; so he used the Miraya FM signal on two Lifeline radios and had a very successful class. The volume was not loud but it was adequate for him to hear instructions and conduct the class even if the children could not easily hear the radio program. 11 AM Broadcasts: EDC has implemented recommendation Rec.1.d.) Medium Wave (MW or AM) Radio Stations and successfully contracted with Radio Malakal to broadcast Learning Village and Terbia in Upper Nile State. While AM is typically of a somewhat lower audio fidelity than FM, there are two reasons that AM radio is being considered for SSIRI: 1) AM radio reaches farther than FM and can deliver SSIRI radio where otherwise there would be none, and 2) SSIRI programs on AM using Lifeline radios are likely to be unaffected by crosstalk. The only immediate AM radio options appear to be in Malakal and Wau. The Director of Miraya-FM said there is a large unused AM station in Juba not slated for service due to high operating costs. Its potential for future SSIRI transmissions was not assessed. To date there has not been an assessment of the strength and reach of the AM signals from either Malakal or Wau, however, it is very likely that they will offer broader coverage than any single FM station in these two areas. As a sign of government buy-in to SSIRI program, the GoSS MoEST sent a letter to the Upper Nile Ministry of Education seeking their assistance in arranging for SSIRI broadcasts on Radio Malakal (AM). Though previously resistant and seeking a higher fee per program, Radio Malakal agreed to air the programs for a fee equal to that paid to other FM stations. (It should be noted that in terms of energy consumption AM is more expensive to run than FM). The Upper Nile government has also agreed to provide resources (e.g. fuel) to cover the additional costs of SSIRI AM transmissions. “The Upper Nile state ministry of information has responded positively to the letter it received from His Excellency the Upper Nile state Minister of Education regarding the broadcast of SSIRI programs on radio Malakal. The Upper Nile state Ministry of Information has agreed to provide additional barrels of fuel to radio Malakal. The Upper Nile state Ministry of Information instructed radio Malakal to broadcast SSIRI programs at the cost of $20.00 (40 SDG) per half hour lesson as indicated in the draft service agreement.” (Tom Tilson in an email to Stuart Leigh, Nov. 24, 2008) In Wau there are both AM and FM stations under the same (state) management. There is also a relatively well-equipped recording studio. EDC has created a relationship with the state radio in Wau, though their agreement is currently only for FM service. While we were in Wau the FM station (95 mHz) was experiencing disabling problems, not the least of which was intense crosstalk from Miraya FM. The mid-term evaluation noted that in a number of locations 95 mHz is a frequency on which Lifeline radios seem to pick up a spurious Miraya-FM signal. However, even if the Wau station were to switch FM frequencies from 95 mHz to another, this might not help much since Miraya crosstalk in Wau was found virtually all across the FM dial - from 92 to 106 mHz. The obvious near-term solution is to get SSIRI onto Wau Radio’s AM band. While there are further steps to take in doing this, the picture is encouraging since the Director of Wau Radio, Louis Pasquale Aleu, is favorably disposed, saying, “We can use the MW as well as the FM. We want to put the programs on AM and FM at the same time.” With regard to the crosstalk problem, the cause is not yet well understood. A Miraya radio engineer suggested in reply to a query from the Consultant that crosstalk needs to be tested on a car radio and if it is not present there then Miraya’s equipment is likely not the source of the problem; however, if it appears all across the spectrum, then it could be related to Miraya’s equipment. Since it did not happen on other radios we tested, Miraya is quite possibly not 12 responsible for the problem. On the other hand it did happen all over the dial. There may be something Miraya could do to reduce the prevalence of crosstalk. This issue requires systematic analysis. The Consultant also queried NDI, the provider of the Lifelines to SSIRI. They forwarded his email the Lifelines maker, Freeplay Energy. Freeplay has begun to investigate: “We once had a report from a USA distributor to which we sold a specific radio, where there was an overpowered bleed across the dial scale from a specific transmitter. To correct this, we added a second stage of FM/IF filtering that attenuated the IF signal on each side of the carrier. We are in the process of testing a similar modification to the overpowered Lifeline that we have in the laboratory and we will let you know the result by the end of the week.” (John Hutchinson, Chief Technology Officer, Freeplay Energy, Ltd., December 03, 2008 email to Tom Tilson.) SSIRI staff feel that there are meaningful differences between the two series of Lifeline radios with which they have been equipped. Specifications for both “AA” and “AB” series have been requested by the Consultant with particular interest in the receiver, the speaker, and any elements that control volume. Freeplay has confirmed the perceptions of the SSIRI staff that there are differences between AA and AB generations. In fact they were built by different manufacturers with what Freeplay termed small audible differences in output level (the “AB” series being lower) and possibly in other attributes (see Annex 8: Freeplay Response). Digital Devices: EDC is using two types of digital audio playback devices. In Juba we observed use of the smaller Nextar MP3 player and small portable speakers. The sound quality is excellent but the volume, which is dependent on the amplified speaker, is limited. This combination is good for groups of up to twenty-five people outdoors, and perhaps for a few more indoors. In either case, this depends on there not being a great deal of nearby noise. These units are powered by triple-A (Nextar) and double-A batteries (speakers). There is an issue with the Nextar digital players and their triple-A rechargeable batteries. These batteries seem unable to provide power for a full 30 minute lesson. While the nominal output voltage of triple-A batteries is 1.5 volts, according to SSIRI management, the triple-A rechargeables actually provide about 1.2 volts and the Nextar players may be so sensitive to voltage requirements that they shut down as soon as the incoming voltage drops below a certain level, and that this threshold is reached by the rechargeables in less than 30 minutes. Encouragingly, and as noted above, the larger 15 watt, 12 volt digital device, the Sanyo MCD UB575M offers sound that is clear, with very good high frequency response, and strong volume. They are powered by eight rechargeable D-cells. Thus, they can also be powered by a common 12 volt car battery. These units are good for groups of 100 or more indoors and 70 or more outdoors. Their use is most pronounced in Kauda where, for a number of reasons, digital players are likely to be the sole mode of delivery for some time to come. 1 1 There are a number of reasons why radio is currently a difficult medium to rely on for daily basic education in much of Southern Kordofan. In Kauda local communities support the schools and determine class schedules. Schools meet at different times and on different days of the week. Internews’ Kauda station, the Voice of Community Radio, does not now broadcast on Saturday and Sunday, which are days when the government schools meet. Local SSIRI staff say that the radio station has a restricted range. In view of such factors, SSIRI’s Outreach Advisor in Kauda strongly argues against trying to use radio for Learning Village and Terbia B in that location. 13 By all reports, the solar panels for the Sanyos and other digital devices are working well, (including the integral ones on Lifeline radios). If the Sanyo unit must be used for three daily Learning Village lessons, exposing the panel to the sun for two and one-half to three hours is adequate to charge the set of batteries, according to teachers and SSIRI Outreach Coordinators. In sum, the Sanyo digital devices, solar panels and rechargeable batteries used by SSIRI in Kauda are working well. We observed a number of classes using these machines (in and out of South Kordofan) and the sound levels and clarity attainable were always very good for all SSIRI applications and settings. Students are consequently attentive to a greater degree than similar students are observed to be when the radio is barely audible. The enabling technological conditions for quality delivery of SSIRI audio in Kauda have certainly been very well met. However, the roll-out of SSIRI lessons in Kauda is just a month old, and the durability of the equipment and related issues of maintenance and support are not known. Mid-term Evaluation Recommendation 1 (a): MP3 for TERBIA B Groups Recommendation is being addressed. In May 2008 EDC was already moving many Terbia B groups to small digital players. We observed successful Terbia B classes that utilized both the small Nextar digital devices (Juba and Yambio) and the larger Sanyo (Kauda, Yei and Maridi).We also observed those that continue to use radio (Kajo Keji – using Grace FM) and (Wau – using SW). Learners respond actively to Terbia B programs when facilitators are comfortable with the demands of the audio programs. Kajo Keji – Terbia B group using Grace FM Terbia A is also being used in secondary schools, as we saw one using the larger Sanyo device: “Terbia A is a valuable part of the curriculum. We use it not only for English but for political education. There are no problems with the digital player technology. With it also being on radio in the evening we can use it after hours.” (Benson Sokiri Lokuli, Headmaster, Kajo Keji Secondary School) Mid-term Evaluation Recommendation 1 (b): Economic Study of IRI by Radio and MP3 (or other codec) for Formal & Informal Settings 14 Recommendation has not been addressed. As stated in the mid-term evaluation report: “While such a study may be beyond the scope of EDC’s current Agreement, this would be an interesting contribution to the field and should be considered for a future year.” EDC appears to have not undertaken a formal study of such issues at various scales. They did provide the following information, which may be used in further economic projections: • US $90.00 per Sanyo machine in quantities of 100 pieces • US $48.00 per complete solar charger for the Sanyo • US $46.86 per set of 8 rechargeable D-cells • US $ 2.86 for wiring the solar panel for 1 Sanyo Consumer electronics being perennially a rapidly changing market, any economic analysis or listing of technology options would need to be updated frequently.2 Mid-term Evaluation Recommendation 1 (c): Marketing and P/R: Rename “Low-Cost Technologies” Recommendation has been implemented. EDC and SSIRI no longer refer to the digital devices they are using as “low-cost devices” but rather simply as “digital devices”. The Audio Programs Themselves (and OTT #7 - Improvement of the lessons) Review Activity k: Assess the extent to which the design, scripts and recordings of the new P4 Learning Village series take into account the findings and recommendations of the mid-term evaluation regarding the quality of the IRI programs. EDC is taking steps to improve the programs currently in production. There is awareness within EDC that to increase the effectiveness of P1-P3 Learning Village programs, some programs should be revised. The EDC July-September 2008 quarterly report describes an internal assessment of the audio lessons. The assessment identified needed improvements in pacing, pronunciation (intelligibility), song design, and in certain games. (A number of these program features were also identified in the mid-term evaluation). In observations of lessons during this review, while not always agreeing on particulars, EDC management, staff, and the Consultant again noted the same problematic lesson design features in P1 - P3. Beyond initial analysis, however, implementation of the recommendations for audio revision has not begun. With a June delivery deadline approaching for LV P4, the project’s focus is necessarily on producing that series. EDC’s IRI Advisor in Nairobi, Sarwat Alam, confirmed that the P4 lessons are informed by the internal assessment. EDC is now conducting formative evaluation (FE) of new P4 lessons in Southern Sudan (Kajo Keji and Juba) rather than in Nairobi. This change conforms with a suggestion of the mid-term evaluation that the Nairobi FE test site and the average South Sudan classroom are likely very different, and that this could skew FE findings. Sarwat Alam indicated that initial findings of the Southern Sudan-based formative evaluation are that the new P4 2 There are thousands of Lifeline radios in SSIRI’s storage in Juba. While these radios did not cost the SSIRI project much, if anything, they were a cost to USAID through an agreement with NDI. In doing a comparative cost analysis one would need pricing figures and lifespan estimates for the Freeplay Lifeline radios. 15 program format, which she said is based on an “immersion approach”, requires more English competence than the Southern Sudanese pupils possess. Further script revision needs to be done. She said EDC plans to revise the design of the first 50 programs to continue using the “translation method”, as used in P1-P3. After that they may move back to a more immersive method. Mid-term Evaluation Recommendation 1 (e): Get the Pauses Right Recommendation has not been addressed. This recommendation for P1-P3 has not yet been implemented. As noted above, with current focus on production of P4 programs, there has not yet been any revision of P1, P2, or P3, nor is there expected to be before June, given the current P4 workload. During this review, the Consultant found that some on the SSIRI staff feel that the pauses are generally not a problem because the main determinant of the success or failure of pupils is the capacity of the teacher to properly manage the lesson. Others (in and out of SSIRI staff ranks) have a more nuanced view and say that, while the teacher is the most important determinant, the “pace” of the programs, or of parts of the programs, is often too fast. In fact, this was a finding of the internal assessment of Grade 3 programs.3 The survey found that “60% of the teachers and pupils were not comfortable with the pace of the program. It seemed to be too fast.” The Southern Sudan-based formative assessment of P4 programs should further illuminate the issue as some of the writers have a chance to observe lessons in use in Southern Sudan. Mid-term Evaluation Recommendation 1 (f): Consider Longer Teacher-Led Activity Sections Recommendation is being addressed. This recommendation is being implemented in the P4 programs only (and we would hope in any future revisions to P1-P3). As with the pauses above, there are various opinions on the need for such an altered format. Most lessons observed have pauses of a maximum duration of 15 seconds. EDC is changing this in P4: There are going to be three one minute pauses per lesson (1 per teaching segment). The reason is that the teachers were feeling squeezed by the short pauses and felt that the moment they start they must stop. That became clear in the P4 formative evaluation … (not so much in the assessment of P3. In the assessment they told us they were not getting all the content – their comprehension was low). (Sarwat Alam, IRI Consultant, EDC) Our recommendation for longer teacher-led activities was made in view of our belief that the SSIRI program should explicitly help teachers develop skills and become comfortable with specific strategies and learning activities that they can readily apply after the radio lesson. Longer teacher-led activities allow for forms of guided practice by teachers of the types of teacher/student and student/student transactions that are more likely to be applicable to teaching after the radio lesson is over. (See further discussion under findings for Recommendation 2 (e). Mid-term Evaluation Recommendation 1 (g): Examine all Songs for Singability 3 The experience of “pace” is related to the pause lengths, though it also encompasses other design features - like the duration of songs, number of choruses allowed for hearing a song and then singing it, time allowed for classroom motion, time allotted for the teacher’s translation followed by the students’ responses, time for other tasks, etc. 16 Recommendation is being addressed. EDC has begun to implement this recommendation. There is general agreement within EDC that some songs are stylistically inappropriate (e.g. the “Landmine Awareness” song), some are too fast to be sung easily, and some are presented as a single short chorus only and children have little chance to actually learn and sing the song. During the Review we saw many classes in which children hummed or garbled utterances while carrying the melody, showing that they knew the tunes but not the words. Teachers apparently do not take time to teach the songs before or after the lesson, which they could easily be trained to do. Slowly teaching the words of English songs is a fine and fun way to help children learn English, but it never happened, even where the children were struggling. EDC appears to be implementing this recommendation in production of P4 by (1) attending to the design of songs during the internal assessment of P3 programs; and (2) by composing songs differently for P4. Sample P4 programs we heard had songs that were slower and more singable.4 While EDC is currently using the same composer for P4 as the one who worked on P3, EDC plans to exert more control over the music production process: “I was not happy with the music in P3. In Agok there was a song about animal life and the children were not happy with it. It was in a style that was wrong. And the Landmine Song is in a style that is happy and that is not a happy subject… Some of the P3 songs, like that one, were – what’s the word? – bombastic. We are still using the same keyboard person; but we used to just contract him to deliver the songs to us and now we are going to act more like a music producer.” (Jane Namadi, SSIRI Radio Producer) As revision of P1-P3 is delayed until a possible extension period, we do not know what criteria EDC will actually use in revising songs for P1-P3. Re: Major Review Question 3: From classroom observations, assess to what extent the SSIRI Learning Village programs seem to be working as a teaching and learning tool in the classroom. And assess to what extent the English Terbia for Beginners programs seem to be an effective approach for teaching English. Review Activity c: At each location, choose schools and listening groups to visit. These classes will represent teachers and facilitators who are using the IRI lessons on a regular basis. The Consultant will also visit some other schools or groups where teachers and facilitators have been trained, but where there have not been regular follow-up visits by EDC staff. The Consultant will also visit some different sights than those visited in May 2008, in order to assess EDC’s capacity to implement evaluation and audit recommendations in all project areas. Review Activity f: Assess the quality of the lessons – apparent preparation and use of teacher's guide by the teacher, the extent to which the children or listeners can hear a good signal, the level of involvement and participation of children or listeners, etc. Observations were made of 17 Learning Village classes in a wide range of settings and at all three grade levels, 7 Terbia B classes, 1 Terbia A class at a secondary school, (See Annex 4, People Interviewed or Contacted During SSIRI Evaluation Review). In that table, the cells 4 On a related note, in listening to sample P4 programs we noted that the character voice(s) (e.g., “Jessica the Fish”) are much more intelligible than was the character voice of “Lotoli the Hare” from P1-P3. 17 highlighted in light blue indicate these observations. In most cases the Consultant was offered a choice from a number of Learning Village schools or Terbia groups to visit. A list of the schools offered in each location from which the choice was made is provided (Annex 9). One was then chosen at random. Occasionally the choice was constrained by considerations of time and distance. Some of these schools had “model classes”, others did not. The Consultant also observed a traditionally-taught P4 class in a school that also did LV in P1 and P3. Classes observed were all using the same generation of unrevised programs the Mid-term Evaluation Team heard in May. Most teachers observed were using the Teacher’s Guide properly and appeared to have consulted it before the lesson. The extent to which children using the Learning Village programs could hear the program varied widely depending on factors cited above – radio or digital device, radio station used, indoors or outdoors, class size, nearby noise, etc. The level of involvement of the pupils was similarly varied. As with the Terbia B programs, it depended largely on the teacher’s ability to engage them while following the program. Where the programs met a minimal standard of audibility, pupils were generally engaged. We observed that while it is best if the teacher works seamlessly and gracefully with each turn in the program, more important was his/her ability to activate students around the content of the lesson, even if this meant that at times he/she disregarded the radio to continue speaking or to continue a productive activity. The effectiveness of Learning Village as a teaching and learning tool, or the effectiveness of Terbia B in teaching English may be assessed from a number of perspectives. In the words of the users: 1) Does it provide the teacher with a practical means of presenting a lesson? “The radio is more encouraging to the pupils. When they see me working the radio they become more excited. The radio keeps their attention. With it they are 90 % attentive. I have 98 in my class.” (Clementina Kidew Arkangelo, P2 teacher, Sadaka P/S, Juba) “There is a difference in the level of spoken English. They are lacking any language books. The program should continue. Teachers are very committed.” (Kirya Julius Modi, Head Teacher, Leikor PS, Kajo Keji) 2) Do the students like it and do they come to class? “You know this Learning Village helped us very much because sometime like this class here was about just 20 but when we start with the radio you find many pupils have come and there are many in the class. So this is one of the goodness of Learning Village. Today there are 65.”(Jojo Hamad, P1 Teacher, Upper Kumo P/S, Kauda) Based on comments from a number of teachers it is clear that the radio and the LV programs are attractive to the students. Parents are demanding that their children have an opportunity to benefit from Learning Village programs. 3) Is it sensibly structured as an educational course? “There is a marked difference in those (pupils) with the SSIRI and those without. The lessons are well structured. They get today what they got yesterday. The children are 18 doing better. I can prove it. My statement that SSIRI students are doing better than the others is based on my observations and not just a few but many. I have visited many schools and see it in all.” Abore George Dinyato, Payam Supervisor AES/SSIRI, Kajo Keji “I use Terbia 3 days a week. It helps me facilitate the lesson. I use Spirit FM. The problems I have are the time of arrival of students. The sound was too low at one time but we solved that with two radios in one room, front and back.” (John Gama, Terbia Facilitator, Yei) Learning Village is based closely on the New Sudan primary curriculum and syllabus. Terbia B is based on the Accelerated Learning Program of the MoEST. Without comparative test data it is hard to be sure, but it seems that both LV and Terbia are meeting educators’ expectations in teaching English and math in keeping with the curriculum and syllabi. Mid-term Evaluation Recommendation 2 (a): Model Schools5 (and OTT #1 - Focus on quality in a few schools and do not undertake the level of scale up anticipated for 2008 - Demonstrate that the system works) Recommendation has been implemented. A model class is one where the teacher has enough English skills to generally understand the LV radio instructions in English, and where visitation by SSIRI staff can be done frequently, perhaps once a week. Model teachers have been identified in Juba, Kajo Keji, Yei, Yambio and Wau. In Kauda, which has a school year that starts in October and where the Learning Village program is just now beginning, model teachers will be identified as initial teacher performance can be observed. The model teacher approach appears to be having dual benefits: (1) Observed lessons are much more successful than those observed in May 2008; and (2) Outreach Coordinators report greater job satisfaction in seeing the concrete fruits of their support in classrooms. Formerly, when they were uniformly responsible for a large number of schools and did not have criteria for prioritizing their efforts, the project was not so likely to produce any quality outcomes. “That document (“One Teacher at a Time” that required the model class approach) not only worked for improving the SSIRI implementation, but I also learned a lot from it. I’ve focused on one thing at a time. I’ve experienced some changes in the way I’ve implemented SSIRI. If I look before and after that document, in Wau it was a difficult situation with 40 schools at a time. It was about quantity. It’s now more about quality.” (Bullen Murangi, SSIRI Senior Outreach Advisor) “Non-model” teachers still receive support visitation, but less frequently – perhaps once a month. Both model and non-model teachers, chosen randomly, were observed during the review. Observations establish that teachers receiving various degrees of support from Outreach Coordinators - those in both model and non-model classes - are able to manage LV classes. Those receiving more intensive attention seem to have somewhat more successful lessons, though some non-model teachers do as well as their model teacher colleagues. 5 Because EDC is not really applying a classic “whole school” approach (with all that may imply), they prefer that rather than use the term “model school” we use the more accurate term “model teacher” or “model class”. 19 Mid-term Evaluation Recommendation 2 (b): Choosing Project Sites Carefully Recommendation is being implemented. In its effort to focus on quality, EDC has pulled back from certain areas due to specific unfavorable conditions likely to impede implementation. For example, EDC has suspended its operations in Jonglei state for the present time due to the absence of radio signals there, moving staff from there to Kauda to augment staff in preparation for growth in what it considers an important expansion area. There is concern within SSIRI about careful use of human and physical resources. At the same time, the staging of inputs to growth areas (e.g. Kadugli) may be creating gaps in support to user groups that could be problematic. For example, training was scheduled for Kadugli for the end of November and yet an Outreach Coordinator for that area had not yet been hired, trained, and equipped. A comment by EDC’s Regional Director speaks to this point: “We should take to heart the pruning that has taken place, for example Raja (e.g. Raja county in Western Bahr El Ghazal). We were in a reaction mode – everyone wanted expansion and so we did. … We should have a document that outlines the 10 things we should require that we have before going into an area – like a radio signal, a place for people to live, etc.” (Kent Noel, EDC Regional Director) Mid-term Evaluation Recommendation 2 (c): Continue to Focus on Relations with the States and Counties Recommendation has been implemented. Even before the mid-term evaluation, SSIRI had held a series of workshops for state, county and payam education officials. Training, outreach, and strategic communication events have been done in a number of states and counties. For example, on October 1, 2008 there was an official SSIRI launch event in Kajo Keji with over 800 participants including the County Commissioner. “The occasion was made colorful by traditional dance, songs from school children and secondary school students, all praising SSIRI as a good program. The County Education Director declared that SSIRI programs be included in the school timetable.”6 During this Review, state and county education officials expressed strong endorsement of SSIRI instructional programs and of the way in which SSIRI staff are assisting and working “hand in hand” with them. “Before implementation they (SSIRI) came to us and together we introduced it. We are working hand in hand. Actually they have achieved many things here. If you go to teachers who’ve been trained they are fully aware. Our cooperation is in full.” (Abdulhamid Haron, Nuba Mountains Region Education Director, REO, Kauda) “The Director General of MoEST – CES appreciated the meeting, which is ‘a wake up call’ for the state and education officials. He urged all the participants to work together and promote the smooth implementation of IRI programs. He thanked EDC for ensuring that literacy is realized among the people of South Sudan. He called on all the respective 6 Annex 7: Implementation of The SSIRI Communication Plan, November 2008, Charlton Doki, Consultant to EDC 20 departments to cooperate with EDC in the implementation of SSIRI programs.” (EDC minutes of meeting of Central Equatoria state officials, county & payam directors, June 28, 2008) Mid-term Evaluation Recommendation 2 (d): Counterpart Identification, Incorporation and Training Recommendation is being addressed. While the mid-term evaluation noted impressive alignment of the SSIRI project with MoEST formal structures (e.g., organogram, nominal designations of SSIRI/AES posts at state and county levels), we did not witness many expressions of actual day to day integration (e.g., coordinated efforts, co-monitoring, etc.). During this Review, however, substantial evidence was gathered that EDC, assisted by the MoEST’s AES Directorate, is implementing the first two aspects of recommendation 2 (d), counterpart identification and incorporation. With regard to counterpart training, however, little has been done since the mid￾term evaluation. Counterparts from the MoEST, SMoEs, and CMoEs (see Annex 4 - People Interviewed or Met During 11/08 SSIRI Evaluation Review) said they were very familiar with the SSIRI project and were working closely and cooperatively with SSIRI staff. For example, note the use of the words “our” and “we” immediately below: “SSIRI is one of our programs. We have the Outreach Advisor and we have a staff of about 5 people. We have 2 Terbia groups…” (William A. Baabe, Director AES, SMoE, Yambio) “This is the program everyone is trying to get now; especially when FM is something we see entering the whole system. This project has brought excitement. Poor children go for this program. The issue of expansion is important. If children are responding we must see how to go on many fronts. The issue of sustaining is something we must all think about. If you give it to Western Equatoria today we’ll ask “how do we manage it”. It is a very good program…. please introduce it as soon as possible.” (Pia Philip Michael, Director for Administration and Finance, SMoE / WES) Counterpart incorporation at an operational level is further revealed in minutes of a Sept. 8, 2008 planning meeting for a community-wide launch of SSIRI on Sept. 25 in Kajo Keji. A launch committee of ten people was formed with seven County Education Office and Payam Education Office staff plus three EDC-SSIRI staff. Such intrinsic signs and other statements of SSIRI￾SMoE/CMoE integration and cooperation were encountered at various times in the review. With regard to training as a key step toward further integration, SSIRI Deputy COP, Maree Melican, suggested that SSIRI must do more to embed SSIRI training and support skills in MoE counterparts: “Programs have to be embedded in MoE programs. Any training of trainers or teacher training that we have as modules should be delivered to them so they can do them … A model by which the scale up can work must really be with the MoE down to the payam level so that the face of the project is not the OA and AC but them. What that model looks like has to be teased out.” (Maree Melican, Deputy Chief of Party, SSIRI) The improved training packages that EDC intended to refine for MoEST counterparts have not yet been fully developed. This was a recommendation of the evaluation. EDC has developed an 21 extensive two-month Scope of Work for this and other training work, but for various reasons, the training consultant hired has not been able to complete the tasks outlined.7 Nor have the modules now being used to train Learning Village and Terbia teachers/facilitators yet been offered to the state and county MoE offices, as the SSIRI Deputy Chief of Party has said should be done over time (in an easy-to-use material form after the above-mentioned training materials development consultancy) to promote transfer of “ownership” to the various offices of the MoE. Mid-term Evaluation Recommendation 2 (e): Relationship of the Directorates of Basic Education and AES Recommendation is being addressed. This recommendation was made because it was not clear that the MoEST Directorate responsible for Primary Formal Education (“General” or “Basic”) and the Directorate responsible for Alternative Education Systems (AES) were clearly on the same page about (1) the role of SSIRI in primary schools, (2) the need for SSIRI periods in school timetables, and (3) the way LV should relate to other teaching of the same subjects. Signs on this trip suggested these phenomena were beginning to be remedied, though it is still unclear how the highest levels of the MoEST view the logical overlap between AES-managed operations in schools and operations controlled by the “Primary/General/Basic” Directorate, however it is best described. “This program is very successful to the extent that we are going to put it within the curriculum. Steps are needed still for there to be a review of the whole curriculum…. There will be a national languages workshop in March that precedes the curriculum decision. The policy must come from the state level.” (Kuol Atem, Director, AES Directorate, MoEST) While there may remain some high level issues in the GOSS MoEST between the AES and Basic Divisions relative to curriculum that will require more time to work through, there is growing evidence from the State and County directors that SSIRI is making its way into the formal timetables. We see this as a defacto embrace by the state Directors of Basic Education of this AES-initiated project. At the Western Equatoria Ministry of Education the heads of the offices of Basic Education and Alternative Education and the State Director General met together with the Review Team and they all voiced their support for the project. While one would expect the AES head to embrace project expansion, it was the Basic Education Director who said: “Why are some not given radios? You should try to cover as many (schools) as possible so that none complain.” (Hezekiah Wayo, Director for Basic Education, SMoE, Yambio) 7 The EDC SOW specifies that these tasks include: 1) “To design an integrated and comprehensive training program for Ministry personnel that would include the senior state Ministry personnel including the Directors for Alternative Education Systems and the senior inspectors (SSIRI, primary education, teacher training, ALP, and English), and relevant officials from the County Education Offices. This training would also likely to include the relevant officials at the payam level.” 2) “To coordinate and support the implementation of the training program for SSIRI staff and Ministry personnel. The training will need to be implemented in the six major states and, perhaps, the Three Areas. Each training will first focus on EDC staff and then invite the Ministry officials for a joint training.” 22 Learning Village timetable, Ronyi School, Yei (still specifying Miraya, not yet Spirit FM) In Western Bahr El Ghazal state the Director of Basic Education chaired the Review meeting and expressed his support for the SSIRI approach: “The need will continue – if you stop, what do you do? There are difficulties – most community people are uneducated. To get them going to explain to them what is the use of these programs. If it is only delivered by radio some of them may not have time so you have to go to them in community meetings, clubs, schools. To make them know. We have to create awareness of this program.” (Donato Ugali Zimbeli, A for /Ag Director General, Director of Basic Education, MoE, WBEG State, Wau) But it is not clear that the bridge between AES and Basic Education with regard to SSIRI is being successfully built in all states, or that EDC is applying as much effort as it might to addressing concerns about SSIRI that may remain among state Basic Education Directors: “Abraham is the Director for Basic Education in Central Equatoria and he has not yet communicated with the schools about SSIRI. SSIRI is waiting for him to revise the draft MOU for the State, MoEST and SSIRI.” (Rejoice Manasseh Binyi, Outreach Advisor for Central Equatoria State, SSIRI) Nor is it clear that EDC has thought through the guidance Basic Education Directors will need to relate the Learning Village radio lessons to the teaching of the same subjects each day when the radio or audio program is over. Relevant to this is the fact that most classes are not “self￾contained” (one teacher teaching all classes). For the foreseeable future in many states, one teacher will typically teach primary English and another will teach mathematics. It will be unlikely that the math teacher, who is often not the one who was the Learning Village teacher, will know what the math content of the LV lesson was that day, or the day before. Thus the content of the other math period each day may not relate to the math content of the LV lesson. A similar issue exists with regard to additional time for teaching English. While the LV teacher is more likely to be the one to do additional English teaching each day, the Teacher’s Guide frequently does not offer any ideas for follow-up activities to be done after the radio lesson. While the head teacher may have put Learning Village into the school’s timetable, it is not clear that either he/she or the county office have thought through the relationship between the daily LV content and what should or might be taught in any additional English or math periods.8 “We have initiated Learning Village into the timetable. The teachers are tuned to that and have to work with that subject because it is a government program and in the syllabus and they have to do it.” (Benjamin Wani, A/Director Admin., Kajo Keji) 8 It is unclear how project schools and local administrators are scheduling LV together with other periods, as it is unclear how SSIRI believes they should offer model timetables or advise schools and MoE offices on possible adjustments of timetables to most effectively synergize LV and other English and math periods. 23 At Ronyi P/S in Yei the school has formally signified the placement of LV in the timetable, but this does not reveal how it relates to the other English and math periods. Mid-term Evaluation Recommendation 2 (f): Promotional and Communication Strategies Recommendation is being implemented. EDC has hired a consultant from Nairobi, Charles Ndoki, to develop and help implement a SSIRI strategic communications plan. The strategies already implemented include: • Launching events (2) • Promotional Radio Spot Announcement • Promotional Materials (such as flyers and T-shirts) • Unpaid Advertising (via Radio and Newspapers) • Press Releases • Paid Advertising • Public Gatherings The strategies still to be implemented are: • Town Crier Announcements • Letters to NGO Partners Mr. Ndoki’s report, as footnoted elsewhere, is provided as Annex 10. Mid-term Evaluation Recommendation 2 (g): Partnering with NGOs for pre- and in￾service teacher training Recommendation is being addressed. The mid-term evaluation singled out Windell Trust as a logical teacher training partner for SSIRI. SSIRI has now begun to implement cooperative work with Windell study groups in Wau. The Consultant asked to see a Windell group using SSIRI materials in Wau but the Senior Outreach Advisor said that none of the 13 Windell-SSIRI groups were meeting that day and others had just completed the course. A SSIRI staff member in Wau was familiar with Windell Trusts’ activities and offered this: “I was a Windell Trust tutor last year and I know them. These teachers enrolled there are teaching in Arabic. It was different last year in the program. Now because of the (SSIRI) radio there are more activities. They (the teachers) stay for 3 rounds of 6 months so it is an 18 month program.” (Simaya Kemji Modi, SSIRI M&E Assistant) EDC also remains a partner of Creative Associates’ HEAR project.9 Mid-term Evaluation Recommendation 3: Monitoring and Evaluation Systems (and OTT #2 - Provide appropriate staffing, training, monitoring and accountability, 9 Though EDC is expanding its support to the Educational Resource Center (ERC) in Kauda where HEAR is managing the center, the EDC Regional Director clarified that this is not through the SSIRI project. EDC has advertised to hire an Educational Technologist to work in Kauda at the ERC to help develop the ERC’s capacity, technologies, services, and materials to enable to Center to better assist teachers in Southern Kordofan. EDC’s assistance to the ERC may start with creating digital libraries like those SSIRI has developed in Maridi. 24 and OTT #4 - Senior level monitoring) Education Audit Findings: EDC reported results not supported by available data (and OTT #3 - EDC staff – capacity building and accountability) Review Activity (g): Assess the quality of the EDC monitoring process including the extent to which there is regular, informative feedback from the field sites, i.e., detailed descriptions of ongoing class visits and reporting of basic data. Recommendation has been implemented. The SSIRI project has implemented the recommendations of both the mid-term evaluation and the education audit and instituted significant reforms to its monitoring and evaluation systems. Weekly reporting by field staff has been intensive. Notable are the 417 Weekly Reports that were submitted to management between June 1 and October 31, 2008. (See Tilson’s spreadsheet recording reports submitted, Annex 11: SSIRI Outreach Staff Reports as of Nov. 10, 2008). With a 32 staff responsible for such reporting and 20 possible reporting periods this represents 65% of all possible reports, not adjusting for illness, scheduled excused leave, etc. We have no data on similar numbers and rates in earlier periods, though the mid-term evaluation did note a severe lack of regular reporting. The Consultant can thus state with confidence that these 417 reports represent a major improvement in SSIRI monitoring and accountability processes. “There was pressure to produce numbers at the expense of quality. We had to reevaluate what the project was achieving. We’re still looking at the numbers but also at the quality. We are putting more pressure on the field staff.” (Maree Melican, Deputy COP, SSIRI) “Before that document (One Teacher at a Time) we had great difficulty getting reports from field staff. It (OTT) compelled them to send them in – and we got metrics – and if we were not getting them the 5 OA’s or Tom could send a note for them to send it in. Some could not send them. This made us relocate them – like two in Magwi and Nimule. They were far from a supervisor. The OA complained that they were far and they were moved to Torit. Those who used to write a few sentences only now are writing 4 pages - and it is quality reporting. It changed drastically.” (Edward Kasran, Senior Outreach Advisor, SSIRI) “In June the reporting system changed. We had to plan the whole month and send weekly achievements, goals and issues.” (Angelo Guido, Outreach Advisor, SSIRI, Kauda) Outreach Coordinators (OCs) and Outreach Advisors (OAs) have also been charged with normalizing and universalizing their application of the field monitoring forms they are required to complete and return to the Juba office.10 These forms include: 1. School Registration Form 2. Teacher Registration Form 3. Terbia Facilitator Registration Form 4. Terbia Enrolment form (for Terbia A and Terbia B learners) 10 “Perhaps the single most important action to be taken in order to improve the implementation of SSIRI programs is to enact a monitoring and reporting process that works well. Each outreach staff member will be held strictly accountable for monitoring and reporting activities.” (SSIRI communication to staff in “One Teacher at A Time”, June 7, 2008) 25 5. Monitoring Form – Learning Village (for classroom/lesson observation) 6. Monitoring Form – Terbia (for learning group/lesson observation) 7. Digital Device Distribution Form 8. Teachers / Facilitators Guide Distribution Form 9. Log form for Learning Village teachers (to track lessons completed) 10. Log form for Terbia facilitators (to track lessons completed) Forms relevant to the PMP indicator figures are now signed and aggregated in Juba in blue binders organized by state. Monitoring form binders Teacher Registration Form For the current review, a desk audit was conducted by sampling some of EDC’s component PMP indicator figures and seeking substantiating documentation for them. This was done in the presence of EDC’s M&E Specialist, Janet Mola, just prior to her leaving SSIRI for doctoral studies in Canada. For the required indicators the Consultant identified figures that represented component numbers contributing to SSIRI’s PMP indicator totals (e.g. the number of teachers in Western Equatoria that were trained in the 2007-08). The forms were counted and/or figures added to validate reported subtotals for key PMP indicators. In each case the subtotal number was supported by the documentation on file. SSIRI’s adjustments in monitoring and data maintenance have addressed the major finding of the Inspector General’s education audit, namely that there was insufficient data available in June to support certain PMP indicator figures presented by EDC. The data sampled for this review provided substantiation of the numbers contributing to these PMP indicators: 1. Teachers/facilitators trained (from teacher registration forms) 2. Number of pupils/learners utilizing the SSIRI programs (from teacher registration forms) 3. Radios and other equipment/materials distributed (from school registration forms) 4. Number of education officials trained (from training reports) With regard to #1, teachers/facilitators trained, the number EDC reported as of a September 30 was 1283 (240 female, 1043 male). With regard to #2, this number must include all enrolled Learning Village pupils and Terbia B learners. The enrolled Learning Village pupils EDC reported as of September 30, 2008 were reported as 30,769 female and 38,445 male. While this sums to 69,214, the EDC report presented a total of 69,208 pupils. The enrolled Terbia B learners reported as of September 30 26 were 6,377 female and 6,065 male. While this sums to 12,342, the EDC report presented a total of 12,444 learners. There is inevitable imprecision in the actual number of pupils/learners using the SSIRI programs. The number that USAID and EDC are using for Learning Village is the number of children reported to be in the P1, P2, or P3 classes of the teachers who were trained this year, as recorded on Teacher Registration forms. It is understood by USAID and EDC that there is a problem with teacher drop-out (and possibly school drop-out) and that a significant number of the pupils counted in this way are not now engaged in SSIRI classes.11 With regard to #3, radios and other equipment/materials distributed, the number EDC reported as of a September 30 was 2591 (1188 Freeplay radios, 109 digital devices, 1194 Teacher’s Guides, 100 programs). With regard to #4, education officials trained, the number EDC reported as of a September 30 was 194 (27 female, 167 male). These figures represent the sum of the MoE officials attending various SSIRI trainings and orientations as identified in various EDC internal reports. The supporting data is not stored in the same binders as the data for teachers, students and materials. With regard to OTT adjustment #2, “appropriate staffing, training, monitoring and accountability”, in keeping with current emphasis on the quality of SSIRI implementation, and after reassessing the project’s viability in previously targeted expansion areas, EDC redeployed a number of its staff. For example, in Jonglei State, the unreliability of shortwave radio reception led to the redeployment of project staff to Kauda until such time as a reliable radio signal is available in Jonglei. There has also been further hiring (M&E assistants) and firing (e.g., a Wau￾based OC due to poor performance). Now, in view of budgetary constraints other staff may have to be laid off, while other staff must still be hired. In Southern Kordofan the project plans adding two Outreach Coordinators to serve that large area. In a November 16 letter the EDC Chief of Party, Tom Tilson, identified the following adjustments related to staff and accountability. The status/verification notes in the Table are by the Consultant: Table 3: Adjustments Related to Staff and Accountability Adjustment Status Assigned five M&E Assistants to the states where our activities are concentrated Verified: through meetings with M&E Assistants Trained M&E Assistants on the use of the tools and verification of data in the field Verified: through statements of M&E Assistants Reviewed data collection tools to allow for the beneficiaries to sign and verify data Verified: data collection tools were observed and photographed Developed a program of regular visits to schools and reports by the M&E Assistants and Outreach staff EDC’s tally sheet of received Weekly Reports showed >65% compliance (n=417). Many further reports (e.g., bi-monthlies) were provided. Scores of such monitoring reports were read. 11 EDC’s 9/30/08 Quarterly Report states, “This figure is based on the number of students in classes of newly training teachers in 2008 as recorded on the school registration forms submitted by the head teachers. These teachers have received training, a radio or digital device, and a teacher’s guide. This number does not reflect the actual number of listeners at any given time during the school year.” The known issue of teacher drop-out after training means the number actually benefitting may be significantly less, though this number is unknown. 27 With regard to OTT adjustment #4, “senior level monitoring”, EDC reports that either the Deputy Chief of Party for Implementation or the Chief of Party have been visiting schools at least every two weeks. References to these visits can be found in weekly and other reports. Mid-term Evaluation Recommendation 4: Investing in teacher training, especially in￾service training (and OTT #8 - Implementation of Professional Studies for Teachers – PS 101) Review Activity (i): Assess progress of implementing the teacher training program PS101 Recommendation is being addressed. EDC has launched the PS 101 course in pilot form. 19 in￾service teachers attended a one-day training given by EDC on November 21, 2008 in Yambio. Two “local instructors” have been hired by EDC on a short-term basis to travel to the schools of the 19 in-service teacher trainees to observe and support them throughout the 12-week course, which is supported by 12 audio programs and a course manual that includes 12 assignments. Each of the 12 units has 4-6 pages of text with three assignments to be done after listening to the audio programs. By the end of each week teachers are to have heard each program two times. For each audio program there is generally one written assignment, one classroom-based exercise and reflection, and one discussion with a fellow teacher followed by a written assignment. Completed assignment pages are to be removed from the book and given to the trainee’s instructor for grading. Teachers are to share experiences with their instructor, who also observes them teaching. Together they are to agree on observed areas of teaching performance that need more work. As the PS101 trainer in Yambio said, “He (the local instructor) is not an inspector, but a friendly observer”. The titles of the 12 weekly units are: 1 – Introduction to PST 101* 2 – Group Work #1 – Think Pair Share 3 – Using Teaching Resources 4 – Time Management 5 – Promoting Learner Attendance 6 – How to Create an Orderly Classroom 7 – Using Lesson Plans 8 – Establishing Classroom Routines 9 – Questioning Techniques 10 – Group Work #2 – Creating Independent Learners 11 – What Makes a Positive Learning Environment 12 – Concluding Program* *To be listened to as a group w/ local instructor There is no service credit given to in-service teachers for this course, but a SSIRI certificate will be awarded.12 12 EDC said that Edward Kokole, MoE Director of Teacher Education permitted SSIRI to go ahead and use the existing materials even if the full four-year in-service course and the Community Education Centers are not ready. 28 There is not yet a plan for evaluating the PS 101 pilot project. In reply to a question from the Consultant about this, a suggestion was made by the Deputy Chief of Party that SSIRI work closely with the local instructors to develop an evaluation component. While there are serious issues delaying the debut of the MoEST’s long-expected in-service course, which was to rely on the inchoate County Education Centers (CEC), we were informed by World Bank/MDTF representative, Getahun Gebru, that between seven and nine CECs are complete or are nearing completion. However, there is much new uncertainty about the way forward for an in-service course for Southern Sudan, since the recent collapse of the contracting mechanism for this program described as being in its final stages. While this is immediately troubling, this could also be an opportunity for new applications of PS 101 and similar follow-on audio-based modules, or for other distance education strategies for in-service teacher training. One suggestion made by the SSIRI COP was that the TTIs might be able to double their annual output through a one-year, rather than two-year residential program followed by a period of distance education. EDC has also provided PS 101 materials to the TTI in Yei for possible inclusion in the pre￾service program. The Outreach Advisor for Special Programs is to follow up by meeting with the professional studies tutors to see how the audio and written materials can be used as a resource for Year 1 student teachers. SSIRI is considering doing the same for Maridi and Arapi. EDC has also provided 43 Year 2 student teachers at the Yei TTI with three days of Learning Village utilization training. Additionally, one tutor attended a one-day training on how to incorporate Terbia A into a teaching program. SSIRI is considering doing the same kinds of trainings at Maridi and Arapi with the objective of providing basic LV teaching skills to all teachers before they graduate from TTIs. This could reduce future LV training costs. Review Activity (j): Assess the use of learning technologies in support of the SSIRI activities including the Lifeline radios and digital devices such as MP3 players. At the Maridi TTI the Consultant will assess the extent to which the computer center with VSAT is being used effectively by the staff of the TTI for their own development and for benefit of their classes. (and OTT #9 - Technologies for strengthening teacher education at TTIs) (See preceding commentary for findings called for above on radios and digital devices.) With regard to TTIs: At Maridi, based on a short visit, the VSAT, which enables Internet connectivity, appears to have a growing impact. The Instructors and Principal at the Maridi TTI expressed continuing gratitude to EDC for the assistance rendered in providing a VSAT, a computer center, and training in computer applications and in video. In particular they noted the powerful role that email, Internet and real-time professional exchange with other TTI colleagues (Skype-chat) have played in their ability to improve the content of the courses they can now offer their students. For example, an Internet usage log form shows educational materials and methods sites that have been bookmarked by tutors. Tutors also provided digital copies of materials they have downloaded and organized for use in courses, under such folders as: Philosophy, Science, Animal Science, and Mathematics. While in May there was little evidence of the Internet 29 actually being used in TTI instructors’ lesson development, on this visit the instructors emphasized that the content of their courses has been positively affected by access to online resources and to the ideas of other TTI instructors they are now relating to on-line. TTI staff are also proud of the outcomes of the video training provided by EDC that resulted in a produced program on personal hygiene, which they showed the review team. (see Annex 12: Video Consultant’s Report) The pacing and clarity of this video program was greatly improved from the one shown to the Consultant in May. Mid-term Evaluation Recommendation 4(a): Re-examine Strategies to Enrich IRI Training for New Teachers (and OTT #5 - English ability of teachers) Recommendation is being addressed. Teachers and Outreach Coordinators and Outreach Advisors report that the training given to new Learning Village teachers has grown from 3 days to 4-5 days. This change was intended to help teachers become more confident in their ability to understand and translate the English instructions in the audio programs. The results of this lengthening were impossible to assess in the short review period. Some teachers observed did not or could not follow certain instructions from the radio, while others performed accurately and well. On the whole, the trained teachers observed who reported having had 4 or 5 days of training seemed to cope adequately with the programs. But we do not yet know how the content of the training for Learning Village or Terbia has changed. Training outlines provided pre-date June 1, 2008 and still reflect the three-day design. The consultant employed to improve SSIRI teacher training strategies worked with the SSIRI project for a period of time but has not delivered any revised training materials or session outlines or other work product. In view of this it would appear that EDC has only partially addressed this recommendation. While lengthening the training from 3 to 4 or 5 days is surely helpful, other strategies are necessary to produce significant improvements in the English capacity of the SSIRI teacher corps. For example, the mid-term evaluation recommended that a module in the standard training be devoted to the basics of communicative language teaching. Also intended to have similar or related positive results were recommendations 4 (b), 4 (c), and 4(d) – see below. Mid-term Evaluation Recommendation 4 (b): Additional School-based Support for Teachers Mid-term Evaluation Recommendation 4 (c): School-based SSIRI Training Mid-term Evaluation Recommendation 4 (d): English Resource Corners for Teachers’ Self-paced English Language Development Recommendations were not addressed. Each of the three recommendations immediately above was offered as a best practice to address English language deficits among teachers in Southern Sudan. Because these recommendations would require expenditures beyond the scope of the original project design, they were recommended for inclusion in any RFP or proposal for an extension to the project. Re: Major Review Question 4: Assess if the learning technologies including radios, digital devices, and computer labs/VSAT equipment are scalable and sustainable. 30 Findings related to this question have been presented above (e.g., assuring clear strong sound, digital devices, economic study, etc.). The question also calls for interpretation of such findings, and this will be addressed further in Conclusions and Recommendations. Review Activity (h): Comment on factors, both within and outside of EDC’s control that may impede the implementation of the SSIRI programs. While there are a great many possible factors, we highlight here a few potential impediments: Systemic weaknesses in the Southern Sudan education sector - As written in an address to the review team by the Director of Basic Education in Wau: “…the infrastructure remains in desolation… the children sit on the floor… the lack of teachers…ratio of teachers to pupils is 1:80… most of them are Arabic Pattern teachers… one third of the school going children are not attending classes due to lack of teachers and learning space… As part of the MoE, SSIRI is not an exception but is unfortunately rowing in the same boat. Most of the teachers SSIRI trains to teach the program are the same teachers described above. It is the same type of schools also described. As a component of the Department of Alternative Education Systems any problem the Ministry goes through SSIRI bears exactly the same.” (Donato Ugali Zimbeli, A for /Ag Director General, Director of Basic Education, MoE, WBEG State, Wau) Unmentioned in this category are: • the failure of the MoEST and SMoEs to pay teachers fully or on time • absence of transportation for inspectors and supervisors • lack of basic learning materials, and any number of other deficit conditions “The biggest problem we face is the challenge of teaching - as they complain every time we go to the schools that the government does not pay them. It makes reluctancy not only for SSIRI but for any education in the school.” (Ale Peter Michael, Outreach Coordinator, Yei) And problems could arise from the inability of the system to provide SSIRI services to all: “Why are some not given radios? You should try to cover as many as possible so that none complain. If there are only 5 counties there will be a challenge from others.” (Hezekiah Wayo, Director for Basic Education, SMoE / WES / Yambio) Miraya-FM’s possible reticence to continue broadcasting SSIRI programs – Miraya’s Station Manager, Huub Gales, is concerned that the Chief of Public Information at the United Nations Mission in Sudan (UNMIS) Mr. Khaled Mansour, is not an enthusiast of SSIRI programming. By contrast, Gales is supportive and believes that SSIRI’s is just the sort of programming that falls within Miraya’s proper mission. Further, Miraya has no good programming to replace SSIRI’s. However, because SSIRI remains on Miraya’s air at the pleasure of Miraya’s U.N. management, Mr. Gales strongly recommends that a formal MOA be negotiated with Mr. Mansour’s office very soon, as decisions may be taken in the next months. Tom Tilson believes that the MoEST should lead or co-lead the request to Mr. Mansour, together with USAID. 31 B. CONCLUSIONS Major Question #1: Assess to what extent EDC has implemented recommendations from the May 2008 evaluation and the success of these interventions in addressing issues identified. Summary of the status of EDC in implementing the recommendations: Table 4: Evaluation Recommendations Status 1. Assuring clear, strong sound in all SSIRI learning environments Being addressed 1.a. MP3 for TERBIA B Groups Being addressed 1.b. Study of Economics of IRI by Radio and MP3 (or other codec) for Formal and Informal Settings Not addressed 1.c. Marketing and P/R: Rename “Low-Cost Technologies” Implemented 1.d. Medium Wave (MW) Radio Stations Being addressed Audio Programs 1.e. Get the Pauses Right Not addressed 1.f. Consider Longer Teacher Led Activity Sections Being addressed-P4 1.g. Examine all Songs for Sing-ability Being addressed-P4 2. Working in Communities Being addressed 2.a. Model teachers/classes Implemented 2.b. Choosing Project Sites Carefully Being addressed 2.c. Continue to Focus on Relations with the States and Counties Implemented 2.d. Counterpart Identification, Incorporation and Training Being addressed 2.e. Relationship of the Directorates of Basic Education and AES Being addressed 2.f. Promotional and Communication Strategies Being addressed 2.g. Partnering with NGOs for pre- and in-service teacher training Being addressed 3. Monitoring and Evaluation Systems Implemented 4. Investing in teacher training, especially in-service training Being addressed 4.a. Re-examine Strategies to Enrich IRI Training for New Teachers Being addressed 4.b. Additional School-based Support for Teachers Not addressed 4.c. School-based SSIRI Training Not addressed 4.d. English Resource Corners for Teachers’ Self-paced English Language Development Not addressed Education Audit Findings EDC reported results were not supported by data (as in #3 above) Implemented Overview: Based on the findings presented, we conclude EDC has implemented or begun to implement 18 of the 20 specific recommendations of the Mid-term Evaluation and of the Education Audit. We include in this calculation those recommendations that conform closely to the requirements of the Cooperative Agreement. We exclude Recommendations 4 (b), 4 (c), and 4 (d), since these were proposed as optional best practices to integrate should there be an extension period. We conclude that concrete progress has been relatively slow in Area 1: Assuring clear, strong sound in all SSIRI learning environments. Progress has been substantial in Area 2: Working in Communities. Progress has been very substantial in Area 3: Monitoring and Evaluation Systems. And progress has been very slow in Area 4: Investing in teacher training, especially in-service training. The discussion below parses and supports these broad conclusions. 32 Major Question #2: Determine if these actions have achieved the changes envisioned in the May 2008 evaluation recommendations. If so what specific changes/outcomes have been achieved? Among the specific outcomes that have been achieved are:    1) Improvement of the audio experience for teachers and learners through a. the use of two radios in a room is becoming common (observed in 80% of cases where radio was the audio equipment used) b. expanded use of digital devices where radio is not a viable option c. more FM stations being contracted and now broadcasting SSIRI programs d. one additional AM radio station in Upper Nile, and possibly one in WBEG 2) New thinking about the design of P4 Learning Village series that is taking into account a) critiques from the May evaluation, b) findings of an internal assessment, and c) results of formative evaluation conducted in Southern Sudan instead of Nairobi 3) Institution of the model class/teacher approach resulting in some schools and learning groups under each Outreach Coordinator and Advisor that are successfully using Learning Village and Terbia B and so are earning the increased confidence of administrators, teachers, parents & learners as well as an increasing demand for SSIRI services 4) A much improved M&E system that has addressed the main issue raised by the audit of the Inspector General and that has resulted in greater staff accountability and job satisfaction. This has improved the performance of teachers and helped establish or deepen positive relations with MoEST offices at all levels down to the Payam; 5) More expressions of SSIRI ownership from the MoEST GoSS and State levels; 6) Raised public awareness and institutional acceptance and integration of the SSIRI project in the MoEST at various levels through co-location of SSIRI and MoEST staff (in some offices) and/or frequent professional visits (in other offices) so the two staffs may work cooperatively and so SSIRI staff may provide transport and other support to counterpart MoE staff; 7) A range of activities in a strategic communications campaign a. e.g. launches in Yei and Kajo Keji; and b. elimination of the term ‘low-cost device” referring to digital playback devices; 8) Initial work with NGO partner, Windell Trust, to build English skills among teachers 9) Use of Terbia A in secondary schools not observed in the May evaluation 10) Launching of the PS 101 course for in-service teachers on a pilot basis 11) Lengthening the initial training of IRI teachers & MoEST staff to 4 or 5 days from 3 12) Contracting for SSIRI broadcasts on a number of additional FM and AM stations so that Miraya is not the only non-shortwave SSIRI broadcaster 13) Removal of the impediment of irregular shortwave signal quality, which is fine for news and personal listening and, possibly for nighttime or early morning basic education but that EDC has assessed as being of insufficient quality for basic education in South Sudan for groups meeting in the daytime. 14) Maridi TTI staff expressing continuing appreciation for EDC technical inputs enabling production by TTI staff of new work products (lesson plans, course outlines and video programs) 33 Area 1: Assuring clear, strong sound in all SSIRI learning environments Given the realities of broadcasting in Southern Sudan at the moment, and given the choice to use one or another of the current SSIRI equipment options, the only way that strong clear sound could be assured in all learning environments would be to deliver all programming in prerecorded form with digital playback devices – preferably the large 12-volt ones. We see the value in a serious study of what this would mean financially and practically. Such a choice would pose a delivery problem in the short term where new programs were being produced continually. However, where the program series have been completed, frequent delivery is not a major issue. Loss of recorded media would be an added problem. And there are many other factors to consider. While the clarity and volume of sound are the most compelling benefits of recorded media, there is also the benefit of being able to stop and start programs and to back up within programs. There are still, however, significant unknowns in Southern Sudan with the suitability of digital playback devices and their power sources. Issues of long term costs, maintenance and durability of equipment, etc. all have to be assessed further - both with regard to utilization of digital devices and radios. Such a study should provide adequate basis for decisions about the alternatives and degree of mix of technologies to be employed should the SSIRI project be extended, or should another similar project seek to apply lessons learned here. While recorded media are becoming more feasible at scale, the possible cost advantages and simplicities that radio may continue to offer need to be factored. An improved next generation of wind-ups could change the current picture substantially, especially if radio transmission issues and timetabling can be managed well. FM Transmitters: Miraya FM has been SSIRI’s mainstay broadcaster. It uplinks from Juba to a passive satellite that distributes centrally originated programming to local downlink transmitters. Its source signal strength is adequate but as it emanates from the many downlink transmitters the audio output is variable and often low. This is a problem when the low signal is received by the Lifeline radios used by the project since they also have a low output level. According to SSIRI’s consulting radio engineer it may be possible to improve this condition by installing low cost signal processing equipment to boost the audio output of Miraya transmitters. EDC has broadened its range of FM options by contracting with a number of new FM stations, and it plans to continue to do this as more become available and as needed. Some of the new stations appear to provide a louder signal than their neighboring Miraya FM station. This is a positive development and has much to do with the possible sustainability and impact of the project. Each of these stations will have its own technical issues, but none seem insurmountable. AM Transmitters: EDC has contracted with the AM station in Malakal. This bodes well for coverage of a wider area than is possible with any single FM station. The quality of the audio from a Lifeline tuned to these AM stations needs to be assessed, but we expect that it will be acceptable. It should also be free of the crosstalk problems SSIRI experiences now on FM. The AM station in Wau has not yet been contracted by SSIRI and this needs to be addressed soon because 1) the problems with FM crosstalk may not be resolved quickly, and 2) the range of the Wau FM is not yet well understood and the AM range should be far greater and in any event will enable a more equitable service pattern. 34 Radio Receiver Technology: The quality of sound from the Lifeline radios is adequate for language education, but should be significantly improved. There is high frequency loss due, we think, to the low speaker quality; and there is generally a low output volume from these radios. This reduces the effectiveness of these machines, which otherwise are a revolutionary aid in rural basic education. Improvements in the specifications for future purchases should be possible. Crosstalk: There is a significant problem in using the current set of Lifeline radios for FM reception of stations other than Miraya-FM due to crosstalk. As a result of this review, this problem is now being researched by the manufacturer. The possible impact of their findings is unclear, since there are some 4500 of these machines in storage and retrofitting an effective solution may be expensive, or nearly impossible. On the other hand, SSIRI’s consulting radio engineer now plans to install a substitute narrow band filter for the “wide band filter” in the Lifeline radio. He estimates that this will take only about 15 minutes per radio. So a fix for some of the inventory may be possible. With deployment of certain modified sets in certain areas where needed, not all would need to be fixed. We look forward to hearing the results of this test. Digital Devices: By far the best SSIRI educational experiences we saw involved the use of the largest of the digital devices SSIRI is using. This is a 12-volt Sanyo “boombox” that accepts MP3 files on a flash memory stick – or can function as a radio or CD player. The audio output is loud and clear. It can serve small or large groups (even 100 or more) indoors or outdoors. Its eight rechargeable “D” cells have an unknown lifespan and can be recharged an unknown number of times. A study of the lifespan of the various elements needs to be done. A 12-volt system like this can also be wired to run off a car battery provided by the community. The small digital device, the Nextar unit, is practical and easy to carry in a purse together with the small stereo speaker unit SSIRI has specified to date. Teachers are succeeding in teaching with this equipment in small Terbia groups. However, because it takes triple-A batteries and because the rechargeable triple-As that EDC has acquired do not last through an entire lesson, there is a usability problem that must be addressed soon. The recharging of these batteries is being done by two means, hand cranking, which seems to take too long to be practical; or by solar panel. If Terbia classes end in the evening it may be difficult for the facilitators to have the time each morning to recharge batteries using solar panels. This is obviously not insoluble and should not in itself inhibit the use of solar panels. Improvement of the Audio Programs: SSIRI has not had time to revise the P1-P3 programs and they do not intend to do so before the end of the current project cycle (June 2009). SSIRI’s production focus has been on the new Learning Village series for P4. In preparation for this it did an internal assessment of the P3 programs and identified certain design features that need to be corrected in current and future production. These features correspond to those identified in the mid-term evaluation. Among the features that are to be addressed in new production are songs that are more stylistically appropriate and more sing-able; program pacing that is not as fast as it was, more intelligible character voices, and more frequent use of longer teacher-led activities. Consequently, P4 should include substantial improvements. Notable is the fact that formative evaluation (FE) of the entire series is scheduled to be done in Southern Sudan for the first time, rather than in Nairobi. Initial FE in Southern Sudan has already highlighted the need for revisions of initial P4 programs. 35 Area 2: Working in Communities Relations with the MoEST, SMoEs and CMoEs – Since June, the relations with the central MoEST have remained as good as they were before or they have improved. At the state and county levels it appears that the SSIRI project is winning new admirers and enlisting enthusiastic partners. We saw nothing to indicate that SSIRI was being merely politely tolerated. The appreciation is real. Counterpart Identification, Incorporation and Training – While formal training of counterparts has not been done, and while development of a training of trainers for SSIRI staff and for their S/CMoE counterparts remains undone, SSIRI – S/CMoE collaborations are deepening. By co-locating some SSIRI outreach offices with state and county AES Inspectors’ offices, and through collaborative travel for monitoring, enabled by motorbikes having been provided to SSIRI local staff, a spirit of shared enterprise is developing between SSIRI and the S/CMoEs (e.g. Yambio). There is evidence of more day-to-day cooperation and integration of monitoring activities of SSIRI and counterpart staff. SSIRI management is aware that the future of the project depends on passing the skills for training and support to the state and county and payam education offices. More needs to be done in this regard. Communications and Marketing: SSIRI has implemented a broad communications strategy, calling on the services of a public relations and communications consultant. This has resulted in successful launch events in Kajo Keji and in Yei, distribution of T-shirts and flyers, free advertising, etc. Public awareness of SSIRI is reported to be growing and with it demand from parents that their children participate in SSIRI-supported classes. Working with NGOs: EDC has begun to work with Windell Trust, providing Terbia programming resources to them for the English language training of teachers, many of whom are Arabic Pattern teachers. The degree of integration of SSIRI into Windell Trust’s English teaching practices and its effects remain to be assessed. Area 3: Monitoring and Evaluation Systems Monitoring and Model Classes: We conclude that two of the most important changes made in the project since the mid-term evaluation have been the model class approach and the regularization of SSIRI monitoring and reporting from the field. Model classes receive the benefits of the One Teacher at a Time approach, being visited frequently and providing Outreach Coordinators with a laboratory in which to study the determinants of SSIRI program success that they can then analyze and report to their managers. While many of the structured monitoring forms had been used in the project already, their further refinement and the requirement that they be consistently applied led to much higher and more dutiful reporting by SSIRI staff. We noted an over 65% compliance rate in weekly reporting with more than 415 weekly reports and scores of other reports and data being submitted to SSIRI management and M&E staff between June 1 and October 31, 2008. Based on the reading of scores of such reports, we conclude that significant and regular attention is now being paid to issues of field implementation and teacher performance. This is borne out by the fact that most lessons observed were well within the range 36 of utility and educational value to the pupils and learners. Senior Outreach Advisors, M&E Advisors and Outreach Coordinators all speak to the efficacy of the new reporting requirements and to improved communication with SSIRI Managers (DCOP and COP). Area 4: Investing in teacher training, especially in-service training Counterpart Training: As noted above in the discussion of Area 2, among the additional critical work that needs to be done are the development of trainer trainings for SSIRI trainers and staff and for SMoE and CMoE counterparts; and packaging of user-friendly training materials and designs for Learning Village and Terbia teacher trainings to be provided to the S/CMoEs. Professional Development of Teachers: EDC has launched the PS 101 course as a pilot effort. It had intended to integrate this course with the long awaited four-year in-service course. However, we learned during the review that the contract for this project (years in proposal and negotiation) has been scrapped. If one is being put forth in other circles, no one shared with us a vision of the future of in-service teacher training. However, if there is a vacuum, this may provide an opportunity for SSIRI to work with or assist whatever new vision is emerging and to apply SSIRI’s core competencies to developing a set of distance learning modules for teachers. There is discussion of resuscitating the Fast Track Initiative as a part of a new in-service training model and if so SSIRI may be able contribute in this context. SSIRI has continued to support the TTIs in Arapi and Maridi with IT support and training. The tutors at the Maridi TTI are clearly excited by their professional growth and greater sense of personal currency with world affairs, and by the up-to-date teaching strategies and methods they are experiencing through SSIRI enabled internet access. Inter-TTI tutor collaborations have also begun. SSIRI has also provided materials from all its courses to the TTI at Yei, and it has trained 43 student teachers there in Learning Village utilization with an eye to collaboratively working out how SSIRI may be integrated in pre-service teacher education. Major Question #3: From classroom observations, assess to what extent the SSIRI Learning Village programs seem to be working as a teaching and learning tool in the classroom. And assess to what extent the English Terbia for Beginners programs seem to be an effective approach for teaching English. The effectiveness of both Learning Village and Terbia for Beginners can only be accurately determined by a structured approach to pre- and post-testing testing with control groups. To the degree that this can be assessed (at the level of “seems”) during these nine days of visits, the Learning Village appears to be effective in assisting teachers in introducing children to basic English and math. The Terbia B programs similarly seem to be engaging both adult and young learners in productive learning. Terbia B learners were observed following English instructions and responding as called for by the teacher and the audio. In the case of LV, observations revealed that during not all but many of the P1-P3 lessons, the frequency of children’s English utterances is low. These lessons could be more effective in developing pupils’ English language skills if the frequency of English speech demanded of both teacher and pupil by the audio program itself can be increased; and if the other suggested 37 revisions to P1-P3 are completed following a systematic review. This said, the lessons are clearly delivering value. Teachers note that students are more motivated and more interested in the lessons, that attendance increases when the radio is used, and that their learning of English, and especially their speaking skills are increased. This assessment is based on many observations and on discussions with many model and non-model teachers, and with administrators in their schools and local education offices. County and state administrators also say that parents are demanding SSIRI services in their schools. These administrators are urging SSIRI to expand. On the other hand, SSIRI staff and some state education administrators say that many teachers have dropped out of the program. By voting with their feet they seem to be revealing that, for whatever reason, teaching and learning with SSIRI programming may be impeded by their own capacities, by problems in radio reception, or by other implementation issues. In fact, we did see at least one class that the Consultant thought would have been better off without the radio and without the struggle that day with crosstalk. However, in no case when there was a program being broadcast was the radio signal so weak that the teacher could not hear it, even if it seemed the children might not be getting much if any value directly from the radio in instances when maximal radio volume was still extremely low. Still if the radio at low volume was providing the teacher with a framework without which there would have been a much inferior lesson, the radio is adding value for the pupils. Teacher drop out could be caused as easily by the evident widespread failure of the MoEST to pay teachers their salaries in full and on time. It is no wonder that teachers often fail to show up for work, or decide to leave the teacher corps altogether – and take their SSIRI training with them. As the Director of Basic Education in Wau reminded the review team, the environment of South Sudan education is a difficult one and SSIRI “is rowing in the same boat.” While serious implementation challenges remain, they can be surmounted. What has been demonstrated through this review is that in every state there are successful SSIRI classes. There is no absolute reason that such success cannot be generalized to a great many more schools. Major Question #4: Assess if the learning technologies including radios, digital devices, and computer labs/VSAT equipment are scalable and sustainable. Practicality and scalability of these technologies will depend on a number of factors. Some of the most important factors, like continued interest and support of funders and governments, are beyond the scope of this assessment or are unknowable. With adequate funds nearly anything physical can be sustained. Of course, expenditure of funds only makes sense if the intervention is providing real benefits in a cost-effective manner compared with other options, or if there are no other good options. To understand the benefits it is important that the learning gains attributable to the Learning Village series (especially) are reliably tested and assessed. If there is no substantial benefit of LV over conventional classes then there is no point in pursuing scalar development. The question also exists as to how much self-financing and management is reasonable to expect of the MoEST and its MoE structures now or three or five years from now. No understanding of scalability and sustainability can be done without economic and financial analysis of the costs associated with all the elements likely to be needed to support SSIRI’s approaches to media-enhanced basic education at scale. For Learning Village, these will include 38 radios and digital devices, solar panels, rechargeable batteries, recording media, necessary print, training provided by project staff or MoE staff, travel, teacher fees, broadcast fees, etc. For TTIs these will include recurrent costs of maintaining internet connectivity, electrical power, etc. Such a study is a key recommendation. EDC has not worked out all the issues in sourcing and importing the particular equipment for LV and Terbia. Certainly the most successful classes we saw were run with the digital devices. It appears, however, that the Lifeline radios (or even a higher quality wind-up successor model) may remain less expensive for some time to come – both to acquire and operate. It would be interesting to make, as part of a control group study, an analysis of the possible learning gain differentials between pupils using the current crop of Lifelines radios and pupils using the larger and much more audible digital devices. We believe these devices may produce added educational results that should be considered before the more expensive digital devices are sidelined in favor of radios (the cost of which must also include recurrent broadcast fees). There is no reason we can see that the VSAT and computer labs should not be sustainable with a modicum of financial commitment from government and/or funders. In the case of Maridi TTI, SSIRI staff have been instrumental in maintaining the network and lab. But such skills can be transferred to the TTI or to a shared staff of the TTI and the CDC, or possibly contracted locally. Review Activity (h): Comment on factors, both within and outside of EDC’s control that may impede the implementation of the SSIRI programs. There are a number of possible impediments to successful ongoing SSIRI implementation. Within EDC’s control are such factors as: • Failing to negotiate an MOU with Miraya. As noted in the findings, it is possible that the officials in charge of Miraya radio may seek to eject SSIRI from its air. This would be a very serious setback to the project. • Even in an extension project of limited scale, failing to correct audio reception issues in places where they are serious enough to reduce effectiveness or cause teacher drop out; • Discovering that the useful life of equipment is much shorter than anticipated, resulting in increases in costs and complexity of implementation; • Failure to revise and correct the problems with P1-P3; • Failure to demonstrate the impact of IRI on student learning; • Having to lay off staff or failing to provide the necessary inputs that are needed to develop and maintain effective field operations and MoE relations due to the current budgetary constraints; • Failure to provide the means of transportation for SSIRI outreach staff and, to the degree possible, for local MoE inspectors and supervisors; and • Further erosion of the trained teacher implementers, which could take place if audio quality in the classroom cannot be guaranteed. Largely outside of EDC’s control are such possible factors as: • Funder fatigue; • A possible decision by Miraya’s senior managers to cancel SSIRI broadcasts; 39 • The lifespan of the hardware now being used, and of most concern being the rechargeable batteries, is unknown and if it is short, this could be costly and interrupt service; • Absence of the means of transportation for local MoE inspectors and supervisors to monitor and train could result in increasingly poor performance and drop out; • If salaries are not properly paid by government, further erosion of the corps of trained teacher implementers could occur. • Unwillingness of government to budget for SSIRI broadcast costs or other costs that over time should be theirs, such as trainer fees, etc. While this is not an immediate problem, scalability and ultimate sustainability of the project may depend on the GoSS being willing to bear such costs. Absent that, transfer to them of “ownership” and responsibility may be very limited or impossible. C. RECOMMENDATIONS 1. Assuring clear, strong sound in all SSIRI learning environments Our recommendations assume that in any extension period there is likely to be a continuing role for radio, though the option for full and exclusive conversion to digital devices (at least for the purpose of Learning Village and Terbia B) should remain on the table pending the outcomes of a technical and cost study that continues to be one of our recommendations. Miraya MoU: The MoEST and USAID should secure a MoU with the UN and Miraya radio to continue to use its morning air for Learning Village programming. This should be done very soon. Mitigate Miraya Crosstalk: EDC should immediately do what is necessary to gain a complete understanding of the reasons why the project is experiencing crosstalk between Miraya and other FM frequencies on the Lifeline radios. This bedeviling problem needs to be fixed. It is not crippling to the project but it is certainly needlessly complicating things and risking further disaffection among users. Where it is a problem, specific remediating instructions need to be given to teachers (as in Yei town that they should use Miraya 101.5 until the problem is fixed and only then use Spirit-FM. In Wau they should do the same relative to Wau FM until Wau AM is contracted.) EDC should lock in a contract to use AM radio in Wau, This is highly recommended in view of the fact that the frequency of Wau FM is at 95.7, just where Miraya crosstalk has most frequently occurred. Improve Inadequate Miraya Volume Levels: Work with Miraya technical staff to resolve the problem of their relatively low audio volume. EDC may be able to assist Miraya in purchasing signal processing equipment to put on the transmitter in Wau and then moving it to do similar tests in Yei or elsewhere, to see if the levels of the stations can be raised. (If the Miraya FM signals can be raised in volume through providing spare parts now on order, or other equipment, the need for alternative FM stations may be reduced and crosstalk may become less of an issue.) Assess Lifeline Crosstalk (in-house): The crosstalk issues may need to be resolved through adjustments to the Lifeline radios themselves, possibly by replacing the wide filter with a narrow one, as recommended by SSIRI’s consulting radio engineer, Nicholas Mwangi. The results of 40 this test should determine if EDC may choose to fix the Lifeline radios that are to be used in areas where there are SSIRI FM signals on stations other than Miraya. (How many of the thousands of radios in SSIRI storage may be distributed to such locations in the future?) If this cannot resolve the problem, then SSIRI may need to seek new improved Lifelines for places where it must rely on alternate FMs and where Miraya crosstalk is remains a problem. Obtain from Freeplay Clarity on how to Resolve Lifeline Volume and High Frequency Response and Crosstalk Issues: EDC should continue direct discussions with Freeplay (and work with NDI as needed) to understand the questions raised about Lifeline receiver quality. Discuss with Freeplay the specifications for Lifelines of both the AB and AA series; being sure to get specifications for the receiver, the speakers, and any elements that control volume. Pay close attention to the variables that Freeplay engineers say are changeable in the current machines or that could be changed (for what price?) in a subsequent order for many Lifelines. Discuss with Freeplay how to improve the machines so that they (1) boost the output level to the speakers, (2) increase output of high frequencies, and (3) reduce or eliminate crosstalk with Miraya (101.5) stations. If some of the +- 4500 radios now in SSIRI storage can/should be returned or altered, do so. The current inventory of SSIRI Lifelines are certainly useful for listeners whose main FM interest is going to be Miraya-FM or who might want them for SW purposes. Consequently, if there is another USAID-funded use for such radios in Southern Sudan, or if improved wind-up radios that could be acquired for another project can be procured for SSIRI and the current SSIRI inventory given to the other project, that may provide a cost￾saving solution. Employ More FM Stations: EDC should continue to identify and contract new FM stations as scheduled expansion and budget allow. The next ones may be in Yambio or Bor. Assess AM Radio Signal Quality: Conduct an assessment the quality of AM signals in Wau and Malakal. While it is most likely that AM will offer broad coverage in these two areas and be unaffected by the similar problems as with the Lifeline radios’ reception of FM signals, there may be other issues that should be identified. SSIRI still does not know at what volume and with what clarity these AM signals will appear in classrooms in Western Bahr El Ghazal and Upper Nile states. The potential for eventual AM service from Juba should be assessed. Employ Wau AM: EDC should adjust its contract with Wau Radio to include AM. Expand the use of Digital Devices: Expand the use of Sanyo-type digital devices where they are needed and where it is economically defensible. This mode of delivery is providing the most value to users. Further Research on Rechargeable Batteries: To address the recurrent questions about the lifespan of rechargeable batteries, do research and seek reputable ratings agencies’ input on the question – possibly with Underwriters Laboratory or Consumers Reports or others. Test Rechargeable Batteries and Solar Panels: Dedicate resources to ongoing research and practical testing of the rechargeable batteries and solar panels now being used to assess such issues as (1) how many charges they can take, (2) under what if any conditions they will leak 41 into the digital player and damage it; (3) the time that the units will play files at nearly full volume after charging from an empty state for various durations. Identify Reliable Triple-A Rechargeable Batteries: Find a brand of solar rechargeable triple-A battery that runs consistently for at least ½ hour in a Nextar or retire the Nextars from service. Alkalines are no sustainable solution. Revisit Hardware Choices: Because innovations in consumer audio are so very rapid, the range of possible choices in hardware should be frequently revisited for improved feature options and for lower costs. 2. Lesson Design and Classroom Application Integrating Learning Village in Timetables: Whether through longer teacher-led activities or through more explicit guidelines in the Teacher’s Guide, there should be some explicit statement about how teachers recycle and extend the content and methods they experience in the audio lesson into teaching time after the radio lesson. Since for the foreseeable future most classes will continue to not be “self-contained” (they will have different teachers teaching the same class in primary English and math), there should be some way, especially for the math teacher who is more likely to not have been the LV teacher, to know what the math content of the LV lesson was (perhaps via an additional copy of the Teacher’s Guide). In this way, he/she can relate the other full period math lesson of the day to the content of the day’s LV lesson, as appropriate. So, too, with the additional English period, which should be at least an additional 30 minutes and not be skimped on because the LV program was slotted into what was formerly the English period. SSIRI should work with the MoE so such guidance comes to the schools directly from the MoE. The Teacher’s Guide should include suggestions at the end of each lesson about what English language communicative follow-up activities should be done to deepen the learning begun during the audio lesson. It may also include suggested math activities congruent with any other math text that may be mandated. As far as is possible, dialog between the AES and Primary (Basic) Education Directorates at the State level should be promoted and/or orchestrated by SSIRI so that common intra-state timetables can be put in place to support SSIRI integration and coordination of content across the additional English and math teaching periods each day. 3. Managing Learner Communities Expand Model Classes: Continue to expand the quality-conscious model class approach, “fading” work at existing successful schools that now have model classes (while never fully abandoning any previously supported school), and initiating or stepping up visitation and advisement in at least an equal number of new or former SSIRI non-model class schools, identifying at least one new model class in each school, but attending to all LV teachers. In this way the benefits of quality implementation as observed in this review can be spread equitably. 42 Careful Geographic Project Expansion: While there is an evident emphasis on judicious use of human and physical resources, additional immediate care must be given to guarantee the timely staging of inputs in growth areas. For example, training was scheduled for Kadugli in late November yet an Outreach Coordinator for that area had not yet been hired, trained, and equipped. What will happen in the first weeks after training and delivery of materials? Plans exist to address this sequencing problem with just-in-time hiring. For some time, however, it may be that these schools will be serviced by staff that are based hours away. Such situations should be avoided. And wherever SSIRI opens new operations, the local MoE’s state, county and/or payam offices should have formally accepted the intervention, made decisions about counterparts, and informed SSIRI about who has been identified to work with them to support the schools. Passing Training to the States: We support and amplify the suggestion of the SSIRI Deputy COP that all the training packages and modules developed by SSIRI be conveyed to the State and County MoE offices and specifically to counterpart trainers. More work should be done to codify improved training designs in manuals and to provide revised training modules to the MoE for cooperative trainings, where MoE staff increasingly take leadership roles. MoE offices can begin to take primary responsibility for the training of SSIRI teachers and support of SSIRI schools - at least at the level of advisement and monitoring. SSIRI should increasingly focus on providing trainings for MoE trainers. It can then begin to move toward a more advisory role with less emphasis on direct implementation, possibly also providing a materials depot accessible to the states under tightly managed conditions. This can begin now with a projected three-year horizon for implementing and meeting certain milestones in this transformational process. 4. Monitoring and evaluation systems PS 101: The effectiveness of the pilot version of this course now being implemented should be evaluated. SSIRI needs to apply staff and/or consultant time to this evaluation. There is not yet such a plan in place. This should be done in close collaboration with the “local instructors”. Provide Motorbikes to the County MoE: Providing motorbikes to AES Inspectors and bicycles to Payam Supervisors would allow a greater number of counterparts to play an active role in the project by visiting a wider range of schools. Formative Evaluation: Continue to do formative evaluation in Southern Sudan, not Nairobi. Summative Evaluation: A rigorous evaluation of the effectiveness of both Learning Village (and possibly of Terbia B) should be done via pre- and post-testing testing with pilot (experimental) and control groups. While there may be budget constraints, it would be best to do this in this year since the arguments for continued funding and support for the SSIRI approach will return all too soon and will hinge on demonstrated benefits. The earlier this testing begins the more likely it will be that longitudinal benefits can be quantified. There were caveats about the results of the former P1 tests. We would recommend re-doing that test and simultaneously testing at P2 and possibly P3 as well in the same schools. While test development and data crunching costs would increase with additional grade levels, travel and administration costs would be negligibly different. Getting a sizable number of subjects for such tests is attainable 43 with paper and pencil testing; and this type of test can assess listening, reading and writing. Testing speaking is much more labor intensive and may be impractical for this round of tests. The test format used for English for Ethiopia IRI might be adopted as a possible model. One exposure variable that might be interesting to control for would be the use of Lifeline radios versus the large Sanyo digital players. 5. Investing in teacher training, especially in-service training Pursue Integration of SSIRI in Pre-service: SSIRI should follow up with the Yei TTI and begin a similar project with Maridi to see if it they would like to integrate LV training in Year 2 pre-service training for student teachers, and PS101 in Year 1. Explore the MoEST’s New Plans for In-service Teacher Training: EDC should begin to explore with the MoEST how SSIRI may effectively integrate PS 101 or other of its possible outputs with whatever project model is now being contemplated for in-service training. Complete Training TA: The refreshing and improvement of SSIRI training designs that was to have been done by a consultant but that remains incomplete should be completed. Training manuals and new materials need to be produced. A consultant should, as was specified in a SOW for this work, design and coordinate the implementation of training programs for (a) EDC field staff (b) Ministry personnel at state and county levels and (c) teachers and facilitators. This would include designing “an integrated and comprehensive training program for Ministry personnel that would include the senior state Ministry personnel including the Directors for Alternative Education Systems and the senior inspectors (SSIRI, primary education, teacher training, ALP, and English), and relevant officials from the County Education Offices. This training would also likely to include the relevant officials at the payam level.” Revisit the Training of SSIRI Teachers: Part of the deliverables for the Training TA above would include a manual codifying the expanded Learning Village training as 4 or 5-day training rather than the 3-day one represented in the only such materials made available for this review. 6. Revision of Lessons and Incorporation of Assessment Insights in New Productions Schedule Revision of P1-P3: If revisions cannot be done until June 2009, be sure to schedule this for the soonest possible time after that. Maintain Commitment to Revision Criteria: When such time exists, attend to lengths of pauses, song forms, increase the number and duration of teacher-led activities, and if it is possible to do so in the revision, call for more English language usage by teachers and pupils. Music: Work closely with the hired composer musician, acting as a true music producer, so that the songs included in P4 are appropriate. If it proves hard for the composer to deliver what is needed under this sort of guidance, find another. All songs should be tested in Southern Sudan for attractiveness and sing-ability. Similarly, any revision of P1-P3 should use the same criteria. 44 7. Sustainability and Scalability Cost projections for digital devices and radio and for TTI support: EDC should do a cost study including all the elements needed to support a SSIRI approach at various scales in Southern Sudan for at least five years, and with a further five year projection if possible. Comparative analysis should be done of options utilizing (1) only wind-up radios, (2) only appropriately sized digital devices, and (3) a context-sensitive appropriate mixture of the two technologies (taking into account radio coverage areas, local school scheduling disparities that argue against radio, etc.). Recurrent costs should be isolated from capital expenditures. Elements to be costed should include radios and digital devices, solar panels, rechargeable batteries, recording media, print components, necessary training to be provided by project staff and/or (preferably) by MoE staff, teacher travel costs, media delivery and updating costs, broadcast and other fees, etc. While the above described study is of paramount importance, because SSIRI has particular objectives for TTI’s and inservice training of teachers, a similar calculation should be done of initial outlays required and recurrent costs for maintaining TTI’s computer labs and internet connectivity, and (possibly at a lower order of priority) basic video production capacity. And, should EDC wish to elaborate their PS 101 course and/or engage more extensively in the new, as-yet-to-be-designed in-service teacher training system, it should analyze the probable start-up and recurrent costs of its component contributions to that system. 45 ANNEX 1: SCOPE OF WORK Technical Review of Implementation of Recommendation of the Southern Sudan Interactive Radio Project (SSIRI) Mid-Term Evaluation (May 2008) & Education Audit Findings Completed in July 2008. Background: The Southern Sudan Interactive Radio Project (SSIRI) is funded by USAID to develop and broadcast education radio programs in support of the Ministry of Education, Science and Technology, Southern Sudan. SSIRI is a formal part of the Department of Alternative Education Systems. Currently, it works in six states and the Three Areas (temporarily excluding Abyei). The radio programs are as follows: • Primary schools – The Learning Village. 100 half-hour programs for each grade P1- P3 teaching mathematics, English, local language literacy, and life skills. • English language – Terbia. There are 180 half hour programs - 120 in the beginning series and 60 in the advanced series • Professional Studies for Teachers. Currently there is one series of 12 lessons known as PS101. In addition to broadcasting these programs on radio, EDC is implementing the use of alternative digital devices such as MP3 players with some Learning Village classes and Terbia listening groups in order to improve access in areas of limited radio reception. Finally, EDC is supporting teacher training institutions and one secondary school with computer centers and VSAT/Internet connections. In May 2008 SSIRI was evaluated by a team consisting of two external evaluators (Stuart Leigh, Team Leader, and Charles Tesar) and USAID officials. Tom Tilson, Chief of Party, also traveled with the team. This SOW is for an external consultant to assess progress since the May 2008 evaluation. RECOMMENDATIONS of MAY 2008 EVALUATION Focusing in 2008-2009 on Quality Implementations of Learning Village and TERBIA Emphasis in the next 12 months of the project should be on creating high quality user experiences in Learning Village and TERBIA classes. This is a multi-part problem that can be broken down to optimizing broadcast and digital playback strategies, getting the balance right between the two audio delivery options and their applications for various locations and learner groups; deploying sufficiently robust radio and digital playback equipment, providing adequate staff and teacher training, effectively monitoring, and continuing to seek ways to impact the English language capacity of Sudanese teachers. 46 Our recommendations are in four main areas: 1. Assuring clear, strong sound in all SSIRI learning environments Audio Technologies (Radio Reception, MP3 and the Audio Experience): Further study should be done rapidly on “low-cost” audio devices and compatible and sufficient solar chargers and rechargeable batteries. EDC must find one or maybe two solid systems that it can buy in large quantities and get out to the field with assurance that they will serve the intended learner groups. NB: There is no virtue in spending a few dollars or even a few scores of dollars less on a system if it is not delivering to the listeners’ ears the essential audio. Teachers should be informed about the phantom Miraya station at 95 MHz that for some still unexplained reason has a more powerful output than the nominal 101 MHz that teachers have been instructed to use. Freeplay should also be queried about why their Lifeline receiver is picking up this louder phantom station. Also on the agenda for that conversation should be whether they could produce a unit with higher sound output. We commend EDC on pursuing with Freeplay the possibility of including an MP3 player in a wind-up radio unit. Such a wind-up Freeplay MP3-equipped unit with stronger output level would be a very useful tool. We tentatively suggest that one designed to receive an SD card (for recorded programs) might be less susceptible to breakage than one with USB port for a memory stick. Teachers and head teachers must be informed that all classes (whenever possible) should have two Lifeline radios operating at the same time and that these radios are school property to be safeguarded nightly by some locally effective agreement; and that no teacher should be so possessive that a radio in his/her care cannot be taken by the next Learning Village teacher who needs it. With sequential broadcasts and a single stream at each grade level an entire school would only need two radios. With two streams four radios, and so on. A back-up radio to temporarily replace lost or broken ones should always be available at the County office. MP3 for TERBIA B Groups: TERBIA B groups should be transferred over time to MP3 players. Shortwave is problematic and often unreliable in the afternoon. Afternoon airtime on FM is not going to be easily available for 30 minute programs. Airtime is also very costly (SW costs are currently about $160,000 per year), and airtime must be purchased on recurrent basis. Such costs must be factored comparatively when considering the extra costs of distributing robust digital playback equipment and recorded media. No longer are the materials and labor costs associated with recordable audio media a tremendous cost item, as was the case with audio cassettes. Refreshing of both TERBIA and Learning Village programs, if required, can be done with a rotating stock of digital media. The project can distribute newly recorded SD cards or memory sticks, and as they do so retrieving those originally distributed to be refreshed themselves with new programs. Moving to recorded media will also answer the complaint of TERBIA B facilitators that their afternoon classes often start late due to learners’ late arrival and thus they cannot tune in on time to the radio programs. Study of Economics of IRI by Radio and MP3 (or other codec) for Formal and Informal Settings: One of the long-held verities of Interactive Radio is that using radios is the most cost-effective way to provide quality educational opportunity to large numbers of learners in developing countries. EDC or some other entity should closely examine this assumption given the changing technological landscape. EDC is already committed to making TERBIA Available by MP3 to some of its TERBIA groups. Might Learning Village follow? At the time that the “cost-effectiveness verity” emerged, the personal computer hardly existed. No study of comparative cost-effectiveness of IRI vs. digital playback devices has yet been done. While such a study may be beyond the scope of EDC’s current Agreement, this would be an interesting contribution to the field and should be considered for a future year. There are many reasons from the educational user’s point of view why recorded audio on disc or memory stick or SD card or some other digital device is superior to radio; especially now that power may be generated by winding a crank or by a solar device.13 With such energy sources the argument that spinning a disc drains batteries faster than a radio is not so persuasive. A robust MP3 solution would free teachers from the need to get a class settled and tuned-in by a certain minute on the clock. From the standpoint of pedagogy, having the chance to repeat some part of the program can be very useful. Reception issues would 13 Even audio CDs could play a part in certain applications, such as teacher education. Audio CDs offer some superior pedagogical options to simple MP3 files (either on disc or flash memory) as they are controlled by simple MP3 players in that one can fast forward or review through the contents of a CD audio program. With a simple MP3 player, hitting the advance or review button jumps to the next program. 47 be a thing of the past. Program administrators would not have to compete and cajole for precious air-time that broadcasters are often loath to surrender for their own good reasons, and are sure to charge annual fees to provide. Marketing and P/R: Rename “Low-Cost Technologies”: In the realm of marketing and public relations, we believe that the term “low-cost device” is appropriate to discussions between contractors and funders, but ineffective and misleading between the project and the public. The term is unspecific and does not help educate the public about this important new educational technology. We suggest norming around the terms “Digital Audio Players” or “Digital Audio (Playback) Technologies”. This will be far more informative, more attractive, and educative in itself. This is a more consumer-focused branding strategy that is more likely to interest and excite users than the current one, which may merely suggest to them that no one is spending much money on this intervention. Medium Wave (MW) Radio Stations: As soon as SSIRI needs to broadcast outside the Malakal town area and the range of strong Miraya signals, it should contract with the medium wave (AM) station in Malakal. The station there is highly interested. This is a government-owned station that we understand (and that EDC should verify) reaches far beyond the range of Miraya-FM in Upper Nile State. This would be a good home for Learning Village and for TERBIA A and for TERBIA-B as long as it distributed by radio. While we did not visit the sister MW station in Wau, we would venture the same recommendation for the MW station in Wau should conditions and willingness be similar there. Audio Programs Get the Pauses Right: The key radio scriptwriters should be brought to Southern Sudan to see a number of their programs at work in Southern Sudan schools. This should help them to internalize the time really needed for teachers and pupils to complete their directives. We understand the scripts are evaluated summatively (FE’d) in Kenya but we wonder what this process is like and if it is being done by one or two teachers who are by now so experienced with the programs that they have atypical expert capacities, or atypical students, or take certain process shortcuts. Consider Longer Teacher Led Activity Sections: Writers should also consider injecting longer teacher-led activities of perhaps 1 -2 minutes in length that can, at least occasionally, if not daily, allow teachers to work with a larger number of children in series; to practice communicative language teaching methods in a more intensive way that will be easier to generalize to further English teaching after the program is over; and to establish more relaxed and naturalistic communication in the class. Examine all Songs for Sing-ability: Those songs that are too fast or too unwieldy for young non-English speakers to voice easily and naturally should be rewritten. Some of the songs are very nice as they are, including the Learning Village signature tune. 2. Working in Communities Model Schools: We recommend that SSIRI adopt a model school approach in every county where it works, making it a priority for each local SSIRI staff and their counterpart(s) to create at least one demonstration site where SSIRI is working very well. Depending on the circumstances, coordinators may extend the same level of attention to more than one school. We would leave it to SSIRI to decide what the limit should be on the maximum number of schools that any coordinator should attend to in this next year. Some already have 30 schools. We do not counsel ceasing SSIRI support to any of the schools that wish to continue, but rather judiciously budgeting staff time so that some maintenance support is given to all, but more attention be focused on a single chosen school so at least one site under each staffer’s management becomes a proper model to the community and to other schools of what should be happening in SSIRI classes. Other benefits should flow from this focused commitment. For example, parent teacher association activities to promote community resourcing may logically follow from discussions initiated at meetings about SSIRI and its requirements. We can imagine school groups deciding to create much-needed latrines, as such, SSIRI generated conversations of concern. In any new county that SSIRI enters this year, the SSIRI / MoEST county team should reflect on how it can best tighten focus on an initially small set of schools within a day’s round trip travel and within range of a good audio signal (or equipped with a sufficient MP3 player). Once things are working well in all these schools, staff 48 energy may be similarly extended to more schools. This chosen model school could in some areas represent a de facto “cluster school”. Choosing Project Sites Carefully: The expansion of SSIRI to new counties should be conditional on demonstrated interest by the local community and the County Education Office. This may be expressed in seconding of staff or counterpart designation and/or the willingness to work with SSIRI to create one or more model schools. The local counterpart should be available most of the time for SSIRI activities; however, if this person has many other required duties, such a requirement may be difficult to apply. In existing SSIRI counties where there are already many schools, this requirement should be more firmly observed since more work will be required of the SSIRI staff and counterpart. Keeping “clustering” in mind, selection of model school sites should be done based on 1) the accessibility of that school to the SSIRI staff person and counterpart, and 2) the centrality of that site to a larger group of schools. Continue to Focus on Relations with the States and Counties: Because the counties receive their directives, budgets and resources through the states, the states must be made efficient before putting major resources into bringing the counties to similar levels of efficiency (or possibly in effective tandem with the counties). No county should receive an Outreach Coordinator before an Outreach Advisor is assigned to the state. Counterpart Identification, Incorporation and Training: EDC’s “Responses to Mid-term Evaluation” of May 29 (Annex 6) indicate that they may now post multiple people to the same county offices. Whether this will be socially comfortable at each local MoEST office will need to be carefully thought through in advance on a case-by-case basis. We urge careful high level consultations before placing multiple EDC staffers in a single location. We recommend that in any event, and certainly before a second SSIRI staffer is placed at any site, a recommendation to the project be received from the MoEST office in question; or at least a counterpart person within that office be officially designated to work in parallel with the SSIRI person(s). While sustainability is conditioned by many factors, perhaps at the top of the list is Sudanese ownership of the ideas and (eventually) the ways and means. If multiple people are posted to a single office it should be clear that only one of them – and if the project is successful, maybe only the MoEST employee – will remain after a year or two. It will likely be hard to shift more highly paid SSIRI staffers to lower government pay scales. Making the counterpart personnel already employed at the state and county levels full partners - now - with expectations and responsibilities that are not trivial will be key to the long range viability of this project. Training in school support and SSIRI-related reporting processes, as well as regular travel facilitation for local MoEST staff, should be delivered with this in mind. SSIRI and County Education Office staff lack skills in instructional technology and media-based learning, as well as in follow-up support and monitoring. Even some senior SSIRI staff appear to have little understanding of effective instructional methodologies. As a result, training programs for teachers and trainers may be of marginal quality and impact. We recommend immediate international technical assistance to craft and implement innovative staff training and training of trainers, which are so vital to the success and sustainability of programming. Relationship of the Directorates of Basic Education and AES: EDC should aim to secure the collaboration, understanding and support of MoEST’s Basic Education Directorate (at all levels). It is advisable that the MOU between the MoEST and the State MoEs being developed by EDC include explicit approvals for Learning Village usage in primary schools by both the State Directors of AES and Basic Education. One could understand a Director of Basic Education questioning the way a radio program that spans three subject areas in a half hour fits into the approved timetable scheme. It may not be self-evident to all that a 30-minute Learning Village program that covers three subjects, each, in brief, should be given one of the six full teaching periods each day. Directors may have in mind allotting single subjects to single periods. They may well say, as we were told by one teacher that an Acting Director of Basic Education had advised his school, that Learning Village does not follow the primary English syllabus. Whether or not this is the case, in some areas (especially where SSIRI is not universal within a county or state) there may be another dominant way of teaching and an alternate dominant mode of school inspection of Basic Education classes. There needs to be clear proactive communication from the Basic Education Directorate that the Learning Village/SSIRI approach is approved and that timetables can and should be adjusted to accommodate it. It is also advisable that the MOU between the MoEST and the State MoEs being developed by EDC include explicit approvals for Learning Village usage in primary schools by the State Directors of AES and Basic Education, and by the Directorate of Curriculum, (as we understand that they manage timetable directives and requirements). 49 Promotional and Communication Strategies: Informational and promotional outreach should be done by SSIRI to the MoEST at all levels, and by the MoEST to its offices at all levels, to its head teachers, and to its teachers so that they understand that the SSIRI programs are designed to model activities in short sections that teachers may expand upon after the broadcast. Other popularization approaches maybe tried. For example, we recommend the immediate launch of the SSIRI’s proposed 50-minute education magazine program that the MoEST and Miraya-FM have already agreed to produce under SSIRI funding. We think this outreach strategy has immense potential to martial community support for schools, from latrine building to establishment or invigoration of parent-teacher associations. Partnering with NGOs for pre- and in-service teacher training: There is much more that SSIRI could be doing with Windell Trust and other NGOs. In particular, the potential multi-state partnership with Windell warrants more attention on the part of EDC. They appear to be a natural ally that can extend the benefits of the project to as many as 3200 teachers. As we were told by the Windell coordinator in Malakal, “There are 10 senior tutors at the national level and all could be given a (TERBIA) training by SSIRI.” As Windell informed us that they “do not deal with pedagogy”, their trainees could also be an accessible well-managed group of beneficiaries for SSIRI’s PD101 audio/print course. Creative collaborations should be pursued at the central (Juba) and local levels. An outcome of such collaboration could be teachers becoming more able to cope with LV and TERBIA teaching. 3. Monitoring and Evaluation Systems The evaluation team and project administrators are faced with a dearth of information relevant to the quality of program implementation. Although continuance of the SSIRI project would be predicated, in part, on an EDC in-house study of Learning Village impact in grade 1 (carried out in 2007 - Annex 1), the team’s cursory analysis of the research concluded that it likely suffers from significant instrumentation and procedural faults. Given these limitations, the study should not be accepted as reliable or valid without further scrutiny. Similarly, lack of quality feedback on the deployment of SSIRI-trained teachers and their use of radios must call into question the reliability of EDC reports and databases, and their accuracy in meeting USAID performance indicators. It is imperative that EDC launch intensive, objective and well-designed impact assessments of effective radio transmission and listenership; incidence of radio use in the classroom, contributions of radio programming to learning, and comparisons of radio with other technologies. The team frequently heard complaints from teachers about the pace of radio instruction, audibility issues, and lack of radios. There is little evidence to indicate that such feedback has been incorporated into revised programs or the delivery infrastructure. We recommend that EDC outreach staff reports be regularized, scrutinized for accuracy, reviewed frequently, and responsive follow-up actions rapidly initiated. Similarly, teachers’ radio-use logs should be gathered and analyzed. In short, if the SSIRI program is to be continued, scaled or sustained, EDC and USAID administrators must be informed with rigorous and current assessments and feedback from target groups. There should also be more frequent and broader field monitoring by EDC, SSIRI and USAID. Although security and distance barriers plague travel to the field, the project would benefit from more informed decisions that can only be formulated through hands-on and direct observations. 4. Investing in teacher training, especially in-service training PS101: The core of SSIRI’s PS101 teacher training program was produced during the SBEP effort and has yet to be implemented. Small investments of time and resources would validate the program to provide support to teacher training programs, some of which are now in implementation without content or learning resources support. We note particularly NGOs supporting in-service English language training. We would urge EDC to join with initiatives supported by the Multi-donors Trust Fund (MDTF) to launch the SSIRI teacher training program (PST) during the next project semester. A high priority should be to complete the next 24 programs to support immediate teacher training. It will be a major contribution if SSIRI can provide the inputs needed to get the first year of the CDC’s recently designed in-service program off the ground. EDC may also consider piloting PS101 in some other manner before the MDTF in-service contractor becomes active in implementing the recently completed CDC design. Re-examine Strategies to Enrich IRI Training for New Teachers: SSIRI’s training model for teachers has been a simple three-day one. It is now being extended to five days to try to address teachers’ need for greater 50 confidence in responding on cue in front of their pupils to English directives from the radio. While the extra two days will surely be well spent and valuable, we do not believe it will substantially affect teachers’ basic English competencies. For that purpose, other longer term English language support strategies are needed. SSIRI’s proposed English for Arabic Patterned Teachers course is one. Another is SSIRI’s establishment and teachers’ use of English Resource Corners in the schools (as outlined below). With regard to the five-day IRI training itself, this training needs to be revised to provide teachers with more hands-on examples of working with the programs. The training should also go beyond standard IRI topics, such as care and use of the radio, and the specific training to use LV, and provide broader instructional skills to this largely untrained core of participants. We highly recommend a training module on Communicative Language Teaching principles and practical methods, both as found in the existing programs and as teachers should be applying after the radio￾assisted teaching period. Additional School-based Support for Teachers: If possible, each qualifying school should have an MP3 player. The criteria for such provisioning are for EDC and the MoEST to decide. Such equipment, appropriate programs, and possibly related print materials will enable a tremendous range of options to improve teachers’ English and teaching skills. EDC has already contemplated an MP3 player per teacher enrolled in the upcoming in-service program. We would suggest a more economical approach: identify at least a pair of teachers from a single school to participate together in the in-service course and provide the MP3 player to their school for them to use as a team, together eventually with other teachers at the school. This equipment would then do double duty as 1) an enabler of the in-service course in that school and 2) the cornerstone of the school’s English Resource Corner (see below). The English Resource Corner itself would have the capacity to support teachers’ English skills development and the further school based training of new and replacement teachers taking on SSIRI classes. For the latter purpose we propose School-based SSIRI Training, as described below. School-based SSIRI Training: Conceiving of this program at scale in a few years, one can see that trained SSIRI teachers are sure to go on maternity leave or be transferred. It is not practical to offer the standard three￾day training every time a new person comes into the picture. There should be a School-based SSIRI Training Guide designed to be used by any SSIRI-trained teachers or head teacher in that school, or by any teacher there with obvious competence in managing Learning Village classes, or by the Outreach Coordinator of his/her MoEST counterpart, if available. The school-based team then trains the new teacher(s) during a series of short afternoon or weekend meetings.14 Quality training assistance, follow-up, and coaching can be done as possible by inspectors or project staff. But, the assumption is that there should never be long delays in having a replacement teacher take over the SSIRI duties of a former teacher. The pupils must come first. Model schools can promote School-based Training by example and even act as informal exemplary “cluster centers.” English Resource Corners for Teachers’ Self-paced English Language Development: A range of school￾based strategies should be explored both for the training of new teachers who need to take over SSIRI classes, as above, and for ongoing, self-paced school-based English language skills development for teachers.15 We recommend setting up English Resource Corners in SSIRI-supported schools. It does not take much to create one. Schools need only to designate a space with a sign and a small shelf to indicate it as an official learning space in the school. It can also be temporarily in a community space or even a teacher’s home if the school conditions are not yet conducive. In the English Resource Corner should be one or more copies of the Learning Village teacher’s guides, an MP3 player and digital media with all SSIRI programs, and possibly other English language teaching and teacher development audio and eventually print. This may be a project most appropriately implemented in all SSIRI schools in a later project year, though piloting the concept should start this year. EDUCATION AUDIT FINDINGS 14 The model IELTLM “School-based Training Guide” is worthy of study. It is a complete design for replicating at the school level the project’s 15-hour training course to enable primary teachers to begin using audio in their classes. 15 We also suggest study of an output of USAID’s Improving English Language Teaching and Learning in Mindanao Project, “Real World English Professional Enrichment Course: A Certificate Program for Teachers.” 51 EDC reported results were not supported—Reported fiscal year 2007 education results for the four indicators received from EDC were not supported, and therefore the reliability and validity of the results could not be determined. For operational plan indicator nos. 2, 4, 5, and 916, the implementing partner did not maintain adequate documentation to support the reported results. The available records did not reconcile with the various locations in which the respective activities were conducted. We were also unable to reconcile the partner’s records with those of the service providers that we sampled for indicator nos. 2 and 44 . At the service provider level, the schools’ records were not always complete, and reporting systems were not consistent. Key Objectives of the Technical Review 1. Assess to what extent EDC has implemented recommendations from the May 2008 evaluation and the success of these interventions in addressing issues identified. 2. Determine if these actions have achieved the changes envisioned in the May 2008 evaluation recommendations. If so, to date, what specific changes/outcomes have been achieved? 3. From classroom observations, assess to what extent the SSIRI Learning Village programs seem to be working as a teaching and learning tool in the classroom. And assess to what extent the English Terbia for Beginners programs seem to be an effective approach for teaching English. 4. Assess if the learning technologies including radios, digital devices, and computer labs/VSAT equipment are scalable and sustainable. Major activities of the Consultant 2. Prior to arrival in Southern Sudan, the Consultant is to: a. Review the report of the program audit of the Inspector General’s office and identify the major findings and recommendations for the current assessment b. Review the major documents since May 2008 including weekly, monthly, and quarterly reports. c. Draft a work plan for conducting the evaluation 3. Based on interviews with teachers and Ministry officials at all levels plus visits to schools and listening groups, the Consultant is to conduct an assessment of the major project activities and progress since May 2008 taking special care of the following: a. The Consultant is to assess the extent to which EDC has adjusted its strategy and program activities to take into account the findings and recommendations of both the mid-term evaluation and the education audit. b. The Consultant is to assess the success of EDC’s efforts to (a) evaluate the reception of Miraya Radio in project sites and, in collaboration with Miraya Radio, improve the reception of Miraya Radio and (b) contract with additional radio stations to broadcast SSIRI programs. c. At each location, the Consultant will be given a choice of schools and listening groups to visit. These classes will represent teachers and facilitators who are using the IRI lessons on a regular basis. The Consultant must choose to visit some other schools or groups where teachers and facilitators have been trained, but where there have not been regular follow-up visits by EDC staff. It is also essential that the consultant visit some different sights than those visited in May 2008 in order to assess EDC’s capacity to implement evaluation and audit recommendations in all project areas. 16 Indicator numbering followed that presented in the USAID/Sudan Education PMP section 2.1. For list of indicators please see Exhibit 1. 52 d. The Consultant is to assess the extent to which the classrooms are receiving good quality sound from radio broadcasts for the Learning Village and Terbia classes and that the sound appears to be adequate for the size of the class. e. In classes that are using digital devices, the Consultant is to assess the adequacy of the devices in terms of the quality and loudness of the sound, the use of the devices by the teacher, and the apparent success of recharging the batteries by the teacher. f. The Consultant is to assess the quality of the lessons – apparent preparation and use of teacher's guide by the teacher, the extent to which the children or listeners can hear a good signal, the level of involvement and participation of children or listeners, etc. g. The Consultant is to assess the quality of the EDC monitoring process including the extent to which there is regular, informative feedback from the field sites, i.e., detailed descriptions of ongoing class visits and reporting of basic data. h. The Consultant is to comment on factors, both within and outside of EDC’s control that may impede the implementation of the SSIRI programs. i. The Consultant is to assess the progress of implementing the teacher training program PS101. j. The Consultant is to assess the use of learning technologies in support of the SSIRI activities including the Lifeline radios and digital devices such as MP3 players. At the Maridi TTI, the Consultant is to assess the extent to which the computer center with VSAT is being used effectively by the staff of the TTI for their own development and for benefit of their classes. k. The Consultant is to assess the extent to which the design, scripts and recordings of the new P4 Learning Village series take into account the findings and recommendations of the mid-term evaluation regarding the quality of the IRI programs. 4. Based on the findings and recommendation of the program audit by the IG office, the Consultant is to : a. Review the current documentation required to support the data required for EDC PMP indicators. b. Give special attention to the validity and completeness of the following data sources: i. Teachers/facilitators trained ii. Number of pupils/learners utilizing the SSIRI programs iii. Radios and other equipment/materials distributed iv. Number of education officials trained c. Make recommendations on how learner outcomes can be measured. The proposed itinerary for this technical review is attached as Exhibit 2. Deliverables 1. Research Methodology & Getting to Answers Outline 2. Review of Key Research Questions 3. Work plan for the assessment 4. Detail Discussion of Findings, Conclusions & Recommendation using PowerPoint presentation with USAID/EDC/MoEST to inform and solicit feedback and comments 5. Draft Report submitted to MSI–SUPPORT before submission to USAID 53 6. Final Report submitted based on feedback from USAID, EDC and MSI-SUPPORT Estimated number of work days 3 travel, 2 preparation, 15 on site, 4 report writing = 24 54 EXHIBIT 1: List of Indicators Indicators and Education Partners Ind. # Basic Education FY07 Partners 1 Number of administrators and officials trained AED,EDC 2 Number of learners enrolled in USG supported primary schools or equivalent non-school-based settings EDC,CAII 3 Number of learners enrolled in USG supported secondary schools or equivalent non-school-based settings CARE 4 Number of adult learners enrolled in USG supported schools EDC 5 Number of teachers/educators trained with USG assistance EDC, CAII, AED 6 Number of Parent-Teacher Associations or similar school governance structures supported CAII, 7 Number of classrooms repaired with USG assistance 8 Number of classrooms constructed with USG assistance 9 Number of textbooks and other teaching and learning materials provided with USG assistance EDC, AED 10 Does your program support education systems/policy reform? If yes, please describe the contributions of your program, including progress any mission-level outcome or impact indicators AED, EDC 11 Number of laws, policies, regulations, or guidelines developed or modified to improve equitable access to or the quality of education services. AED 55 EXHIBIT 2: DRAFT PROPOSED TRAVEL SCHEDULE Thursday, Nov. 14 Consultancy start date: Consultant reviews project documentation before departing for Juba, Sudan Saturday, Nov. 15 Arrive in Nairobi. Key documents sent ahead of time for review Sunday, Nov 16 Fly from Nairobi to Juba Meet with key EDC and USAID staff. TPM with MSI-SUPPORT: Review draft of work plan for conducting the evaluation. Draft research questions and getting to answers template Monday, Nov. 17 TPM with MSI-SUPPORT: Finalize research methodology and work plan Meet with USAID, MoEST. Visit Learning Village school and Terbia listening group in Juba Tuesday, Nov. 18 Fly to Kajo Keji. Meet with county officials and visit1-2 Learning Village classes, one secondary school class (Terbia), and one Terbia listening group. Spend night in KK Wednesday, Nov 19 Fly to Yei. Meet with county officials and visit 1-2 Learning Village classes, one secondary school class (Terbia), and one Terbia listening group. Fly to Yambio to spend the night Thursday, Nov. 20 Meet with State officials and visit 1-2 Learning Village class and 1 Terbia listening group. Spend night in Yambio Friday, Nov 21 Fly to Maridi Visit 1-2 Learning Village class, one Terbia listening group.. Fly to Wau to spend night Saturday, Nov. 22 Fly to Kauda Meet with county officials. Visit Learning Village class Spend night in Kauda Sunday, Nov 23 Visit Learning Village and Terbia classes Fly to Wau Monday, Nov 24 Meet state officials Visit Learning Village and Terbia classes Spend night in Wau Tuesday, Nov 25 Fly to Juba Meet with EDC M&E Specialist re. PMP data Wednesday, Nov 26 Report writing Thursday, Nov 27 Review findings, conclusions and recommendations with MSI-SUPPORT Thursday, Nov. 28 Out-brief with USAID using PowerPoint Presentation Continue drafting assessment report Friday, Nov 28 Submit to MSI-SUPPORT draft report and depart Juba Friday, Dec 1 MSI-SUPPORT submits draft report to USAID for comments Monday, Dec 8 MSI-SUPPORT receives comments from USAID. MSI-SUPPORT sends comments to EDC for clarification and addition input as appropriate Wednesday, Dec 10 MSI-SUPPORT receives comments from EDC after reviewing USAID’s comments. MSI-SUPPORT sends USAID and EDC comments, questions, etc. to consultant to incorporate in the final report. Friday, Dec 19 MSI-SUPPORT submits final report to USAID 56 ANNEX 2: WORK PLAN - SSIRI EVALUATION REVIEW ACTIVITY BACKGROUND Stuart Leigh (Consultant) was MSI’s team leader for the midterm evaluation of the SSIRI project conducted in May of 2008. As such he is already familiar with the context and nature of the SSIRI project. He has been asked to look at the project again to assess, among other things, the degree to which the issues and recommendations cited in the midterm report have been addressed by the project team. As such, this evaluation is in the nature of a review. As in May, the process and product assessment will actively engage stakeholders USAID-Sudan, the Southern Sudan Ministry of Education, Science and Technology (MoEST) and the contractor, Educational Development Center (EDC) in an analysis of the processes and outcomes of the SSIRI activity, largely focusing on the period from June 1, 2008 to the present. The participants in the review will travel to project sites throughout Southern Sudan. Members have agreed to employ an open and consultative format; however, the ultimate responsibility for gathering and interpreting information rests with the MSI evaluator. PURPOSE The principle deliverables of the consultancy will be an oral debriefing and a written report. The evaluation will provide answers to a set of questions posed by USAID Sudan: 1. Assess to what extent EDC has implemented recommendations from the May 2008 evaluation and the success of these interventions in addressing issues identified. 2. Determine if these actions have achieved the changes envisioned in the May 2008 evaluation recommendations. If so, to date, what specific changes/or outcomes have been achieved? 3. From classroom observations, assess to what extent the SSIRI Learning Village programs seem to be working as a teaching and learning tool in the classroom. And assess to what extent the English Terbia for Beginners programs seem to be an effective approach for teaching English. 4. Assess if the learning technologies including radios, digital devices, and computer labs/VSAT equipment are scalable and sustainable. METHODOLOGY The evaluation process will incorporate the following methodologies: 1. Desk review of SSIRI project documents produced since the end of the previous evaluation, including work plans, bi-monthly and quarterly reports of the COP, weekly monitoring reports by field staff, other SSIRI office monitoring documents including registration forms for trainees, internal evaluations, consultant reports and radio program scripts and audio recordings; 57 2. Debriefings with EDC project management and staff and USAID Education team members charged with SSIRI project oversight; 3. Discussions with MoEST officials, including: those GoSS employees responsible for integrating SSIRI assets into educational operations; State Directors as available, school headmasters, and teachers; 4. Field visits to radio broadcasting organizations and interviews with personnel; 5. Site visits to at least three States in Southern Sudan (Central Equatoria, Western Equatoria, Western Bahr al Ghazal), one of the Three Areas (Southern Kordofan), with visits to project sites in the following municipalities: Juba, Kajo Keji, Yei, Yambio, Maridi, Kauda, and Wau); 6. Site visits to a teacher training institute (TTI) to assess the sustainability and scalability of learning technologies and their consequent impact on pre-service and in-service teacher professional development; 7. Classroom observations of the impact of radio instruction on students, and follow on discussions with students and teachers; 8. Observations of TERBIA learning groups; 9. Examination and review of sample radio and video instructional units produced since June 1, 2008 together with any integrated print materials and descriptive documents related to the planning of the instructional programs. OUTCOMES and DELIVERABLES The review is expected to document performance of EDC since the last evaluation in adjusting the processes of the SSIRI activity to address issues identified in that evaluation. This will include the consideration of primary and secondary research (with special focus on the findings of the consultant) that describes progress since the last evaluation. The Final Report will provide documentation, findings, conclusions and recommendations specific to the: 1) performance of the EDC team in remediating identified problems in the SSIRI project; 2) further corrections or revisions that would likely make further contribution to the project attaining its objectives. The activities of the review are to include the following: a. Assessing the extent to which EDC has adjusted its strategy and program activities to take into account the findings and recommendations of both the mid￾term evaluation and the education audit. 58 b. Assessing the success of EDC’s efforts to (a) evaluate the reception of Miraya Radio in project sites and, in collaboration with Miraya Radio, improve the reception of Miraya Radio and (b) contract with additional radio stations to broadcast SSIRI programs. c. At each location, choosing schools and listening groups to visit. These classes will represent teachers and facilitators who are using the IRI lessons on a regular basis. The Consultant will also visit some other schools or groups where teachers and facilitators have been trained, but where there have not been regular follow￾up visits by EDC staff. The Consultant will also visit some different sights than those visited in May 2008 in order to assess EDC’s capacity to implement evaluation and audit recommendations in all project areas. d. Assessing the extent to which the classrooms are receiving good quality sound from radio broadcasts for the Learning Village and Terbia classes and that the sound appears to be adequate for the size of the class. e. In classes that are using digital devices, the Consultant will assess the adequacy of the devices in terms of the quality and loudness of the sound, the use of the devices by the teacher, and the apparent success of recharging the batteries by the teacher. f. Assessing the quality of the lessons – apparent preparation and use of teacher's guide by the teacher, the extent to which the children or listeners can hear a good signal, the level of involvement and participation of children or listeners, etc. g. Assessing the quality of the EDC monitoring process including the extent to which there is regular, informative feedback from the field sites, i.e., detailed descriptions of ongoing class visits and reporting of basic data. h. Commenting on factors, both within and outside of EDC’s control that may impede the implementation of the SSIRI programs. i. Assessing the progress of implementing the teacher training program PS101. j. Assessing the use of learning technologies in support of the SSIRI activities including the Lifeline radios and digital devices such as MP3 players. At the Maridi TTI the Consultant will assess the extent to which the computer center with VSAT is being used effectively by the staff of the TTI for their own development and for benefit of their classes. k. Assessing the extent to which the design, scripts and recordings of the new P4 Learning Village series take into account the findings and recommendations of the mid-term evaluation regarding the quality of the IRI programs. 59 Based on the findings and recommendation of the program audit by the IG office, the Consultant will: a. Review the current documentation required to support the data required for EDC PMP indicators. b. Give special attention to the validity and completeness of the following data sources: i. Teachers/facilitators trained ii. Number of pupils/learners utilizing the SSIRI programs iii. Radios and other equipment/materials distributed iv. Number of education officials trained c. Make recommendations on how learner outcomes can be measured. The Final Report will be submitted in a format consistent with MSI’s agreed-upon format with USAID Sudan. A draft table of contents is attached as Exhibit 1. Exhibit 1: ANNOTATED REPORT OUTLINE Table of Figures and/or Tables Project Summary Acronyms Executive Summary I. Introduction II. Background (A lengthy “Statement of the Problem and USAID’s Response” was submitted by the Consultant just over five months ago as part of the previous SSIRI evaluation. MSI and USAID have agreed that there is no need to re-author a background statement covering the same ground and that it eill be acceptable to submit for this section a much shortened edited version of the earlier background statement.) III. Purpose of the Evaluation IV. Research Design and Methodology a. Methodology b. Data Types V. Findings 60 a. Process Findings b. Outcome and results findings VI. Recommendations VII. Conclusions VIII. Annexes 61 ANNEX 3: INTRODUCTION AND BACKGROUND STATEMENT TO THE MID-TERM SSIRI EVALUATION I. INTRODUCTION Southern Sudan is one of the lowest-ranked nations on Earth by any index of human development. In the field of education, the area could arguably be considered a candidate as the poorest. The UNDP classifies Southern Sudan in the very lowest decile of: literacy, school access, gross enrollment and primary school completion. Its burgeoning population is composed of a myriad of language and ethnic groups who have little in common. The current Government of South Sudan language policy is English instruction for all grades over P3 but the great majority of teachers use local Arabic in instruction, especially in the more northerly areas. Additionally, many returnee teachers from the north are “Arabic-patterned” teachers. Five decades of civil war, inter-tribal conflict, and internal and external displacement have sapped coherence from the educational system; and ongoing warfare in the three “Transition Areas” that border Khartoum-controlled Sudan remains a constant. As this mid-term evaluation began—in early May, 2008—hostilities resumed anew in the Abyei region of the “Three Areas”. Reports are that many have been killed and injured and it is estimated that as many as 90,000 persons have fled the region as refugees. As a result of this and other rising tensions, security restrictions are increasingly curtailing operations throughout the whole of Southern Sudan. Unfortunately, the current security situation is more the norm than an exception. Since independence from the UK in the 1956, and under the regime of the Khartoum Sudanese government, Southern Sudan has borne the brunt of benign and overt neglect, frequent and brutal military assaults and cultural domination from the north. Major upheavals in the 1960s and again in the 1980s led to the creation of the Sudan Peoples Liberation Army (SPLA) and other military factions and outright civil war in the south. By 2000, the Khartoum Government had lost virtually all control of the countryside and maintained tentative presence only in major “garrison towns.” That period saw an unprecedented exodus of refugees to neighboring countries and the largest internally displaced population in the world. This resulted in the creation of a quasi-Southern Sudanese government by the Sudan People’s Liberation Movement in exile in Kenya. There are various estimates of the number of people dislocated but, of the 8-11 million inhabitants living in Southern Sudan, at least 95 percent were adversely affected or dislocated during the violence. Transhumance of this magnitude has transformed the demographic ecology, and with the continuous relocation of refugees, continues to weaken Southern Sudan’s social fabric, more than three years after the Comprehensive Peace Agreement (CPA) was signed. Throughout this period of upheaval, the United States Government has maintained a neutral position between the Khartoum Government and the people of Southern Sudan. The overriding concern has been for the physical and social welfare of Sudan’s people, irrespective of political alliance or affiliation. The concern has been largely reflected in effecting peace. Even today, some eight years after launching support programs, the USAID/Sudan, Office of Southern Sudan Programs Mission’s principal objective is to foster and preserve peace between the newly formed 62 Government of Southern Sudan (GoSS) and the Khartoum Government. Immediate goals are to support the CPA as a measure for securing peace; continuation of direct services to the Sudanese people; and with respect to education, strengthening the GoSS Ministry of Education, Science and Technology (MoEST) to sustain innovative educational programs. Because of ongoing support for the Southern Sudanese population and various iterations of their government, relations between the United States and the GoSS remain good and productive. Faced with some of the most formidable, unique and often dangerous challenges, USAID has been partner to the establishment and maintenance of GoSS and an especially critical contributor to the emergence and effectiveness of MoEST. Nevertheless, many deeply-rooted and systemic problems remain. Since the signing of the CPA, thousands of refugees returned to settle in former villages with little more than basic household items. They return mostly from refugee and displaced persons camps where the most essential human services were minimal. Depending on their country of exile, many Sudanese were treated as pariahs or ignored and left to fend for their own. The majority of returnees have lived for the better part of a generation dependent on donor subsistence, and many have developed a culture of dependence that borders on addiction to handouts. They join those who remained in Sudan, or were similarly encamped near Khartoum, and share similar circumstances: food insecurity; housing shortages; no means of livelihood; broken families; disrupted traditional village support systems; values and attitudes cast in a crucible of diminished hopes, morale and empowerment; and perhaps most critical of all, inadequate life, work and coping skills to prosper in a modern environment. Despite, these shortcomings, Sudanese are a resilient people and share the drive and motivation that mobilizes so many of the world’s poor: hope for a better life for their children. In this environment, any positive intervention for child welfare and human dignity, irrespective of its impact or scale, can have an immeasurable impact on social advancement. To quote the USAID Education Team Leader, “even the mere fact that children are attending school, regardless of the quality of education, is a quantum step in human progress for the people of Southern Sudan.”17 It is with this modest, but potentially transformative goal in mind that USAID has supported the Southern Sudan Interactive Radio Instruction (SSIRI) project since 2004. This report describes a mid-term, formative evaluation of the progress of that operation. II. BACKGROUND – DEVELOPMENT OF THE PROBLEM & USAID’S RESPONSE Southern Sudan USAID operations in Southern Sudan are tempered by its tenuous status as a potential member in the family of nation states. Born of the Comprehensive Peace Agreement of 2005, the area has sustained social, economic and military upheavals for over two generations. USAID must operate on the premise that support is not necessarily extended to the Government of Southern Sudan, but to the people of Sudan. The agency’s priorities are to preserve peace, engage citizens in support of the CPA and to advance human welfare in the region. 17 Inez Andrews. May 2008 63 Although the present Government of Southern Sudan was largely crafted in exile in Kenya, the period immediately following execution of the CPA has seen a massive and most impressive nation building process. The government may be in a nascent stage of development, but there has been marked and substantive progress in establishing order, basic human services and rule of law that far exceeds many African states of much longer and assured existence. Primary education in Southern Sudan According to an April 2008 survey conducted by the MoEST, there are estimated to be between 1.7 and 2.1 million students formally enrolled in primary school, grades 1-8. They are served by approximately 3,100 schools and taught by 24,000-27,000 teachers. It is variously estimated that 90 percent of teachers are competent in Sudanese Arabic, while about 20-30 percent have rudimentary English skills. In terms of geography, English and local languages prevail as the core language of instruction only in states near the Ugandan and Kenyan border, with Arabic dominating classrooms in the Three Areas and the northern and central states of Southern Sudan. Primary education is supported and delivered principally by the Government of Southern Sudan (there are few private or NGO-run schools) which receives funding through a joint revenue sharing agreement with the Khartoum Government. This accord was memorialized in the Comprehensive Peace Agreement of 2005. Despite rapid growth of the school age population and pressures on all educational resources, the national budget available for primary education has been in a state of decline since the CPA went into effect. In 2006, the MoEST had a budget US$136 million; however that amount was reduced to US$111 million in 2007; while in the 2008 school year, the budget has been further reduced to US$96 million. In effect the operating budget has declined by about 30 percent, while enrollment, because of increased stability and refugee returns has likely doubled. The current budget provides for about US$45 per student per year (as compared with about US$9,500 in the US state of Missouri). There are no accurate data, but it is reckoned that about 95 percent of the budget is spent on salaries, with the remainder allocated to administrative costs. Teacher salaries, while also difficult to measure accurately, ostensibly range from a base of US$85 per month for beginning teachers up to US$400 per month for senior teachers. The MoEST also indicated that some 6,100 teachers involved in after-school adult learning programs, such as the SSIRI TERBIA Activity, are to be paid an additional US$235 per month. Were this salary scale a reality, this would place Southern Sudanese teachers amongst the highest paid in sub-Saharan Africa. However, based on conversations with teachers, their pay often lags for up to several months, unwarranted deductions are made by administrators, and the official salary scale is rarely honored. The primary education system is overseen by a decentralized administration in which the MoEST and the Ministry of Finance supply funds to the 10 State Governments. Funds are, in turn, allocated to County Education Directors who distribute salaries directly to teachers. State and County governments are involved with hiring teachers and monitoring and inspecting schools. Their involvement with schools however is minimal, as they have no means of transport and even less training as instructional administrators. Although personnel in these 64 posts are new and without many resources, most are former teachers and have been involved with a smattering of professional development programs. Since funds are dedicated almost exclusively to salaries, there is no budget for capital investment or for the purchase of texts and learning materials. As a result, schools have not benefited from rehabilitation or expansion in many years. Classes, principally for lower grades that often enroll over 200 students, are often taught under trees or in open, lean-to type structures, where they are exposed to heat, rain and numerous distractions. As students abandon schools in early years, higher grades, characterized with 20-30 students, benefit from classrooms. In all instances, however, there is a dearth of desks and seats, with pupils relying on stones, fallen logs, or just dirt. Facilities have no electricity, running water, or toilets, even though school enrollments reach near 2,000 pupils in some urban centers. Few schools (only one in 15 observed) have feeding programs, so children are expected to participate in a 6-period learning program from 8:30 AM until 2:05 PM without food. Most teachers use lessons drawn from small workbooks that guide the curricula. These are printed in English, so many Arabic-speaking teachers experience difficulty in their use. Each class has a chalkboard, on which a teacher typically writes a few sentences before retiring to a seat outside the classroom, where they await individual pupils to show them their work. Other than these two learning media (teacher workbook and chalkboard) there are no learning resources to assist learning. As with many developing nations, teacher absenteeism is high, schools are often closed for unofficial reasons. In April 2008, for example, schools were interrupted for nearly a month so that teachers could help conduct the first national census. With malaria and parasititis endemic to Southern Sudan, teachers and students often miss prolonged periods of class with illness. There is little tradition of parental involvement in Southern Sudan schools and in many schools children are required to pay a fee to attend. These funds are ostensibly used to ramp up pay for “volunteer” teachers, who often comprise up to a third of the teacher force but receive lesser pay than “government” teachers. Volunteer teachers represent only one group of poorly trained teachers. It is estimated that 90 percent of all teachers lack fundamental pedagogical skills. The majority come to schools with little more than primary schooling, and perhaps less than 10 percent have had post-secondary training in either of the two teacher training institute (which together graduate some 150 teachers annually). Although there are a number of well-trained teachers (mostly trained in normal schools and universities in Uganda and Kenya) who have applied for appointments, State education budget shortfalls have severely limited hiring new qualified teachers. Central Equatoria State, for example, the fastest growing urban district in Southern Sudan, has not hired a new teacher in three years. There are few reliable estimates of children who do not enroll; however, a CCRI survey in Jonglei concluded that “only 30,000 out of an estimated 100,000 school age children are enrolled in school in the eastern State.” The survey also found that adult literacy is below one percent and the worst affected are women who have a literacy level of below 0.05%. “There are an estimated 65 500 teachers teaching in 80 makeshift schools (under trees and Tukuls). About 90% of those teachers are untrained volunteers.” As to primary school dropout rates, simple arithmetical calculations support an estimate that only about one in four pupils reach grade 8. Without a substantive increase in the education budget to meet current needs and prepare for a burgeoning school-age population, the quality of education will inexorably continue to decline, school abandonment will accelerate, and more children will choose not to enroll. Systemic shortcomings notwithstanding, the Southern Sudanese primary education is in an inchoate state of development. Schools, however limited in space, resources and amenities are in operation. Teachers, despite their limited capacities and high absenteeism do hold classes, and receive sufficient pay (however irregularly) to remain in the profession. GoSS, State and County education administrators can be found at work most times of the day, and there is some semblance of a structured learning program in schools. There is substantial if guarded cause for hope. Problem Stated Southern Sudan poses a number of unique challenges to conventional human services programming. After nearly 40 years of intermittent and savage conflict, most of the area has achieved a degree of stability that fosters the return of refugees, many of whom have been away from Sudan for a generation. They have arrived with internally displaced persons from camps near Khartoum to deserted villages, fallow fields, a near-devastated physical infrastructure, limited or no human services, and deeply-ingrained fears for personal security. Families and traditional social units have been dispersed throughout the world and most have survived the last 15 years as dependents of food programs and a smattering of educational services offered by host governments. The cycle of transhumance and expulsion from one nation to the other, each with internal problems spawning internecine savagery, has left Sudanese with indelible fears. Few of those who stayed or fled have had exposure to life skills or values needed for self sustenance; as a consequence, Southern Sudanese are only just now beginning to develop an ethos of work, self reliance, or trust in higher authority. The problem is further complicated by living in an area that has no status as a nation-state. The Government of Southern Sudan does exist for certain, but there are no assurances that it will endure, perhaps for no longer than the 2011 plebiscite. Where formerly there were defined ethnic groupings, today there are a mosaic of peoples and languages. After long association with the Khartoum Government, most older Southern Sudanese speak Arabic and very limited English. But some youth and refugees converse in English and GoSS has dictated that English is to be used as the language of instruction. A large percentage of rural Sudanese speak neither Arabic nor English and must rely one of some 54 local languages. Illiteracy rates are among the highest of any region on Earth. Limited communications and travel pose formidable obstacles to any social or economic transactions. And the situation 66 continues to deteriorate. As security is perceived to improve, more refugees arrive, overwhelming the fabric of human services. Health and educational services are taxed beyond their already limited capacity. While families are anxious to educate their children they have no resources to devote to anything but survival, and although the GoSS invests some six percent of the annual budget in the education sector this fails to meet even minimal programming standards. Humanitarian aid for Southern Sudan faces the contradiction of extending relief on the one hand and trying to build internal capacities on the other. How much energy and resources are to be spent on each fuel a continuous debate. The dilemma is sharpened by a flagging interest in the international donor community, which has reduced earlier commitments of some US$150 million to education to less than US$50 million. And trumping all solutions is the constant threat of a renewal of conflict, which has already been reignited in the Three Areas, triggering another round of dislocation, suffering and death. History of USAID Intervention in Southern Sudan USAID-Sudan launched support for the people of Southern Sudan with the Sudan Basic Education Program in 2002. The purpose of the program was to support the development of an education system where one none existed. The Cooperative Agreement was managed by CARE￾USA as the prime contractor with American Institutes of Research and the University Of Massachusetts School of International Education. The program began as support at the local and county level and later developed into support of a newly-formed Secretariat of Education for Southern Sudan, to construct schools, and to establish a minimal education infrastructure. The goal of this project was to increase access and quality educational services directly to the people of Southern Sudan in the promotion of peace. As Southern Sudan became a more consolidated and formal political entity and peace appeared on the horizon, USAID chose to expand programming into the south through radio transmission of English language basic education and civics programming for adult and out-of-school audiences, as well as for in-school audiences of primary school children and their teachers. In 2004 USAID Sudan selected a media-based approach to delivering vital basic education. The USAID Mission chose to supplement a global education facility—DOT-EDU—as the delivery mechanism and chose EDC the executing contractor. EDC brought substantial programming experience in radio and distance education to the USAID education portfolio. The project was driven by USAID Sudan’s strategic objectives to deliver direct relief services to Southern Sudanese people and to provide “Improved Equitable Access to Quality Education” (SO 6). Absent a formal government in the area, and with ongoing civil and military conflict, it was decided that EDC would collaborate with a nascent ministry of education based in Nairobi. The goal early on was to develop and launch radio programming directly to schools, and to help set the stage to establish and support a more formal ministry when Southern Sudan was to gain a degree of autonomy and stability. During the initial days of the project, EDC established a media production center in Nairobi, Kenya to design and broadcast two major radio initiatives: the Learning Village, which broadcasts lessons set in a virtual Sudanese village, and TERBIA, which broadcasts radio 67 programs aimed at improving English and civic education for adults and out-of-school youth. USAID/EDC programming has rapidly devolved operations to Southern Sudan. With the signing of the CPA in 2005 between the SPLM and the Government of Sudan (GoSS), there was a basis for ongoing peace and development in the country. USAID noted that there are high expectations that peace would bring programs to improve education, health, water, livelihood opportunities and other basic services—the “Peace Dividend.” In reality, however, the Government of South Sudan (GoSS) has lacked the capacity to respond to these needs. Logic held that if the GOSS failed to meet rising demands and expectations, the risk of conflict would increase. In this volatile situation, USAID’s strategic objectives were modified a number of times. Under the Fragile State Strategy USAID’s strategic objectives relevant to SSIRI became “Averting and Resolving Conflict” (SO 9) and “Promoting Stability, Recovery and Democratic Reform” (SO 10). Currently USAID and EDC-SSIRI are focused on “Investing in People”, “Supporting the Comprehensive Peace Agreement (CPA)” and bringing peace dividends to the citizens of Southern Sudan through improved educational opportunities. USAID now seeks to assist the Ministry of Education, Science and Technology to develop policy, programs and procedures in all areas of basic education. USAID chose to implement direct educational services to Southern Sudanese schools and children through the medium of radio, which provides an opportunity to impact on the largest number of beneficiaries, rapidly and with the least cost. USAID selected EDC to create radio programming and complementary instructional support services, implement the programs and explore alternative learning technologies. The SSIRI project is major element of USAID education investments in Southern Sudan. It includes supporting the use of educational technologies through orientation and familiarization activities, training, developing improved policy structures, and creating and implementing appropriate applications, especially in teacher training. SSIRI Program Description The SSIRI operation is based on the premise that radio is the most efficient and effective means for extending good education to populations that are dispersed across wide and remote areas, and in need of qualified teachers. Instructional programs are intended to provide knowledge and skills needed for individual and national development, as well as support USAID’s goal of fostering peace and acceptance of the CPA. EDC implements five broad and interrelated programs: 1. Provision of direct instruction to primary school teachers and children via radio: the Learning Villages series; 2. English language and civic education to out-of-school youth and adults by radio: the TERBIA program; 3. Professional Studies for Teachers through face to face training, outreach and radio support; 4. Institutional strengthening to the GoSS MoEST (especially teacher training institutes) and its executing entities: State, County and Payam educational agencies; 68 5. A final and enabling strategy is to explore, test, pilot and validate other alternate and low cost technologies that might augment or replace radio where that option is not available or of quality. The Learning Village comprises interactive radio programming for grades 1-4, and is based on the Southern Sudanese primary school syllabus. Lessons (in English) are designed to support classroom instruction in English, math, local language literacy and life skills. The 100 lessons for each grade are designed and sequenced to be broadcast daily, for 20 weeks a year, in half hour segments. Actual programs are 28 minutes long. Lessons are designed to be engaging, interactive and incorporate pupil-centered instructional practices. They are designed and created by Southern Sudanese scriptwriters and producers whose target audiences include the nearly 80 percent of teachers with limited English skills. Not only are the lessons intended to structure and support English learning by students, they also help Arabic-only teachers acquire English skills themselves, in their fulfillment of requirements to implement the national curriculum in English, which is the policy of GoSS MoEST. Learning Village programs are extended to the classroom by training selected teachers in the use and care of radios, providing them with a comprehensive teacher’s guide, and offering follow-up support by outreach staff to ensure the radios are in use. During the last year, project administrators have taken steps to integrate SSIRI training and support into the routine of County Education Officers, who are expected to coordinate the program after the life of project, as well as support IRI inspection staff at the MoEST level Teaching English through Radio-based Instruction for All (TERBIA) is a radio-based program produced by EDC-SSIRI to deliver English, civics, life-skills and nation-building education to out-of-school youth and adults. Although the program is framed as an English instruction initiative, an important collateral objective is to foster understanding of the CPA. Topics of the TERBIA programs change regularly and include themes such as disarmament, demobilization, landmine awareness, women’s rights and other programming in support of civil society and good citizenship. TERBIA targets audiences with four levels of English skills through the broadcast of 60 lessons per level (low beginner, high beginner, intermediate, and advanced), for a total of 240 half hour programs. TERBIA B (for beginners) Levels 1 and 2 target Sudanese with little or no English, and are designed for listeners who meet in groups with teacher/facilitators. TERBIA B1 began broadcasting in late 2007 and TERBIA B2, was launched in May 2008. TERBIA-A (advanced) is intended for those who already have some competencies in English. A focus of the advanced broadcasts is to expand conversational English as well as stimulate learning and dialogue in civic education, independently or in small groups. The program began broadcasting in early 2007. TERBIA is transmitted in some FM reception areas and nationally by shortwave in the late afternoon (4 PM), or by shortwave in the morning (9:30 AM). While it is aimed at the general listening public, lessons are also intended to supplement instruction for adults and out-of-school 69 youth enrolled in programs of the Alternative Education System (AES), such as the Accelerated Learning Program (ALP) where students can receive eight years of primary education in four years. TERBIA is used as an adjunct to this curriculum in an extension period of the same learner group meetings. SSIRI has trained teachers to organize and lead formal TERBIA classes, often in primary schools after their regular teaching hours. The MoEST has signaled intent to provide supplemental incentive pay to such teachers, who would weave TERBIA Broadcasts into the literacy and accelerated learning programs designed by MoEST. The need to address English learning among those with modest skills led to a decision to create TERBIA-Intermediate. These programs are under development and broadcasting is projected to begin in late 2008. Professional Studies for Teachers (PST) is designed as a radio-based, distance learning course to improve instructional practices of Southern Sudanese teachers. Since the preponderance of primary school teachers lack even the most basic training in subject skills, pedagogy and instructional methods, PST is also intended to help them earn certification. The evaluation team notes later in this report that there are approximately 25,000 teachers in Southern Sudan, yet only 19,000 actual teachers are listed on the payroll. The remainder (and many others on the payroll) are local volunteers likely lacking even the most fundamental teaching skills. The PST course currently consists of 12 programs/modules that focus on effective classroom management, soon to be supplemented by additional audio programs/modules to further support in-service teacher training. According to EDC’s Cooperative Agreement implementation plan, the PST was to launch by 2007 and be integrated into broader and more extensive in-service professional development initiatives supported by MoEST and the Multi-Donors Trust Fund (MDTF). While a four-year curriculum has been completed, implementation is currently stalled. Institutional Strengthening of the MoEST and local education offices is an expectation implicit in the recent Modifications (No. 4, No. 9) of EDC’s Cooperative Agreement with USAID. This is to take several forms: installation of an interactive radio inspectorate and capacity-building in the central GoSS-MoEST so that SSIRI services will be sustained after the life of the project. Since the GoSS MoEST has devolved most school operations to the States, and they in turn to Counties, EDC’s development strategy is to upgrade the quality of educational management that is nearest to stakeholders and clients: schools and students. Thus EDC intends to focus on developing local capacities at the inspectorates of the County Education Offices and the Payams (local governments). Strengthening of MoEST institutions has also included building the capacities of the three pre￾service teacher training institutes (TTIs). In this instance, EDC has chosen to strengthen the TTIs as vehicles for accomplishing the goal of implementing alternative and low cost learning technologies. USAID has had a longstanding commitment to creating computer centers to develop innovative teaching resources through distance, e-based and video instruction. Alternative learning technologies. USAID and EDC have recognized the limitations of radio technology in reaching Southern Sudanese student and teacher populations. Shortwave signals fluctuate widely over distance and time of day, making them difficult to hear. FM broadcasts are 70 of more uniform quality but the range of effective broadcasts from each transmitter is greatly limited. Similarly the timing of radio broadcasts presents a problem. TERBIA classes, for instance, tend to form and operate for adults after regular school hours and outside broadcast schedules. A fixed schedule for all listeners is simply not possible. To meet the needs of current and potential teachers, EDC and MoEST have opted to embrace innovations and broader technologies than that afforded by radio. In addition to investments in IT at teacher training institutes, EDC has proposed to expand into other audio options, such as MP3 players, which offer portable and unscheduled training (via a USB flash-memory stick, SD card, etc.). This will allow teachers to access programs when they want, as opposed to depending on radio broadcasts. EDC has also noted that large class sizes militate against the effectiveness of single Freeplay, wind-up (crank-style) radios, which have a limited range of audibility. The current Cooperative Agreement calls for the piloting and validation of MP3 players, plus small speakers for use in classes of appropriate (small) size. Similarly, CD/FM/USB-MP3 players “boom boxes” are to be tested for viability and utility in large classes. EDC’s work plan also called for the installation of VSAT facilities in a number of TTIs by mid-2008, the testing and validation of alternate “low￾cost” learning devices by early 2008, and expansion of the chosen device(s) by late 2008. 71 ANNEX 4: PEOPLE INTERVIEWED OR MET DURING 11/08 SSIRI EVALUATION REVIEW Date Person Position NAIROBI 1 11/16 Janet Mola M&E Specialist, SSIRI (no longer on staff as of November 15 having left for doctoral studies in Canada.) JUBA 2 11/17 Yunas Barsaba Amin P1 teacher, Juba Model P/S 3 Chaplain Soso Lupai Head Teacher, Juba Model P/S 4 Benji James Taban P2 teacher, Juba Model P/S 5 Clementina Kidew Arkangelo P2 teacher, Sadaka P/S 6 Kuol Atem Director, AES Directorate, MoEST 7 Huub Gales Director, Miraya FM 8 Brando Keabilwe Producer, Miraya FM 9 Jane Apollo & Ruth Zabedayo Terbia B1 Facilitators, Kator Bari, Juba MARIDI 10 11/18 Shaban Lado P1 Teacher, Haddow P/S 11 Mr. Boniface Nathana P1 teacher, Haddow P/S 12 Woodman Jawo Morris P2 Teacher, Mabrindi P/S 13 Beneth Surur IT Officer, SSIRI, Maridi 14 Athanas Mwamba IT Coordinator, SSIRI, Juba 15 Duret Ezbou John Instructor, Maridi TTI 16 Kiliyona Samba Yorama Instructor, Maridi TTI 17 Wanjala Joseph English Instructor, Maridi TTI 18 Kodani Bosco Roman Instructor, Maridi TTI 19 Bort Odongi Severino Instructor, Maridi TTI 20 Kanullo Dominic Instructor, Maridi TTI 21 Kenji Millimon Instructor, Maridi TTI 22 Christopher Loboyong Instructor, Maridi TTI 23 Oka Friday Head CDC, Maridi 24 Isaya Wani Principal. Maridi TTI 25 Fulgentio Nono Mark Instructor, Maridi TTI 26 Wanjala Joseph Terbia B Facilitator, Maridi Girls P/S KAJO KEJI 27 11/19 Poni Joice, P1 Teacher, Leikor PS 28 Kirya Julius Modi Head Teacher, Leikor PS 29 Kgji Jang Justin P3 Teacher, Wudu P/S 30 Wani Henry Lubajo Supervisor AES, Kangapo II Payam, Kajoke County 31 Kisakye Grace Media Officer, Grace FM 32 Julie Ashmude Studio Supervisor, Grace FM 33 Victor Tombe Outreach Advisor, SSIRI 34 Benson Sokiri Lokuli Headmaster, Kajo Keji Secondary School 35 Justin Ladu Erasto Terbia A (English) Teacher, Kajo Keji Secondary School 36 Moses Koduroi Keni Inspector of Secondary Schools, KK 37 Monojasuk Simaya D/Director AES, KK 38 Benjamin Wani A/Director Admin. KK 39 Bojo Yelemana Henry A/D/D AES KK 40 Elisama Poe? Elisa D/D fpr Training, KK 41 Edward Maku Moda D/D for Religious Studies, KK 42 Abore George Dinyato Payam Supervisor AES/SSIRI 43 Sebi ? Dukudun ? Payam Supervisor AES/SSIRI 72 Date Person Position 44 Elizai Modilubazo D/D for Science Education, KK 45 Evans Paul Wani Book keeper, KK 46 Jokudo Alice Typist Clerk, KK 47 Rombe Francis Modi Terbia B Facilitator, Mundari Pentecostal Church ALP, KK YEI 48 11/20 Alex William Loburi General Manager Senior Editor 49 Mutto Robert Studio Manager 50 Evelyn Poni Ayobo P1 teacher at Kinji P/S 51 John Juma Mike P2 teacher, Ronyi P/S 52 Regina Alfred (?) or substitute ? P3 Teacher, Ronyi P/S 53 Kepo James Principal, Yei Teacher Training College 54 Amule Felix County Education Director, Yei 55 John Gama Terbia B Teacher 56 Newton Crispo Wilson Outreach Coordinator, SSIRI, Yei 57 Ale Peter Michael Outreach Coordinator, SSIRI, Yei YAMBIO 58 11/21 Robert Singira OA 59 Joseph Kpiboroano OC 60 Peter Kasiano County Education Director, Nzara County, WES 61 Jovanina Bapeyo P1 Teacher, Yabna P/S, Nzara County 62 Madeline Louis Peter P2 Teacher, Kpirabe CGS, Yambio County 63 Martin Luke Local Instructor, PST101 64 Moses Samuel Local Instructor, PST101, AES Inspector, Nzara County 65 Batista Elias Terbia B Facilitator, New Sudan Women Federation 66 Joseph Ngere Paciko Deputy Governor, Western Equatoria State 67 Kungum Stanley Director, Teacher Education and Training, WES/Yambio 68 Pia Philip Michael Director for Administration and Finance, SMoE / WES 69 Nama R. Bullen Beshir Director General – SMoE / WES / Yambio, namabullen@yahoo.com, 0477187948, +88216 55582418 70 Hezekiah Wayo Director for Basic Education, SMoE / WES / Yambio Way0027@yahoo.com 71 William A. Baabe Director AES, SMoE, WES, Yambio, Wabaabe1964@yahoo.com 72 Dut Joseph Gov’t. Representative, SMoE 0477197805 73 Moses Robert Sambia Outreach Coordinator, SSIRI, Nzara RUMBEK 74 11/22 Ellie Dobing Trainer Manager, Internews, edobing@internews.org KAUDA 75 11/22 Angelo Guido Outreach Advisor, SSIRI 76 Mr. Abdulhamid Haron Nuba Mountains Region Education Director, REO, Kauda, tawasalara@yahoo.com 77 Habil Stephanus Salley Education Officer, REO Kauda 78 Amia James Friday Technologies Assistant, SSIRI 79 80 Khalil Said Morsal Outreach Coordinator, SSIRI 81 Ochan Bongo M&E Assistant, SSIRI 82 Jabra Tutu Facilitator, Terbia B 83 Fabsu P. Orandi IRI Coordinator, formerly with HEAR, now SSIRI 84 11/23 Sausan Hissen P1 Teacher, KDA P/S 85 Isaac Yahya P3 Teacher, Kudi A P/S 86 Jojo Hamad P1 Teacher, Upper Kumo P/S 73 Date Person Position 87 Charles Karumba Mwangi Manager, Educational Resource Center, HEAR, Kauda WAU 88 11/24 Bullen Murangi Senior Outreach Advisor, SSIRI 89 11/25 Getahun Gebru Senior Operations Officer Human Development, AFTH3 World Bank Sub-Office 90 Donato Ugali Zimbeli A for /Ag Director General, Director of Basic Education, MoE, WBEG State, Wau 91 Simon Njuku Advisor, AED 92 Mario Alex Ali Acting Director Planning, SMoE 93 ???? Director for AES, SMoE, Wau 94 Alfred Ahidemor (?) Makasio P1 teacher, Zoglona P/S 95 Andrea Henry Akuor Outreach Coordinator for Jor River County, SSIRI 96 Simaya Kemji Modi M&E Assistant, SSIRI ?? P4 Teacher, Billfam 97 Makuc Madut Meyar P3 Teacher, Billfam Model Primary School 98 Mayen Mayuom Kala HeadTeacher, Billfam Model School 99 John Adriano Ayor Technical Engineer, Radio Wau 100 Louis Pasquale Aleu Director, Wau Radio 101 Arkangelo Kur Majok Terbia B1 Teacher, also P1 Teacher, Bar Yar P/S 102 William Akot P3 Teacher, Bar Yar P/S 103 Peter Madut Madut P2 Teacher, Bar Yar P/S OUT OF SEQUENCE ADDITIONAL INTERVIEWS 104 Rejoice Manasseh Binyi Outreach Advisor for Central Equatoria State, SSIRI 105 Edward Kasran Senior Outreach Advisor 106 Kiri James AES Agroforestry Inspector, MoE, Juba 107 Emmanuel Wala Education Officer, USAID 108 Maree Melican Deputy Chief of Party, SSIRI 109 Robert Singira Outreach Advisor, Yambio 110 Tom Tilson Chief of Party, EDC SSIRI DEBRIEFING MEETING WITH REVIEW TEAM IN WAU Question: What were the results of the midterm evaluation and the “One Techer at a Time” document that Tom Tilson sent to staff? 11/24 Bullen Murangi Senior Outreach Advisor, SSIRI, Wau Maree Melican Deputy COP , SSIRI Edward Kasran Senior Outreach Advisor Kiri James AES Agroforestry Senior Inspector Simaya Kemji Modi M&E Assistant, SSIRI and former Windell Trust tutor Tom Tilson Chief of Party, EDC SSIRI Kent Noel Regional Director, EDC Question: What about Scalability and Sustainability? Tom Tilson again Maree Melican again Kent Noel again Maree Melican again Tom Tilson again Kent Noel again Tom Tilson Bullen Murangi Maree Melican Design of the audio programs and materials 74 Date Person Position Edward Kasran Maree Melican Tom Tilson Kiri James Bullen Murangi What about PST 101? Tom Tilson Maree Melican Tom Tilson Maree Melican What about the work at TTIs and the VSAT / Internet Tom Tilson Simaya Kemji Modi Tom Tilson WAU - JUBA 11/25 Radio Wau 111 Joseph Akol Angony Head Teacher, P2, Machakos Basic School 112 Peter Anyat Manyok Co-Head Teacher, Machakos Basic School 113 George Remise Ogweka Teacher, P1, Machakos Basic School 11/26 Athanas Mwamba NAIROBI 114 Jane Namadi Radio Producer, SSIRI Sarwat Alam IRI Consultant 115 Cisella Alex Scriptwriter, SSIRI 116 Donald Thige Digital Editor, SSIRI 117 Nicholas Mwangi Radio Engineer, Consultant 118 Bruno Musumba Digital Editor, SSIRI 119 Wani Stephen Scriptwriter, SSIRI 120 Duku David Scriptwriter, SSIRI 121 Jesca Wude Murge Scriptwriter, SSIRI 122 Wairimu Gachie Production Assistant, SSIRI Kent Noel Tom Tilson 75 ANNEX 5: LIST OF CLASSES & LEARNING GROUPS OBSERVED List of Classes and Learning Groups Observed Date Location Grade Level Teacher/Facilitator Course 1 11/17 Juba P1 teacher, Juba Model P/S Yunas Barsaba Amin Learning Village 2 11/17 Juba P2 teacher, Juba Model P/S Benji Taban Learning Village 3 11/17 Juba Terbia B1 Facilitator, Juba Jane Apollo, Ruta Zabedayo Terbia B 4 11/18 Maridi P1 Teacher, Haddow P/S Shaban Lado Learning Village 5 11/18 Maridi P2 Teacher, Mabrindi P/S Woodman Jawo Morris Learning Village 6 11/18 Maridi Maridi Teacher Training Institute Group of Tutors Support to TTIs 7 11/18 Maridi Terbia B Facilitator, Maridi Girls P/S Wanjala Joseph Terbia B 8 11/19 Kajo Keji P1 Teacher, Leikor PS Poni Joice, Learning Village 9 11/19 Kajo Keji P3 Teacher, Wudu P/S Kgji Jang Justin Learning Village 10 11/19 Kajo Keji Terbia A (English) Teacher, Kajo Keji Secondary School Justin Ladu Erasto Terbia A 11 11/19 Kajo Keji Terbia B Facilitator, Mundari Pentecostal Church ALP, KK Rombe Francis Modi Terbia B 12 11/20 Yei P1 teacher at Kinji P/S Evelyn Poni Ayobo Learning Village 13 11/20 Yei P2 teacher, Ronyi P/S John Juma Mike Learning Village 14 11/20 Yei P3 Teacher, Ronyi P/S Regina Alfred Learning Village 15 11/20 Yei Yei Teacher Training Institute Kepo James, Principal Support to TTIs 16 11/20 Yei Terbia B Teacher John Gama Terbia B 17 11/21 Yambio P1 Teacher, Yabna P/S, Nzara County Jovanina Bapeyo Learning Village 18 11/21 Yambio P2 Teacher, Kpirabe CGS, Yambio County Madeline Louis Peter Learning Village 19 11/21 Yambio Terbia B Facilitator ? Terbia B 20 11/22 Kauda Facilitator, Terbia B Jabra Tutu Terbia B 21 11/23 Kauda P1 Teacher, KDA P/S Sausan Hissen Learning Village 22 11/23 Kauda P3 Teacher, Kudi A P/S Isaac Yahya Learning Village 23 11/23 Kauda P1 Teacher, Upper Kumo P/S Jojo Hamad Learning Village 24 11/25 Jor River County P1 teacher, Zoglona P/S Alfred Ahidemor Makasio Learning Village 25 11/25 Wau P4 Teacher, Billfam ? Traditional 26 11/25 Wau P3 Teacher, Billfam Model Primary School Makuc Madut Meyar Learning Village 27 11/25 Jor River County Terbia B1 Teacher, also P1 Teacher, Bar Yar P/S Arkangelo Kur Majok Terbia B 28 11/26 Wau Teacher, P1, Machakos Basic School George Remise Ogweka Learning Village 29 11/21 Yambio Local Instructor, PST101 Martin Luke PS 101 76 ANNEX 6: ONE TEACHER AT A TIME June 7, 2008 To: EDC/SSIRI Staff From: Tom Tilson, Chief of Party Re: Mid-term Evaluation and new implementation strategies “One teacher at a time” This is a very important document! My apologies because this document is long and detailed, but there is much information to be conveyed immediately. Please read this document carefully and begin working with others on implementing the tasks right away. Each field staff member from SOA to OC is to acknowledge receipt of this document no later than 10 June to his or her supervisor. • Each OA is responsible for reporting receipt of this document by every OC he or she supervises, and submit the report as follows: o W. Equatoria, C. Equatoria, E. Equatoria, Jonglei, and Upper Nile to Edward (then to Maree) o W. BeG to Bullen (then to Maree) o The Three Areas to Maree • Janet Mola is responsible for reporting on the M&A Assistants to Tom. • Athanas Mwamba is responsible for reporting on all IT staff to Tom. Acknowledging receipt of this document is an important first step regarding our ability to communicate with each other. However, the most important activity is to review carefully this document. Using the reporting guidelines above, Edward, Bullen, Maree and Tom are to review this document in detail with the SOAs, OAs, M&E Specialist, and ICT Coordinator. These people, in turn, are to review this document in detail with the OCs, M&E Assistants, and IT staff no later than 15 June. This will mean setting up face-to-face meetings with every staff person or, if that is not possible, lengthy phone calls. Education Development Center, Inc. Southern Sudan Interactive Radio Instruction (SSIRI) Project UMEME Plaza (Off Ngong Road on Old Naivasha Road) Œ P. O. Box 25010 00603 Œ Nairobi Œ Kenya Œ Telephone: 3870906/3873088/3872269 Fax: 254-20-3876520 77 The SOAs, M&E Specialist, ICT Coordinator, and Deputy Chief of Party will need to confirm by 15 June that this document has been reviewed in detail by every field staff member. 78 Deadlines There are several deadlines included in this memo. They are listed below for your reference. See the text later in this document for additional information for each the tasks listed in the table below. June 10 1. Acknowledgement of receipt of this document. 2. Send to Juba a list of needed radios for all previously trained IRI teachers June 15 1. Confirm that his document has been reviewed in detail with all field staff. 2. Complete communications plan for each outreach staff. 3. Submit a plan for activities in June 4. OCs to have begun DAILY visits to chosen schools and learning groups 5. Inform all IRI teachers of new broadcasting schedule. (See Annex II for the broadcasting schedule.) 6. Arrange for distribution of radios to all previously trained IRI teachers as part of refresher course. 7. Communications Specialist to submit updated plan on activities to promote IRI programs. 8. Begin working on an expansion of PS101 with the Curriculum Development Center 9. IRI Advisors to establish plan for reviewing the length of pauses and for doing FE in Southern Sudan June 30 1. Submit State and County Plans for 2008 2. Complete MoU for each state 3. Every outreach staff to submit July activity plans to supervisor with copies to Deputy Chief of Party and Chief of Party (tentatively plan on EDC training the week of 14 July) 4. Complete implementation plan for PS101 79 Mid-Term Evaluation I just completed participating in a three-week evaluation of the SSIRI project. A Mid￾Term Evaluation is standard for all USAID projects. Although this review was later than normal in the life of a project given that our contract only has one more year, it is not too late for us to make some important corrections18. This memo focuses on the results of the recent Mid-Term Evaluation of SSIRI and some of the key implications for our work in the days and months ahead. Summary Report of the Mid-Term Evaluation The Team visited Juba, Torit, Arapi, Yambio, Maridi, Wau and Malakal. The trip to the Three Areas was cancelled. In Arapi and Maridi, the Team only looked at the technology initiatives at the TTIs. In other locations, the Team observed Learning Village and Terbia classes and held discussions with teachers, head teachers, education officials, and representatives from other USAID education projects and the World Bank. I went into the evaluation thinking that the project was in pretty good shape, but came out of the experience recognizing that we have some major problems. I will begin with an outline of some findings and recommendations of the Evaluation Team. I should receive the full report of the Evaluation Team later in June: Summary of Evaluation Team Report Overall Findings 9 Radio program production progressing according to plan 9 Listener Guides are of high quality, well integrated 9 Major problems with radio and digital technologies 9 Fewer than 1 in 7 Learning Village trained teachers using radios Comments on the IRI Programs 9 Program design is solid / good, clear objectives 9 Regularizes learning 9 Some good interactivity – potential for more 9 Teachers should be able to learn some generalizable skills useful in other classes 9 Some questions: Some pause lengths seemed too short; the pace often seemed too fast 9 Speed of some songs (do they reinforce natural English?) 9 Level of English in the programs demands too much in relation typical teacher’s capacity 18 The team consisted of two external evaluators ( Stuart Leigh, Team Leader and a highly experienced IRI Specialist) and Charles Tesar; two USAID staff (Lucy Kithome and Inez Andrews);a representative of the AES Department/MoEST (Reuben Mayen); and a Sudanese communications specialist (Sarah Mursal). I served as a resource person. 80 Major issues (A) radios and digital technologies 9 Broken radios (~5% in one year) 9 Observed radio experience for learners - not good 9 SW signals: problems with strength and clarity / times of day (Terbia) 9 Miraya FM signal too soft in some locations (problem with simultaneous broadcast of Miraya FM on a different frequency) 9 Radio volume too low for large classes 9 MP3 players with solar panels – batteries failing; even the JWIN not loud enough for large class (B) Teachers not using programs 9 Teachers dropping out - frustrated by sound issues or by their insufficient English skills to participate Recommendations 9 More study of digital technologies 9 Teachers need to be instructed in the use of radios: e.g., tuning shortwave, using 2 or more Lifeline radios in larges classes, tuning in Miraya FM on alternate frequency (around 95 on the dial), etc. 9 Audio programs need to be systematically analyzed and edited 9 Create at least 1 Model School per SSIRI county 9 Fully involve the states since their influence flows to counties 9 Train EDC staff and counterparts at all levels 9 Ensure full involvement of Basic Education Directorate as well as AES (because Learning Village is part of basic education) 9 Work with other ESL/EFL trainers, e.g. Windell Trust, starting in Malakal 9 Assess radio transmission and listenership, incidence of radio use in classrooms, radio programs’ effect on learning, comparisons of radio with other technologies 9 Focus on Feedback: Outreach staff reporting should be regularized and reviewed frequently, and responsive follow-up rapidly initiated 9 Broader more frequent field monitoring including senior staff (EDC/SSIRI – USAID) 9 Implement complete PS 101 soon and develop the new programs In short, the Mid-Term Evaluation was a wake-up call. Yes, we have much to be proud of – we’re enthusiastically supported by the Ministry of Education, Science and Technology at all levels, we have good radio programs, great teacher's guides, we’re on schedule in terms of production, and we have good partners such as Miraya FM and Bakhita Radio. But our radio programs still need improvement and, most importantly, we fell short in the one place where it matters the most – in the classroom. Almost all classes that the Team 81 observed had problems – the radios didn’t work right, the batteries in the digital devices ran down, the volume of many devices was too soft, and many teachers had difficulty carrying out their roles. And when the Team members talked to teachers about their experience with SSIRI last year, they found out that most of the teachers had stopped using the programs after a few weeks. We fell short (including myself) in not visiting classes regularly, in not reporting the problems that we saw, or when problems were reported in not understanding the depth of the problem and taking action. We didn’t train our field staff adequately or hold them accountable. We didn’t give adequate training to classroom teachers and facilitators or provide follow-up support. Implications for SSIRI of the Mid-Term Evaluation We now have a great challenge and opportunity. It’s our last chance to demonstrate to USAID, to ourselves and, most importantly, to the people of Southern Sudan that SSIRI can make an important difference to the lives of children, youth, adults and teachers – that SSIRI can enrich the lives of others by providing good quality educational programs. Fortunately, USAID and the Ministry remain very supportive. We are all committed to making sure that SSIRI works well, but we have to change some key elements of our strategy. We have to begin implementing our new strategies IMMEDIATELY, as there is another USAID team that will visit several of our sites in just over two weeks. See my other memo on the upcoming USAID Education Audit. The following sections describe findings and actions to be taken. The topics are in rough order of priority. Implementation at the classroom level 1. The highest immediate priority is for each OA to review in detail this document with each OC that he or she supervises and to start working on new the plans and activities accordingly. This review is to be completed with a confirmation sent by 15 June. 2. Our overall top priority, which is to start immediately, is to ensure quality learning experiences in Learning Village and Terbia classes. • An OC in every county where we are working is to focus on ensuring excellent IRI lessons in just a small number of classrooms, starting with “one teacher at a time.” • We will establish at least one Model School in each county, beginning with a few model teachers. (A Model School may have just 2 or 3 teachers who are exceptionally good, not all teachers.) 3. We will reduce our plans for expansion this year. These plans were based on the assumption that we were having a good experience with IRI lessons in the classes and learning groups, but this is not the case. • We will expand activities only modestly in six states (W. Equatoria, C. Equatoria, E. Equatoria, Jonglei, W. Bahr el Ghazal, N. Bahr el Ghazal); 82 maintain activities in one state (Upper Nile), and expand in the Three Areas (Abyei is doubtful at the moment). We will postpone activities in Lakes, Unity, and Warrap States. • We will consolidate staff in some states by shifting OCs from some new counties to more central counties. SOAs and OAs should consider selecting counties, in part, on the interest and willingness of the officials to become fully engaged in SSIRI. Also, ease of access should also be a consideration. 4. We will develop an improved training program with better materials for our own staff so that each person has the necessary skills and tools to carry out the work. • With the help of a new Senior Training Advisor and a consultant, we will revise, improve and expand our training materials starting in June. • We will make greater use of multimedia materials including video. • There will be a training workshop for all EDC outreach staff and, possibly, Ministry counterparts, in mid July. There will be follow-on training in the states. 5. We will improve the training program and materials that our outreach staff need for training teachers, facilitators, and Ministry officials. 6. We will work to ensure that all of our outreach staff report regularly – initially, there will be weekly reports on daily visits to schools as well as the usual bimonthly reports. Each staff member will be held accountable for the activities and timely reports. This will be a condition of continued employment. 7. For OCs who do especially well, we may create a new position called Senior OC with some additional responsibilities. 8. The Outreach Advisor for each state will have overall responsibility for implementing our new approach. There will be supervision and support from the Senior OA, the Deputy Chief of Party, the M&E Specialist, and the Senior Training Advisor (to be hired). 9. In order to continue efforts to integrate fully the SSIRI programs within the SMoE, all EDC staff will coordinate with and seek to fully involve state, county and payam staff in activities. Technologies Radios and digital technologies There were major problems with the radios and digital technologies in the classrooms observed by the Evaluation Team. The following is a table indicating the problem and action to be taken: Problem Action to be Taken The Team found that many teachers who had received IRI training last year, especially in Wau and Malakal, did not receive radios because our supply of radios had run out. Work with the Juba office to obtain the necessary radios and plan for distribution by 15 June. I would suggest that the radios be distributed as part of a refresher training to be held as soon as possible. Timetabling in the schools was a OCs to work with the Head Teacher to try 83 Problem Action to be Taken significant problem because either a grade 2 or grade 3 class is missing the Learning Village because of recess to arrange a staggered recess period so that no class would miss the IRI lesson Listening to programs on shortwave was problematic. Part of the problem is that the programs are broadcast during the daytime when shortwave signals are the weakest 1. Diagnose the shortwave reception during class time in the mornings and afternoons. Report your findings to the OA, SOA, and Deputy Chief of Party. Seek alternatives including a request for digital devices if the shortwave reception is not good and FM or medium wave is not an option. 2. Train teachers on tuning in shortwave stations and on using the antennae 3. Provide further separation in the frequencies for Terbia for Beginners -1 and Terbia for Beginners -2 (done effective June 9). See Annex II for broadcast time and frequencies. 4. Additional FM stations will be used for broadcasting the Terbia lessons. 5. EDC will explore the possibility of using medium wave stations in Malakal and Wau for Terbia and, perhaps even for the Learning Village. 6. Expand the use of digital media players Volume of the Lifeline radio too soft 1. In larger classes, provide a second (or third) Lifeline radio at the sides or back of the classes. Use one Freeplay radio in a class for every 40 students. 2. For Miraya FM, if the sound is too soft on 101, try tuning in on 95 on the dial. Volume on JWIN radios too soft when using an MP3 player 1. EDC will check on sound levels of programs recorded on the Nextar. 2. EDC will test the use of flash drives and SD cards in the JWIN radios 3. EDC will identify larger boom boxes Batteries on the digital devices go dead, especially with the Nextar MP3 player 1. EDC will look for an alternative to the Nextar MP3 player 2. EDC will provide a larger solar panel for charging the AA batteries in the small speakers Digital systems not yet of sufficient quality 1. Try out iPod system connected to a Freeplay radio being used by EDC in neighboring countries 84 Problem Action to be Taken 2. EDC will continue to assess alternative devices, some of which have already been identified 3. EDC will institute more rigorous M&E of the devices in the field, especially with support from the Technologies Specialist Change the term “low-cost technologies” We will start using the terms “digital devices” or “digital media players” Teacher training technologies 1. The observations by the Mid-Term Evaluation team regarding the visit to the TTIs include the following: a. There is a good start regarding the use of the Internet, but more work needs to be done. b. The team appreciated the fact that some staff at both institutions have already mastered the basic techniques of video production, but much work is needed on designing appropriate videos that will enhance the classes at the TTIs. 2. EDC will now focus activities on Arapi and Maridi TTIs plus the Curriculum Development Center on using the Internet and video production (TTI only). We will also provide basic support to the new computer/VSAT center in Malakal. 3. We will postpone the development of additional centers until our model is better developed at Maridi and Arapi. Improved monitoring and reporting A major shortcoming of our activities to date has been the poor monitoring and reporting process. Perhaps the single most important action to be taken in order to improve the implementation of SSIRI programs is to enact a monitoring and reporting process that works well. Each outreach staff member will be held strictly accountable for monitoring and reporting activities. • In order to implement the strategy outlined in this document, we must be able to communicate readily among all staff, but especially between the OCs and OAs. The first activity to test this premise is for all outreach staff to report on the receipt of this document by June 10. • We can only continue to support staff who are able to establish and maintain regular communication – at least every other day – either though email or by phone (Thuraya or mobile phone). That is, we will be forced to close down activities and/or replace staff in counties where we are not able to maintain regular communication. • EDC will provide computers and Thuraya phones to all staff including solar panels where needed, and even an Internet connecting device for isolated locations. • We can make greater use of SMS messages for regular and quick updates or queries (Every initiated SMS needs a reply so that the sender knows that the message has been received.) 85 • A communications plan and agreement (communications channels, time, problems to be addressed, etc.) is to be worked out with each outreach staff - Deputy Chief of Party with SOAs, SOAs with OAs, OAs with OC, M&E Specialist with M&E Assistant, and ICT Coordinator w IT staff. It is the responsibility of each staff member with a Thuraya to ensure that the batteries remain charged and that he or she has the phone turned on at least during certain specified times of the day. (Janet Mola can assist in designing a format for the communications plan.) This plan is to be completed by 15 June. • See Annex IV for a list of current staff, phone numbers, and email addressed. (Please send me corrections or updates to this list. • Each OA and SOA needs to prepare a log of communications with OAs and OCs respectively, and to submit the log at the end of each month. (Janet Mola can assist in preparing a log sheet. The M&E Specialist and M&E Assistants can monitor these logs.) • In order to assist outreach staff to improve their computer skills, EDC is hiring two IT Officers who will move from one location to another to provide training and support on computer skills, the use of email, and how to work with EDC computer-based forms. Lack of computer skills will not be an excuse for failing to provide reports on time, as telephone is an option. • Direct all computer-related queries to Athanas Mwamba. • Each OC is to visit an IRI class almost every day, provide support to the teacher/facilitator, and report on the findings and support activities. For the next few months, the OC will provide a WEEKLY detailed report to the OA and M&E Assistant on activities – name and date of schools and listening groups visited, any new data, problems identified, assistance given, training activities carried out, meetings with others, etc. • The M&E Assistant in each of the six states, plus additional coverage for the other locations, is to coordinate with the OA in developing a monitoring plan for each OC and to ensure that the progress is reported weekly. The M&E Assistant with support from the OA is to send the weekly report to the OA, who is to forward the reports to the SOA, M&E Specialist, and Deputy Chief of Party. • Senior level staff including the Chief of Party, Deputy Chief of Party, Senior Training Advisor, and the M&E Specialist will regularly visit field sites including observation of Learning Village and Terbia classes. At least one of these people will conduct such a visit every two weeks. • In September 2008, an EDC team including the Regional Director plus USAID representation will visit several sites using a charter flight in order to visit classes and talk to teachers and officials. Such monitoring visits will be conducted quarterly. Planning • The Communications Specialist needs to expand a range of activities to advertize the IRI programs including occasional special launching events. The major purpose is to increase the listenership of Terbia programs, but also to inform teachers and communities about the Learning Village. An update of such a plan should be submitted by 15 June. 86 • One of the findings of the Evaluation Team was that IRI teachers from last year were not aware of that IRI had started again this year. Please inform all teachers of the new broadcast schedule in Annex II by 15 June. o Note that P4 will begin broadcasting on June 16 and that ALL the Learning Village and Terbia programs will be repeated starting on that date. This will give OCs time to continue training. o It would be best if the County Education Officer, perhaps together with the OC, issued a written memo to all IRI teachers about the broadcast schedule. The OC is encouraged to draft this document for the County Education Officer incorporating the broadcast schedule in Annex II. • Each OA and OC is to have a state and county level plan respectively by 30 June. Some components of the plan are to include: o The Deputy Chief of Party with assistance from the SOAs is to provide an outline for sample state and county plans by 15 June. • A MoU is to be drafted among MoEST, EDC, and the State by 30 June. See Annex V for a draft MoU. The details for EDC and each state have to be modified to meet the circumstances in each state. • All outreach staff are to submit monthly plans to their supervisors by the end of the previous month, e.g., the plan for July is to be submitted no later than 30 June. In preparing the plans, assume that there will be an EDC training workshop during the week July 14. • Any individual having difficulty completing these plans and reports, is to consult with his or her supervisor well before each deadline. EDC will do everything possible to give assistance, but, in the end, it is the responsibility of each person to submit all reports on time.. • The submission of all the above plans (state/county annual plan, monthly plan, and MoU), as with the required reports (weekly and bimonthly), is a condition for continued employment with EDC. Improving our programs The Mid-Term Evaluation Team appreciated the quality of our radio programs, and they especially liked our teacher's guides. However, they also had the following comments for improving our programs: • Provide more opportunities to actively involve the learners. One of the evaluators was critical that the lessons were not child centered. • Review to ensure up-to-date English language instruction pedagogy • Review the pace of the lessons to ensure that they are not too fast • Review the pauses to ensure that they are not too short • Provide occasional longer pauses so that teachers can work with children on specific activities • Simplify the language used for instruction to make it as easy as possible for teachers to do the translations • Provide more songs and ensure that the pace and content are appropriate for reinforcing the teaching of English or other objectives 87 • Strengthen formative evaluation through broader and more systematic review as lessons are being developed and introduce more FE in Southern Sudan, perhaps by increasing the number of scriptwriters so that 1-2 can rotate in Southern Sudan in order to carry out FE. • In the future, consider replacing the local language literacy component with additional, improved English language instruction. Some of these suggestions can still be applied to the remaining lessons being developed, but for already completed lessons, probably the only thing possible without a major effort would be to adjust the pauses. The following actions can be taken by the IRI Advisors and scriptwriters by 15 June: • Establish a plan for reviewing pauses • Begin implementing a plan for conducting FE in Southern Sudan Professional Studies for Teachers The Evaluation Team emphasized our shortcoming in not implementing PS101. Not only is this a deliverable under our contract with USAID, but it has important content for teacher development. We have postponed implementation for several reasons including the fact that the MoEST has not made progress on implementing the in-service teacher education program through County Education Centers. In addition, we have given priority to implementing the Learning Village and Terbia. EDC will do the following: • For PS101 develop an implementation plan by 30 June including: o At least one SSIRI county where we can involve IRI teachers. One model is to engage teachers who may be attending IRI refresher courses. o At least one county (to be determined) that is implementing the in-service program through a CEC • By the week of 15 June, begin working with the CDC in Maridi on the development of additional audio series, probably continuing with Level 1 of the in-service program Additional support for teachers in English Many teachers would like to use the Learning Village programs, but are struggling because of their weak English skills. EDC plans to do the following in order to help these teachers: • EDC has expanded the length of initial training workshops from 2 days to 4-5 days so that teachers can have more practice using the IRI programs, especially for translating instructions from English into the local language • EDC will consult with MoEST regarding testing teacher's guides that include instructions in Arabic as well as English. • EDC is working with the scriptwriters to reduce the information burden on teachers, especially instructions that must be translated – reduce the length of instructions, break up instructions into small pieces, simplify the language, etc. 88 • EDC plans to create a new English for Arabic Pattern Teachers course based on the Terbia programs • In the future, EDC will consider developing a new audio series for teachers, perhaps weekly, and both in Arabic and English, that would provide specific information on upcoming lessons, such as content, English usage and vocabulary, and suggested activities. Some specific activities to be initiated by 30 June include the following: • Establish a plan for translating some of the teacher’s guides into Arabic and for assessing the effectiveness of using bilingual guides • Establish a plan for initiating the new English for Arabic Patterns Teachers based on the Terbia programs Administrative Issues Staffing In order to carry out our activities with sufficient support, EDC will hire the following new staff: 1. Deputy Chief of Party for Operations (Nairobi) – This person is to provide management of administration, finance, logistics, procurement, and HR and will help to ensure that the SSIRI activities proceed as effectively as possible. This person will have responsibility for SSIRI activities in both Kenya and Southern Sudan. Candidates are being interviewed. 2. Administration and HR officer (Juba) - Candidates are being interviewed. 3. Financial Assistant (Juba) – Candidates are being interviewed. 4. Short-term Logistics Assistant (Juba) – To be advertized. 5. Senior Training Advisor (Juba) - This person will to oversee all new training activities and materials development (contract being offered) 6. Training Consultant (Juba – two months) – The consultant will assist in the development of new training materials (Phoebe McKinney is to begin on 9 June). 7. Senior Teacher Training Advisor (Maridi) – This individual will oversee the development of new teacher training programs including the expansion of PS101 and a new English for Arabic Pattern Teachers based on Terbia. (Candidates being reviewed) 8. Teacher Training Consultant (Maridi). This person will work with the CDC to begin the design for expanding PS101. (Nick Boke is to arrive on June 10.) 9. Learning Technologies consultant (Maridi and Arapi - two months) – This person will focus on training and developing materials on the use of the Internet for curriculum development and staff improvement plus training on video production. (Richard Cavagnol is to arrive in Nairobi June 10.) 10. Recruitment Specialist (Juba) – This person will recruit staff for new positions and for replacing departing staff as may be needed. (Position being advertized) 11. Teacher Training Specialist (Juba) – This person will be responsible for implementing PS101. (Position is yet to be advertised) 89 In addition to the new staff positions listed above, we are also considering adding more OCs to certain counties. If at all possible, one of these positions should be covered by a county education staff member. The responsibilities would be as follows: • One OC is to focus on just a few schools to ensure quality implementation. The OC is to visit classes at least twice a week and provide support and mentoring to teachers. This OC is to develop at least one Model School per site. • One OC is to monitor and support other participating schools in the county through occasional visits to most of the schools, collecting good data on implementation, and organizing periodic refresher courses for the teachers. • One OC is to support the Terbia programs • And, where PS101 is implemented, possibly one additional OC to support this program Transportation The success of our initiatives will depend in part on transportation, both for our staff and for at least some staff at the GoSS, state, county, and payam levels. Thus, EDC will procure the following: 1. One vehicle is being procured for each OA (Procurement has been initiated) 2. One vehicle is being procured for assignment to the MoEST/AES Department 3. If approved by USAID, one motorbike will be procuring for each country and state (area) where we have staff. (Waiting approval from USAID) 4. One bicycle for each payam where we are working (OCs need to move ahead on this procurement immediately) 90 Annex I – Some characteristics of Model Class 1. A good radio with loud volume (We found in several locations that the Miraya FM signal was much stronger around 95 on the dial instead of the official frequency of 101) 2. The radio is sufficiently charged for the lesson 3. There is a second Lifeline radio in the back of large classes – one radio for each 40 students 4. The teacher has the radio clearly tuned in before the time of the lesson and places it in a good location in the classroom 5. If a Terbia group is using a digital device, ensure that it is working well, that the batteries are well charged, and that you have the correct lesson ready on the device. Have spare sets of fully charged batteries. 6. The teacher has read the teacher's guides prior to the broadcast and is prepared with all required materials and with information written on the blackboard as may be needed 7. The class is ready and attentive at the beginning of the lesson 8. The lesson begins right on time - not late 9. The teacher does an excellent job translating the instructions, organizing the children as requested by the radio teachers, writing on the blackboard as needed, and fully engaging the children. The teacher is clear about what instructions get translated into the local language and which are to be given in English. 10. The children are attentive and participating actively 11. The teacher follows up well with post-broadcast activities based on the teacher’s guide. 12. The teacher uses the SSIRI program regularly - every day, or three times a week for the Terbia class 13. The teacher fills in the proper SSIRI form after the lesson. 91 Annex II – Time and frequencies of broadcasts Broadcasts begin 12 May 2008. The whole series will start again on 16 June. Radio Service Schedule Program Category Frequency Shortwave Mo-We-Fr 9:30 a.m. Mo-We-Fr 4:00 p.m. Terbia for beginners 1 (Repeat) 15660 KHz 15760 KHz Mo-We-Fr 9.30a.m. Mo-We-Fr 4.00 p.m. Terbia for Beginners 2 (Repeat) 17660 KHz 15390 KHz Mo-We-Fr 9.30 a.m. Mo-We-Fr 4.00 p.m. Terbia Advanced (Repeat) 15530 KHz 12070 KHz Bakhita FM Radio (Additional FM stations to be added) Mo-We-Fr 4:30 p.m. Tue-Thu-Sat 4.30 p.m. Mo-We-Fr 10.30 a.m. Tue-Thu 8.30 p.m. Terbia for beginners 1 Terbia for beginners 2 Terbia Advanced Terbia Advanced (Repeat) 91 FM Voice of Kauda (Additional Internews stations to be added) Th-Fri-Sat 5.30 p.m. Tentative FM Shortwave-The Learning Village Mon-Fri 9.00 a.m. Primary 1 15215 kHz Mon-Fri 9.30 a.m. Primary 2 11905 kHz Mon- Fri 9.00 a.m. Primary 3 15750 kHz Mon-Fri 9.00 a.m. Primary 4 (Beginning 16 June) 15760 kHz Miraya FM Mon-Fri 10.00 a.m. Primary 1 FM 101(*) Mon-Fri 10.30 a.m. Primary 2 FM 101 Mon-Fri 11.00 a.m. Primary 3 FM 101 Mon-Fri 11.30 a.m. Primary 4 (Beginning 16 June) FM 101 (*) In some locations, try tuning to 95 for better reception. 1 June 2008 92 Annex III – List of Telephone Numbers and Email Addresses No. Full Name Title Duty Station Phone Number E-mail 1 Achuil Arop Outreach Coordinator Malakal 8821643332477 aarop@edc-ssiri.org 2 Ale Peter Micheal Outreach coordinator Yei 0477-122315 apeter@edc-ssiri.org/ ale_lomichael@yahoo.com 3 Amuda James Outreach Coordinator Kajo Keji 882166790 0951 0477-103225 amuda_james@yahoo.co.uk / jamuda@edc-ssiri.org 4 Angela Wangechi Receptionist-Nairobi Nairobi 0722-759776 awangechi@sudanradio.org 5 Angelo Guido Outreach Advisor Kauda 882165026-2502 tambua2003@yahoo.com / aguido@edc￾ssiri.org 6 Angelo Ochan Abariamoi M & E Assistant Torit 0477-215900 abariamoi2000@yahoo.co.uk 7 Athanas Mwamba ICT Programs Coordinator Maridi 8821667900954, 0722-826562 0477- 153505 amwamba@edc-ssiri.org 8 Awori Emmanuel Festo Outreach Coordinator Kajo Keji +256-782713199 aworiemmy@yahoo.com 9 Beatrice Anusu Custodian-Nairobi Nairobi 0734-342074 banusu@edc-ssiri.org 10 Bullen Nginzo Senior Outreach Advisor Wau 8821643341906 0733-788281 0477- 153502 bnginzo@yahoo.com / bmurangi@edc￾ssiri.org 11 Charlton Doki Communication Specialist Nairobi cdoki@edc-ssiri.org 12 Chirillo Chol Outreach Coordinator Agok 8821643341905 cchol@edc-ssiri.org 13 Cho Bol Outreach Coordinator Panyagor 8821643341968 726821229 cbol@edc-ssiri.org 14 Cisella Alex Script Writer Nairobi 0722-719852 calex@edc-ssiri.org 93 No. Full Name Title Duty Station Phone Number E-mail 15 Daniel Kenyi Dima IT Officer Malakal dkenyi2003@yahoo.co.uk] 16 David Duku Script Writer Nairobi 0723-944780 dduku@edc-ssiri.org 17 Donald Thige Digital Editor Nairobi 0722-997400 dthige@edc-ssiri.org 18 Edward Kasran Senior Outreach Advisor Juba 8821643339903 0722-072438 0477- 153508 ekasran@yahoo.com/ ekasran@edc-ssiri.org 19 Esther Ndung'u Senior Admin & Human Resource Officer Nairobi 0722-203313 endungu@edc-ssiri.org 20 Evans Sebi Script Writer Nairobi 0724-079590 esebi@edc-ssiri.org 21 Evans Igu Tibi M & E Assistant Juba igutibi@yahoo.com 22 Francis Mwangi Digital Editor Nairobi 0722-409203 fmwangi@edc-ssiri.org 23 Grace Wambui Admin Assistant Nairobi 8821643331043 0733-282592/ 0722- 861838 gwambui@edc-ssiri.org 24 Irene Njeri Custodian Nairobi 0721-237735 injeri@edc-ssiri.org 25 Jane Namadi Production Advisor Nairobi 0721-555982 jnamadi@edc-ssiri.org 26 Janet Mola M&E Specialist Juba 882164334-1907 0733 716209 / 0720- 216385, 0477-204057 jmola@edc-ssiri.org 27 Jesca Wude Script Writer Nairobi 0735-455306 Jwude@edc-ssiri.org 28 John Friday Amia IT Assistant Maridi 882166790 0935 jamia@edc-ssiri.org / amiaj109rf@yahoo.com 29 John Mark Opoka Outreach Coordinator Magwi 882166790 0008 0477-161256 okolongoopoka@yahoo.com / jopoka@edc-ssiri,org 30 Joseph Njama Ndirangu Project Driver Nairobi 0722-332762 jndirangu@edc-ssiri.org 94 No. Full Name Title Duty Station Phone Number E-mail 31 Josephine Osike Script Writer Nairobi 0727-435931 josike@edc-ssiri.org 32 Josh Agukoh IT Coordinator Nairobi 0720-589838 jagukoh@edc-ssiri.org 33 Joyce Geri Terbia Script Writer Nairobi 0722-679010 jgeri@edc-ssiri.org 34 Khalil Said Morsal Outreach Coordinator Kauda 8821643336174 kuwa khalil@yahoo.co.uk /kmorsal@edc-ssiri.org 35 Kpiboroano Joseph V. Outreach Coordinator Yambio 0477-198601 36 Ladu Mikaya Project Driver Juba 8821650262504 37 Maree Melican Deputy Chief of Party Juba 8821643331658 0721-831561 / 0477- 116990 mmelican@edc-ssiri.org 38 Martina Costa Outreach Coordinator Wau 8821643341967 mcosta@edc-ssiri.org 39 Mary Oyella Goretti M&E Assistant Juba 8821650262507, 0477-153509 moyella@edc-ssiri.org 40 Mercy Karuri Senior Finance & Operations Administrator Nairobi 0733-747969 mkaruri@edc-ssiri.org 41 Mercy Kolok Scriptwriter Nairobi mkolok@edc-ssiri.org 42 Moses White Outreach Coordinator Mundri 882166790 0927 +249-29436529 mwhite@edc-ssiri.org 43 Moses Ayiga IT Officer Juba 0477-133238 mosesayiga@yahoo.com 44 Mukwasa Marjan M & E Assistant Yambio +249-9129322246, 0477-110305 mukwasamarjan@yahoo.com 45 Nicholas Lugalia General Clerk/Custodian Nairobi 0721-170625 nlugalia@edc-ssiri.org 46 Nicodemus Bior Script Writer Nairobi 0720-473870 nbior@edc-ssiri.org 95 No. Full Name Title Duty Station Phone Number E-mail 47 Obang Okumbul Outreach Coordinator Pochalla 8821643341904 ookumbul@edc-ssiri.org 48 Onek John Orach Outreach Coordinator Juba 8821643336175, 047-114189 onekakila@yahoo.com/oorach@edc￾ssiri.org 49 Ochan Richard Bongo M & E Assistant Jonglei ochanbongo@yahoo.co.uk 50 Phillip Yorka Jaraba Finance Officer Juba 882165026-2506 , + 249 912874466 , 0477-153506 pjaraba@edc-ssiri.org 51 Rejoice Binyi Outreach Advisor Juba 882166790-0929, 0 477-121582, +249-121920429 kidenbinyi@yahoo.co.uk/ rbinyi@edc￾ssiri.org 52 Remijio Joseph Jamba Outreach Advisor Bor +249-919440781 adropiako@yahoo.com 53 Reuben Ohisa Outreach Advisor Torit 882166790-0009 256 , 0477-142941 ohisa_reuben@yahoo.com / rohisa@edc-ssiri.org 54 Richard Aluma Administrative and Logistics Officer Juba 882165026-2505, + 249 913437984 , +249129083653, 0477-153500 raluma@edc-ssiri.org 55 Robert Singira Outreach Advisor Yambio 8821667900933, 0477-156804 singira_robert@yahoo.co.uk / rsingira@edc-ssiri.org 56 Robert Wuda Outreach Coordinator Maridi 8821643341969 wudarobert@yahoo.co.uk / rwuda@edc￾ssiri.org 57 Serah Kiden IT Officer Arapi 882164334-1966 skiden@edc-ssiri.org / skiden@yahoo.com 58 Simaya Kenyi Kodi M & E Assistant WBEG 59 Sheila Mweha Finance Coordinator Nairobi 0721-556240 smweha@edc-ssiri.org 96 No. Full Name Title Duty Station Phone Number E-mail 60 Solomon Chike Outreach Coordinator Kurmuk 8821643341373, + 249- 924140639 solochike@yahoo.com / schike@edc￾ssiri.org 61 Tom Tilson Chief of Party Nairobi 8821643336173, 0477103936, 0733- 440036 ttilson@edc.org 62 Tiondi Partrick Uga Outreach Coordinator Magwi/Nimule +256-714266428 0477-215900 tiondipartick@yahoo.com 63 Tito Achire Outreach Coordinator Ikotos lapatokok2000@yahoo.com 64 Victor Vasquez IRI Advisor Nairobi 0722-209622 vvasques@edc.org 65 Victor Tombe Lako Outreach Advisor-Secondary Schools Juba +249-918275563 vlako@edc-ssiri.org, vtombe@hotmail.com 66 Wairimu Gachie Production assistant Nairobi 0722-825011 wgachie@edc-ssiri.org 67 Wani Stephen Scriptwriter Nairobi swani@edc-ssiri.org 68 Wesley Kibet Senior Project Accountant Nairobi 0721-206070 wkibet@edc-ssiri.org 97 ANNEX IV - EDC-SSIRI ORGANISATIONAL CHART- SUDAN BASED STAFF EDUCATION DEVELOPMENT Deputy Chief of Party ICT Programs Senior Outreach Advisor WES. CES, EES Outreach Advisor Three Areas (Abyei, S. Kordofan, S. Blue Chief of Party Th T l Senior Outreach Advisor NBeG, WBeG M & E Specialist Admin & HR Officer OA- W. Equatoria OC’s Robert W￾Maridi OA- C. Equatoria OC’s John Onek-Juba Amuda S￾Kajokeji OA- E. Equatoria Oh R b OC’s Opoka John￾M OA-Jonglei Remijio OC- Upper Nile OC’s Chol Bol￾OC’s Martina Lino C.- Wau VACANT- Jur OC’s VACANT- Aweil OC’s Khalil Said Morsal- Kauda Chike Solomon- M & E Assistants Mary Oyella- Juba Mukwasa M- Yambio Simaya Kenyi- WB G Outreach Advisor (Secondary IT Officers Serah Kiden￾Arapi M A IT Assistants Amia John F￾Finance Officer Finance Assistant Logistics Officer Richard Project Drivers 98 Annex V - Draft Memorandum of Understanding Memorandum of Understanding Between The South Sudan Ministry of Education, Science and Technology (Ministry), State Ministry of Education (State) and Education Development Center, Inc. (EDC) I. Introduction Education Development Center, Inc. (“EDC”) has been awarded Award No. 623-A-00-04- 00054-00 from the U.S. Agency for International Development (“USAID”) entitled Support to Education in Southern Sudan (“Prime Award” or "SSIRI"). The aim of this project is to improve the performance of children in literacy, English and numeracy in Southern Sudan. This Memorandum of Understanding ("MoU") is drawn up between EDC and the South Sudan Ministry of Education, Science and Technology (“Ministry” or “MoEST”) and the State Ministry of Education (“State”); there is no relationship between the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education and USAID under this MoU. II. Purpose of the Memorandum of Understanding The purpose of the MoU is to define clearly the roles of EDC and the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education. This MoU verifies the intention of all parties to collaborate with the ultimate aim of improving the performance of children, youth and adults in literacy, English and numeracy in South Sudan. Each party’s relationship to the other under this MoU is that of an independent contractor. Nothing in this MoU is intended to constitute a partnership, agency, employer, employee or joint venture relationship between the parties. Neither party may incur any debts or make any commitments for the other. III. Period of Performance of the Memorandum of Understanding The MoU describes specific areas of cooperation as they relate to EDC and the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education. This MoU will take effect on April 1, 2008, and will end on June 30, 2009. IV. MoU and Project Management The EDC Program Director is Kent Noel or his designee, Thomas Tilson, Chief of Party. The Program Director is duly authorized to act on behalf of EDC in all technical matters pertaining to the performance of the Program, except where specifically indicated otherwise in this MoU. The EDC Agreement Officer for this MoU is Robert Rotner, Senior Vice President and Treasurer or his designee, Kathryn Sarken, Assistant Director, Office of Sponsored Programs. The EDC Agreement Officer is ultimately responsible for all matters related to the administration of this MoU on behalf of EDC. The terms of this MoU cannot be changed without the approval of the EDC Agreement Officer. The South Sudan Ministry of Education, Science and Technology and the State Ministry of Education must direct all communications with USAID regarding this Agreement through EDC’s Agreement Office. V. Program Areas of Cooperation SSIRI is an integral program of the Ministry of Education, Science and Technology at the central, state, county and payam levels. The specific programs include the following inter￾related radio-based programs: • The Learning Village for primary schools 99 • Teaching English through Radio-Based Instruction for All (Terbia) • Professional Studies for Teachers The Learning Village programs are implemented throughout South Sudan in primary schools. Terbia programs are implemented in secondary schools, AES learning centers, and other listening groups. The Professional Studies for Teachers program is part of the MoEST in￾service teacher training program and is usually implemented through County Education Centers. VI. Roles and Responsibilities The parties agree that EDC will, in order to implement the SSIRI Project, require contributions of time, Ministry and State resources, collaboration and cooperation; therefore, EDC, the State and the Ministry agree as follows: 1. The South Sudan Ministry of Education, Science and Technology through the Department of Alternative Education Systems agrees to provide the following in support of SSIRI: • Provide leadership to the States on the development of strategies and plans for implementing SSIRI radio programs • Support the States on the development of an annual plan for SSIRI • Sponsor workshops with State officials on SSIRI activities • Provide support as may be necessary through the Curriculum Development Center in Maridi for the development of SSIRI programs • Provide access and support to national secondary schools, teacher training institutes, and County Education Centers as may be necessary for implementing SSIRI programs • Collaborate with EDC in providing data and statistics on schools and alternative education centers • Provide EDC outreach staff with information on school and alternative learning center calendar/s • Provide office space for 1-2 senior EDC staff in the MoEST compound • Pay salaries to AES/SSIRI teachers/facilitators within each State in accordance with MoEST policy • Identify partners to assist with program implementation • Provide ongoing monitoring of SSIRI activities at the State level The Ministry accepts full and sole responsibility for the payment of all costs associated with carrying out the above tasks. 2. The State Ministry of Education agrees to provide assistance with the implementation the SSIRI programs as follows: For the Learning Village • Provide information on the location of all schools and Community Girls’ Schools in the state • Report on the number of teachers by grade level in the state For Terbia Report on the potential number of Terbia groups in the following centers: 100 • ALP • IELC • Adult Literacy • Other groups such as women, youth, church, etc. For Professional Studies for Teachers • Identify County Education Centers (CECs) implementing the in-service teacher education program In order to support the implementation of the SSIRI programs, the State agrees to contribute and carry out the following: • Develop a State Annual Plan for SSIRI activities • Identify and arrange for appropriate State education officials (Alternative Education Systems, primary and secondary, and teacher education ) to participate in SSIRI orientation, planning, and Training of Trainers workshops • Provide TOT trainers to co-facilitate with EDC outreach staff in the training of facilitators for Terbia and Head Teachers and teachers for the Learning Village • Provide regular, ongoing monitoring of SSIRI activities in each county/payam to ensure effective implementation • Provide a report in writing to EDC on a quarterly basis with information regarding the progress of the radio programs including updated enrollment figures, successes, challenges or difficulties that they have experienced, and data on any recruitment of more schools or Terbia groups • Prepare and submit reports to EDC/SSIRI on all SSIRI activities including training workshops, monitoring reports, etc. • Provide leadership to the county education officers on the development of plans for implementing SSIRI programs. The County plans should include: • The names and locations of participating schools, listening groups, and groups of teachers for Professional Studies for Teachers • Training activities • Process for distributing radios and guides • Monitoring process • Collaborate with EDC in providing school and alternative education center data and statistics • Provide EDC outreach staff with information on school and alternative learning center calendar/s • Identify appropriate County education officials (primary, secondary, AES inspectors/field education officers, and CEC trainers) to participate in SSIRI orientation, planning, and training of trainers workshops • In collaboration with EDC outreach staff, plan and facilitate workshops to train facilitators for Terbia, Head Teachers and teachers for the Learning Village, and CEC officials and local instructions for Professional Studies for Teachers • In collaboration with EDC Outreach staff and payam officials, County officers will conduct regular, ongoing monitoring of SSIRI activities in each county to ensure effective implementation 101 • Provide a space for EDC outreach staff in the State education office and, if possible, in the county offices as well The State accepts full and sole responsibility for the payment of all costs associated with carrying out the above tasks. 3. Education Development Center, Inc. agrees to provide technical assistance, equipment and materials. Specifically EDC agrees to: • Provide the following personnel per State: Outreach Advisor at the State capital level (some States may share an Advisor), Outreach Coordinator(s) at the county level, and M&E Assistant at the State level (some States may share an M&E Assistant). • Provide radios: Radios for each participating school equal to the number of grade 1 and grade 3 streams, a radio for each AES learning group, and radios or an MP3 device for teachers in the Professional Studies for Teachers program • In some circumstance, EDC may provide alternative technologies in the place of radios such as a MP3 player with speakers • Provide teacher's and facilitator’s guides: One guide for each teacher or facilitator. • Orientate GoSS, State and County education officials to SSIRI • Work with education officials to identify schools and P1, 2, 3 and 4 teachers • Work with education officials to identify Terbia listening groups and facilitators • Work with education officials to identify teachers for the Professional Studies for Teachers program • Train education officials, Head Teachers and P1 to 4 teachers on the Learning Village • Provide implementation guidelines for each of the Terbia programs • Provide Learning Village and Terbia monitoring forms • Provide ongoing technical support for CEO and Learning Village schools and teachers • Provide ongoing technical support for LEO (Locality Education Officer) and Terbia facilitators. • In collaboration with State and county officials, EDC will conduct TOT training on SSIRI radio programs • EDC will provide for the broadcast of SSIRI programs on shortwave and, to a more limited extent, on FM stations • EDC may also provide on occasion promotional materials such as a brochure, flyer or banner • Provide some financial assistance for conducting workshops, possibly including supplies, tea breaks, meals, and travel, accommodations and per diem for teachers having to travel from another location. Such payments will be consistent with SMoE regulations, guidelines, and practice as well as with EDC and USAID regulations. • Provide a bicycle for each participating payam. The payams agree to use the bicycles for travel required to support the implementation of SSIRI activities. VII. Payment No payment will be made by EDC to the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education for the work performed by the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education under this 102 MoU. No payment will be made by the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education to EDC for the work performed by EDC under this MoU. VIII. Ownership of Work Product Work Product shall mean all data, reports, curricula, specifications, outlines, drafts, software, videotapes and any other materials or deliverables, in any medium, which are prepared and/or developed by the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education in the performance of its obligations under this MoU. Subject to the terms of this MoU, all intellectual property rights in the Work Product prepared and/or developed by the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education hereunder (the “Work Product”) are the sole and exclusive property of the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education. Notwithstanding the foregoing, in order to permit EDC full and effective use of the Work Product as contemplated by this MoU, the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education hereby grants EDC a non-exclusive, irrevocable, worldwide, fully paid-up, royalty-free license (the “EDC License”) to the Work Product, with the right to sublicense, make, use, reproduce, distribute, display, perform, transmit, improve, and create derivative works based upon such Work Product, in any manner and in any medium by any means now known or hereinafter invented, for purposes of the Program and in other ways consistent with EDC’s educational mission including development of commercial applications of the Work Product. EDC will include in the Work Product that are so used an acknowledgement of the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education as the developer and owner of copyright of the Work Product, using substantially the same form of acknowledgement as is included by the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education in the Work Product as delivered to EDC. If the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education intends to use or incorporate any work owned by third parties, images of people or entities into the Work Product, the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education must first, (a) identify the work to be included in writing to EDC, (b) identify in writing any limitations on use of the work and (c) obtain all rights in such work necessary for EDC to exercise all rights to the Work Product arising from this MoU. IX. Amendments The MoU may only be amended by formal written modification. Amendments may be initiated by EDC to the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education or by the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education to the EDC Agreement Officer. Amendments shall not be binding on either party until the amendment has been signed by the authorized representatives of both parties. X. Termination and Suspension 1.EDC may terminate the MoU at any time, in whole and in part, upon written notice to the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education, whenever it is determined that the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education has materially failed to comply with the terms and conditions of the MoU and has failed to cure said breach within thirty (30) days of notification. 103 2.The South Sudan Ministry of Education, Science and Technology and the State Ministry of Education may terminate the MoU at any time, in whole and in part, upon written notice to EDC, whenever it is determined that EDC has materially failed to comply with the terms and conditions of the MoU and has failed to cure said breach within thirty (30) days of notification. 3.This MoU may be terminated at any time, in whole or in part, by mutual written consent of the parties. Both parties shall agree upon termination conditions, including the effective date, and, in the case of partial terminations, the portion of the MoU to be terminated. 4.If at any time USAID determines that continuation of all or part of the funding for the Prime Award should be suspended or terminated, then EDC may, following notice to the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education, suspend or terminate the MoU in whole or in part. If the Program is suspended and the situation causing the suspension continues for sixty (60) days or more, then EDC may terminate this MoU in whole or in part upon written notice to the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education. XI. Executive Order 13224 The South Sudan Ministry of Education, Science and Technology and the State Ministry of Education shall comply with Executive Order 13224 and U.S. laws that prohibit transactions with, and the provision of resources and support to, individuals or organizations associated with terrorism. It is the legal responsibility of the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education to ensure compliance with these Executive Orders and laws. XII. Limitation It is expressly understood that neither EDC nor USAID has any obligation to provide any support to the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education in addition to that specified in this agreement. It is further understood that the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education does not have any obligation to provide any support to EDC in addition to that specified in this agreement. XIII. Indemnification The South Sudan Ministry of Education, Science and Technology and the State Ministry of Education agrees to defend, indemnify, and hold harmless EDC and USAID, its trustees, directors, officers, employees and agents, and their respective successors, heirs and assigns from and against any liability, damage, loss, cost, fee or expense (including reasonable fees of attorneys and other testifying or consulting professionals, and expenses of litigation) incurred by or imposed upon EDC or any one or more of them in connection with any allegations, claims, suits, actions, demands, whether threatened or pending, or judgments arising from or relating to the performance of this MoU, to the extent such liability, damage, loss, or expense is caused by or results solely from the negligent or intentional acts or omissions of the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education, its directors, officers, employees or agents. EDC agrees to defend, indemnify, and hold harmless the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education, its trustees, directors, officers, employees and agents, and their respective successors, heirs and assigns (the “South Sudan 104 Ministry of Education, Science and Technology Indemnitees” and the “State Ministry of Education Indemnitees”) from and against any liability, damage, loss, cost, fee or expense (including reasonable fees of attorneys and other testifying or consulting professionals, and expenses of litigation) incurred by or imposed upon the South Sudan Ministry of Education, Science and Technology Indemnitees and the State Ministry of Education Indemnitees or any one or more of them in connection with any allegations, claims, suits, actions, demands, whether threatened or pending, or judgments arising from or relating to the performance of this MoU, to the extent such liability, damage, loss, or expense (i) is solely caused by or results solely from the negligent or intentional acts or omissions of EDC, its trustees, directors, officers, employees or agents, or (ii) arises from the breach of any other warranty or representation made by EDC. XIV. Liability Neither EDC nor USAID assume liability for any third party claims for damages arising out of this MoU. XV. Disputes Any dispute or disagreement which cannot be resolved by EDC and the South Sudan Ministry of Education, Science and Technology and the State Ministry of Education and any controversy, claim, or dispute otherwise arising out of or in connection with this MoU or breach thereof, or the Program, shall be resolved by arbitration. XVI. Authority; Binding Effect By his or her signature below, each signatory hereto represents and warrants that he or she is duly authorized to enter this MoU on behalf of the party he or she purports to represent such that, upon execution and delivery, this MoU shall be a binding obligation of such party. This MoU constitutes the entire agreement among the Ministry of Education, Science and Technology, the State Ministry of Education, and EDC with respect to the subject matter hereof and supersedes all prior agreements and understandings, whether oral or written, between us relating to the subject matter. The work and services required herein shall not be transferred or assigned by the South Sudan Ministry of Education, Science and Technology or the State Ministry of Education without the prior written consent of EDC. In witness thereof the South Sudan Ministry of Education, Technology and Science, State Ministry of Education and Education Development Center, Inc. The South Sudan Ministry of Education, Technology and Science Name: Title: Date: The South Sudan State Ministry of Education Name: Title: Date: 105 Education Development Center, Inc. Name: Robert Rotner Title: Senior Vice President and Treasurer Date: 106 ANNEX 7: EDC/SSIRI - RADIO BROADCASTING CONSULTANCY 1.0 Introduction The Southern Sudan Interactive Radio Instruction (SSIRI) Project is a program of the Ministry of Education, Science and Technology with funding from USAID. It is implemented in all 10 states of Southern Sudan and in the Three Areas of Abyei, Southern Kordofan (Nuba Mountains) and Blue Nile. The main focus of SSIRI is to develop and broadcast educational radio programs including: (a) Primary school subjects to children in schools, (b) English to youth and adults, and (c) Staff development programs for teachers. The focus of this consultancy is to identify field strength for FM broadcasts in the areas where SSIRI is operating. The purpose of the this exercise is to survey FM reception of the FreePlay LifeLine Radio in comparison to other FM receivers with the objective of identifying strength and weaknesses in FM broadcasts in these areas. 1.1 Scope of Work: My responsibilities as outlined in the SOW were defined as follows: ƒ Visit FM broadcast station in Juba, Maridi and Wau in Southern Sudan ƒ Identify other possible means of broadcasting services that SSIRI can utilize ƒ Perform radio reception site survey for the FreePlay LifeLine in comparison with other FM receivers ƒ Provide report on radio reception of SSIRI broadcasts in FM in comparison to SW broadcast ƒ Provide recommendations for improving radio reception using the FreePlay LifeLine and the Sanyo radio ƒ Provide recommendations on the use of digital devices with FM reception, like the AWP6040R Wireless PA System 1.2 Deliverables: 107 1. Report on the findings of FM broadcast 2. Field surveys for FM and SW coverage (maps) 1.3 Supervision: In the course of this assignment I have been reporting to Thomas Tilson, Chief of Party. And while in the field to the ICT Coordinator, Athanas Mwamba. 2.0 STRATEGY I used an Audemat-Aztec FM Explorer connected to a laptop computer loaded with FM Explorer ver.7.0.1 software to carry out the tests. The Audemat-Aztec Navigator FM Meter shown above is used for; • FM Modulation measurements • RF coverage analysis • Multi-station field strength measurements (mobile) • Field surveys. • Sub-carrier and RDS/DARC analysis Navigator FM is a compact FM product for field surveys as well as on site/studio monitoring of a station’s complete FM signal. The Navigator is frequency agile and can be set up to measure automatically one or multiple frequencies. It combines in the same unit RF, modulation, pilot, RDS and DARC measurements and analysis. At each of the sites I carried out the following tests using the FM Navigator. • FM band scan, this is a scan of the entire FM band of 88 to 108 MHz aimed at picking any fm broadcast in the region. • RF Levels: This is a measure of the carrier power from a particular transmitter. • MPX level: Measure of modulated signal level. 108 • Pilot: Measure of the 19 KHz reference signal needed to decode stereo. • RDS, DARC, This are additional sub-carriers that are used to carry some additional digital information like the stations name, title of song playing, etc. • L+R, L-R, L, R: These components of the broadcasted signal are explained further below in the overview of FM broadcasting. 3.0 Overview of FM Broadcasting (Literature review): Around the world, the FM broadcast band falls within the VHF part of the radio spectrum. Usually 87.5 - 108.0 MHz is used, or some portion thereof, with few exceptions: • In the former Soviet republics, and some Eastern Bloc nations, an older band from 65.9 - 74 MHz is also used. Assigned frequencies are at intervals of 30 kHz. This band, sometimes referred to as the OIRT band, is slowly being phased out in many countries. • In Japan, the band 76 - 90 MHz is used. The frequency of an FM broadcast station (more strictly its assigned nominal centre frequency) is usually an exact multiple of 100 kHz. In most of the Americas and the Caribbean, only odd multiples are used. In some parts of Europe, Greenland and Africa, only even multiples are used. In Italy, multiples of 50 kHz are used. There are other unusual and obsolete standards in some countries, including 0.001, 0.01, 0.03, 0.074, 0.5, and 0.3 MHz. In FM broadcasting it is important that stereo broadcasts should be compatible with mono receivers. For this reason, the left (L) and right (R) channels are algebraically encoded into sum (L+R) and difference (L-R) signals. A mono receiver will use just the L+R signal so the listener will hear both channels in the single loudspeaker. A stereo receiver 109 will add the L+R and L-R signals to recover the Left channel, and subtract the L+R and L-R signals to recover the Right channel. The (L+R) Main channel signal is transmitted as baseband audio in the range of 30 Hz to 15 kHz. The (L-R) Sub-channel signal is modulated onto a 38 kHz double-sideband suppressed carrier (DSBSC) signal occupying the baseband range of 23 to 53 kHz. A 19 kHz pilot tone, at exactly half the 38 kHz sub-carrier frequency and with a precisely defined phase relationship to it, is also generated. This is transmitted at 8–10% of overall modulation level and used by the receiver to regenerate the 38 kHz sub-carrier with the correct phase. The final multiplex signal from the stereo generator contains the Main Channel (L+R), the pilot tone, and the sub-channel (L-R). This composite signal, along with any other sub-carriers (SCA), modulates the FM transmitter. Converting the multiplex signal back into left and right audio signals is performed by a stereo decoder, which is built into stereo receivers. In order to preserve stereo separation and signal to noise parameters, it is normal practice to apply pre-emphasis to the left and right channels before encoding, and to apply de￾emphasis at the receiver after decoding. Stereo FM signals are more susceptible to noise and multipath distortion than are mono FM signals. This is due to imbalance of FM sideband ratios of the additional modulating signals created by the pilot tone and the sub-carrier channel. In addition, for a given RF level at the receiver, the signal to noise ratio for the stereo signal will be worse than for the mono receiver. The point at which the receiver input RF level reaches maximum monaural signal to noise ratio will be 23 dB lower than the receiver input RF level for maximum stereo signal to noise ratio. For this reason many FM stereo receivers include a stereo/mono switch to allow listening in mono when reception conditions are less than ideal, and most car radios are arranged to reduce the 110 separation as the S/N ratio worsens, eventually going to mono while still indicating a stereo signal is being received The range of an FM mono transmission is related to the transmitter RF power, the antenna gain and antenna height. The FCC (USA) publishes curves that aid in calculation of this maximum distance as a function of signal strength at the receiving location. For FM stereo, the maximum distance covered is significantly reduced. This is due to the presence of the 38 kHz subcarrier modulation. Vigorous audio processing improves the coverage area of an FM stereo station. 4.0 FINDINGS 4.1 JUBA In Juba within the UNIMIS compound is where the Miraya Fm Studios, transmitter and satellite uplink facilities are. The equipment used within this facility are of high standard (professional) and the members of staff have a good understanding of audio and broadcasting standards which they observe. The pictures above show the studio and up/downlink facility in juba. It can be seen that the equipments and setup conforms to professional broadcasting standards. The following readings and observations were taken in Juba. RF 85 dB MPX 75 KHz Pilot 7 KHz Left 10 dB 111 Right 10 dB SUMMARY FOR JUBA • The carrier and modulated signal are at good levels. • The transmitted signal components are of good transmitter configuration. • Studio equipment and signal processing of good broadcasting standards. • Audio levels good. • Satellite uplink equipment of good standard. • Transmitting using a 5 KW Eddystone Transmitter at 101 MHz. • Measured reception using a standard car radio receiver up to a distance of 80 kms from the transmitter. 4.2 TORIT Miraya FM gets its signal to Torit via satellite for re-broadcasting on FM. According to the briefing I had from the station technician the station transmits on a 1KW transmitter at 101MHz. I made 2 visits to Torit, this was because during the first visit I could not carry out any measurements as there was nil transmission. During my first visit I went to the UNIMIS compound where the transmitter is located and found there was a problem with the satellite downlink as the satellite receiver had no sync. This meant there was no audio into the transmitter, it was just transmitting the carrier and pilot signal only. This problem was reported to Miraya FM immediately. I went back to Torit after 5 weeks and found that though the problem had improved (there was signal from the satellite) the broadcasted audio levels were very low as indicated in the test results shown bellow in comparison to those in JUBA. JUBA TORIT RF 85 dB 45 dB (Torit uses a 1KW TX, JUBA a 5 KW) MPX 75 KHz 15 KHz 112 Pilot 7 KHz 6.5 KHz Left 10 dB -28 dB Right 10 dB -28 dB SUMMARY OF TORIT • Carrier level good. • Modulated power levels very low. • Audio levels very low. • Transmitting using a 1 KW Eddystone Transmitter at 101.5 MHz. • Frequent power failure due current surges caused when changing over between the power supply generators. 4.3 WAU Miraya FM gets its signal to WAU via a satellite dish connected to a receiver as shown above. The audio signal from the receiver is then fed directly to a 5 KW Eddystone FM transmitter shown below for re-broadcasting. The transmitter was however transmitting at 2.7 KW as indicated in its front display panel shown below. At the time I arrived at the transmitter site, I found that the power was off because of a tripped circuit breaker. On further inquiry I was informed that this was a common occurrence as all electricity within the UNIMIS compound is obtained from 2 generators that run on a rotation basis and so after a couple of run hours there is a manual switch over from 1 generator to the other. When this happens (because the transmitter is not connected on a UPS) the transmitter goes off. When the second generator kicks in, the transmitter draws a lot of current thus causing a power surge that causes the circuit breaker to trip. Someone has to notice this and reset the circuit breaker otherwise the transmitter stays off indefinitely. The following is a summary of my findings in WAU. 113 SUMMARY OF WAU • Carrier level good • Modulated power levels low • Audio levels low • Eddystone Transmitter rated at 5 KW but only transmitting 2.7 KW which is slightly above half the rated power at 101 MHz. • Frequent power failure due to current surges caused when changing over between the power supply generators. 4.4 YAMBIO In Yambio, Miraya FM is using a 1KW Eddistone transmitter, transmitting at 101.5MHz instead of the required 101 MHz. Here I found that the transmitter was not transmitting the 19 KHz pilot that is required by radio receivers in order to decode the stereo. There for reception in yambio was all Mono. This is shown below. The following readings were obtained and tabulated in comparison to JUBA. JUBA YAMBIO RF 85 dB 75 dB (Yambio uses a 1KW TX, JUBA a 5 KW) MPX 75 KHz 12 KHz Pilot 7 KHz 0.0 KHz Left 10 dB -7 dB Right 10 dB -7 dB SUMMARY OF YAMBIO • Carrier levels good • Modulated power levels low • Audio levels low • Transmitting using a 1 KW Eddystone Transmitter at 101.5 MHz. 114 • Transmitted signal has no pilot signal so reception in mono only. • Frequent power failure due current surges caused when changing over between the power supply generators. 4.5 MALAKAL Malakal was transmitting using a 5KW Eddystone Transmitter at 101MHz but was only running at 3.4KW. I found that the transmitter enclosure was very hot because the air conditioning unit could not cool it effectively. This was one of the reasons why the technician decided to down rate the output power to save the transmitter from cooking itself. JUBA MALAKAL RF 85 dB 68 dB (Malakal uses a 5 KW TX, JUBA a 5 KW) MPX 75 KHz 55 KHz Pilot 7 KHz 6 KHz Left 10 dB 2 dB Right 10 dB 2 dB SUMMARY OF MALAKAL • Carrier levels good • Modulated power levels low • Audio levels low • Eddystone Transmitter rated at 5 KW but only transmitting 3.4 KW which is lower than the rated power at 101 MHz. • Frequent power failure due current surges caused when changing over between the power supply generators. 115 5.0 OTHER OBSERVATIONS These were observations I found that were common in all the areas. I have divided them into two categories, one is Audio and the other is Hardware. 5.1 AUDIO The SSIRI programs are at a lower power level than other Miraya FM normal programs. Some of the sound effects, coral verses and songs are also recorded at a higher level than the teacher’s voice. The programs are also compressed when writing on CDs. AUDIO 1 AUDIO 2 AUDIO 3 AUDIO 1 In the first sector of audio 1 was an Intro to the SSIRI program then there is a sharp decline in the audio levels when the program starts visible in the second sector. AUDIO 2 These were the levels recorded during the SSIRI program. AUDIO 3 First sector SSIRI program followed by an interlude from Miraya in the second section. 116 5.2 HARDWARE FM broadcasting is supposed to suppress static and eliminate heterodynes from adjacent stations. These claims may be true, but I’ve consistently experienced problems listening to FM broadcasts in most places. Part of the difficulty is that manufacturers of modern "Hi-Fi" receivers are spending less and less on the AM/FM/SW tuner, while concentrating on the feature of the moment – whether it’s combined audio/video, Dolby Pro-Logic surround sound or a DTS decoder. Stereo tuners of the past included a signal strength meter and narrow/wide band selectivity – when was the last time you saw those features in a modern receiver? Another problem for modern equipment is that manufacturers try to squeeze the AM/FM/SW tuner into the same box as the digital signal processor and multiplexed display. As a result, spurious signals from the digital circuitry can find their way into a poorly shielded tuner, especially affecting the AM section. After subjecting the freeplay radio to various tests in comparison to other standard radio receivers (Sanya, Jwing, Sony), I found that it has problems with selectivity, Low output volume and poor audio reception. The other receivers performed much better. 6.0 MEDIUM WAVE RADIO BROADCASTING 6.1 WAU RADIO • Transmitting on 1071 KHz using a 5 KW transmitter • Equipment inventory is of old generation (for non-professional studio). • Non professional play out software. • Studios not automated, most of the processes are done manually. • Audio levels good and clear (this was assessed using the FreePlay receiver at a distance of 2-3 KM from the transmitter). 117 • Power source; Generator. • Expected radius of coverage should be larger than an FM transmitter of similar power rating but distance is affecter by time of day (day or night) or cloud cover. The waves will travel further at night. 6.2 RADIO MALAKAL • Transmitting using a 5KW transmitter. • Equipment inventory inadequate but functional. • Non professional play out software. • Studios not automated, most of the processes are done manually). • Audio levels good and clear (this was assessed using the freeplay receiver at a distance of 2-3 KM from the transmitter. • Power source; Generator. 75 KVA for the studio, 40 KVA and 27 KVA for the transmitter. • Expected radius of coverage should be larger than an FM transmitter of similar power rating but distance is affecter by time of day (day or night) or cloud cover. The waves will travel further at night. 7.0 RECOMMENDATIONS I have divided this section into 3 parts, one with recommendations to EDC the other to Miraya FM and finally a recommendation on Medium Wave Broadcasting. 7.1 EDC • Selection of receivers to be used in the project should be carefully done and a number of aspects considered, this include but are not limited to: o Selectivity – This is the ability to lock onto (spot on) a particular frequency and be able to suppress any adjacent frequencies. o Sensitivity – Ability to receive low broadcast signals, filter out any noise and amplify it to give a powerful output. 118 o Output Volume – The output volume should be high enough to cater for the size of listeners groups. o Audio clarity – The output audio should be clear of noise, hums, distortions that could be caused by audio amplifiers, transformers, inferior speakers etc. EDC should opt for better and more powerful radio receivers especially for areas that are far away from the transmitters. • Stereo FM signals are more susceptible to noise and multipath distortion than are mono FM signals. This is due to imbalance of FM sideband ratios of the additional modulating signals created by the pilot tone and the sub-carrier channel. In addition, for a given RF level at the receiver, the signal to noise ratio for the stereo signal will be worse than for the mono receiver. The point at which the receiver input RF level reaches maximum monaural signal to noise ratio will be 23 dB lower than the receiver input RF level for maximum stereo signal to noise ratio. For this reason many FM stereo receivers include a stereo/mono switch to allow listening in mono when reception conditions are less than ideal, and most car radios are arranged to reduce the separation as the S/N ratio worsens, eventually going to mono while still indicating a stereo signal is being received. For this reason I would recommend the use of receivers that offer this choice option. • The FreePlay radio is designed to operate at very low current levels and as such this compromises its output power levels. It is my recommendation that other receivers should be considered and that the freeplay radio should only (if not avoidable) be used in areas which are in close proximity to the transmitters and for smaller groups of people. • In certain areas the use of external FM antennas (Yagi) could improve the reception. • Make sure that during production of the SSIRI programs, all the sound effects, songs and coral verses are recorded at a lower level than the teachers’ voice. This is because when a broadcaster is relaying the signal they set the mixer to detect 119 the highest peaks on the audio signal and maintain the entire broadcast within that range so as to avoid clipping. The teachers Voice is what is important in the SSIRI programs and so it should be recorded so as to give the highest peaks. • Increase monitoring of the SSIRI programs by making it the responsibility of all staff members to listen and give feedback of the programs broadcasted. 7.2 MIRAYA FM • The CDs containing the SSIRI programs given to MIRAYA FM are normally compressed (MP3) and so Miraya FM should apply normalization at 100 percent. This helps to reconstruct and rebuild the audio back to the levels it was recorded at (before compression). Despite the fact that Miraya uses very professional software that can be used to normalize the compressed audio we can not rule out the human element whereby a presenter or a DJ might not know how or may ignore the normalization. A way to avoid this happening would be to record the audio in WAV format which would not need de-compression. The only down side to this is that the programs would use more CDs than when recorded in MP3 format. NOTE: All recording should be done at 0 dB. • The satellite system being used is a passive system and so there are a lot of power losses due to the long distance the signal travels from the uplink facility to the satellite and down to the respective transmission sites. To compensate for these losses it would be advisable to use an audio processor at every downlink center to improve fidelity and power levels of the signal received from the satellite before plugging it into the transmitter. This will improve the modulated signal power in these areas thereby increasing audio levels. • Carry out power audit of every transmitter site, take down the load details of each and match this with circuit breakers that can handle the loads. • Power all transmitters from UPS and use the generator power to (only) charge the UPS battery. This will increase the transmitter lifetime, reduce amplifier faults and transmission downtime. 120 • Power the various components of the transmitter through different circuit breakers. (Split the power). • Increase monitoring, this can be done by having a radio receiver at each UNIMIS site where there is a Miraya FM transmitter and also by installing remote monitoring equipment. This would greatly reduce the off air time as it would be detected much faster and necessary action taken. 8.0 FM BROADCASTERS WITHIN SOUTHERN SUDAN BASED ON RESULTS OBTAINED FROM FM BAND SCAN To view the soft copies of the Band scan results below you need the FM explorer ver 7.0.1 installed in your computer. JUBA Miraya FM at 101.0 MHz (Has the best FM signal quality in juba) 90 MHz 88.2 MHz TORIT Miraya FM at 101.5 MHz Voice of Eastern equatorial at 97.5 MHz (Has the best FM signal quality in Torit) YAMBIO Miraya at 101.5 MHz WAU Miraya at 101.0 MHz MALAKAL Miraya at 101.0 MHz Further research needs to be carried out on these or any other available options. This would help identify the equipment and transmission power from each broadcaster. These aspects have a great bearing in estimations for area of coverage and station broadcasting reliability. NOTE: 121 Some of the broadcasters also broadcast at specific times of the day and may have been missed out during the band scan. 9.0 MEDIUM WAVE BROADCASTING The main advantage of Medium wave transmission is that it has a wider footprint of coverage in comparison to the FM broadcast. However, it is susceptible to atmospheric and electrical interference. Medium wave and short wave radio signals act differently during daytime and nighttime, during the day, AM signals travel by groundwave, diffracting around the curve of the earth over a distance up to a few hundred miles (or kilometers) from the signal transmitter. However, after sunset, changes in the ionosphere cause AM signals to travel by skywave, enabling AM radio stations to be heard much farther from their point of origin than is normal during the day. The SSIRI programs are broadcasted during the day and as such would loose on the advantage of skywave propagation. AM broadcasting now attracts mainly talk radio and news programming, while music radio and public radio mostly shifted to FM broadcasting. Radio WAU and MALAKAL though not very well equipped in terms of Studio setup and play out software have powerful transmitters and are able to transmit a strong and clear signal over their areas of coverage. More thought should be given to them as they present an alternative that would be used to meet EDC’s objectives in these areas. Further research should be carried out to assess and suggest some simple solutions that may be used to improve the setbacks. 10.0 CONCLUSION There is great advantage in using one broadcaster who has a wide coverage to broadcast the SSIRI programs. This is because it would be easier for EDC to manage the process and there will also be uniformity in the programs broadcasted in the various geographical locations. However the essence of this project is for the radio programs to reach teachers and students and when a single broadcaster can not be sourced or like in the case of 122 Miraya FM has unresolved issues that affect the achievement of such goals, alternatives should be sought. The Shortwave transmitters are far away and reception during the day suffers from fading. This might have been a good solution for yester years when there were no alternatives within Southern Sudan but at the moment there are numerous options available that deliver better results like the use of FM and AM stations within Sudan. There are also other FM stations that have been set up in the various regions that I found had a strong signal that could be considered. Consider the use of local (i.e. not via satellite) FM or AM stations, this is because there are less challenges when the studios and transmitters are in one geographic location as any problems would be quickly detected and resolved and there are no power losses associated with satellite transmission. The choice of such broadcasters should however be critically vetted to ensure that they are able to deliver and meet the objectives of the SSIRI project. When recommendations to both EDC and Miraya FM are implemented there should be a significant improvement to the quality of broadcast. The use of the Freeplay radio should be limited or used in areas near the transmitters until such a time when their electronic design can be improved to measure up to standard radio receivers. 123 ANNEX 8: FREEPLAY RESPONSE Freeplay Response Subject: FW: Lifeline radios Date: 12/3/2008 8:33:58 A.M. Eastern Standard Time From: ttilson@yahoo.com To:Iandrews@usaid.gov, Sleigh64@aol.com, amwamba@edc-ssiri.org, galesh@un.org, Bshanmuganathen@ndi.org, knoel@edc.org, mmelican@edc-ssiri.org, EWalla@usaid.gov Sent from the Internet (Details) From: John Hutchinson [mailto:jhutchinson@freeplayenergy.com] Sent: Wednesday, December 03, 2008 3:32 PM To: Tom Tilson (E-mail) Cc: Kristine Pearson (E-mail); Chhavi Sharma (E-mail) Subject: Lifeline radios Dear Tom I hope this finds you well. May I give you an update on the current Lifeline radio issues. • Difference in audio output between Lifelines. We received two Lifelines (one "good", one "bad" sample). The "good" sample was manufactured by Solar Electronics, the original manufacturer of Lifeline radios (Solar's factory closed several years ago). The "bad" sample was manufactured by City Electronics, who took over the manufacturing from Solar (we no longer use City Electronics as a manufacturer for any products. Lifeline is now made by our major supplier, Nicco Industries). On FM, there was a marginally noticeable audio output difference between the two radios. However, both were within specification. The "bad" radio was at the bottom end of the specification and the "good" radio was at the top end of the specification. On weak SW signals, this difference was much more marked both under laboratory measurement and listening to off-air signals. Our investigations showed that, although the circuitry was identical, there was a difference in the sensitivity at the SW front-end of the two radios (however, again within specification: one at the very top and one at the bottom). Due to the fact that we no longer use either of those factories, there is no action I can take, nor could I take, given that both were within the spec. As said above, we moved Lifeline manufacture to Nicco Industries in 2007, and they now manufacture the Lifeline radio. When we transferred manufacturing to Nicco, we completely redesigned the SW front-end of the Lifeline radio to make it more sensitive. Current Lifelines are substantially more sensitive than older generation radios and offer considerably better performance, especially with weaker signals. This means that reports of audio/performance differences in various versions of Lifeline radio will continue to arise, especially when new generation Lifeline radios are placed in the field next to the older model. • Miraya bleed issue. I have discussed this with our radio engineer, and given that we are not able to experience the problem personally from an off-air signal, we have done our best to interpret the cause of the problem. 124 Making the assumption that there is nothing wrong with the Miraya transmission, we can only conclude that the signal strength is so high that it completely overpowers the front-end of Lifeline, causing repeated duplications across the dial scale. This is not a conclusion that we are comfortable with, due to the fact that other radios do not experience the problem, and we do not believe that Lifeline is so super-sensitive when compared to other radios, that it suffers from the Miraya effect (however, this may be the case and we will find out in due course). In the laboratory, we have taken a Lifeline radio and radiated a very strong (10 000 times stronger than the sensitivity rating of the radio) FM signal and experienced the presence of the radiated signal across the FM band. This seems to be similar to the Miraya effect. We once had a report from a USA distributor to which we sold a specific radio, where there was an overpowered bleed across the dial scale from a specific transmitter. To correct this, we added a second stage of FM/IF filtering that attenuated the IF signal on each side of the carrier. We are in the process of testing a similar modification to the overpowered Lifeline that we have in the laboratory and we will let you know the result by the end of the week. Meantime, any further information that you have with respect to the Miraya effect on Lifelines and other radios would be useful, including any transmitter data that you can get hold have. I hope this helps. Please let me know if there any further questions I can answer prior to us giving you further Miraya effect feedback. Regards John Hutchinson Chief Technology Officer Freeplay Energy Ltd Tel : +2721 552 4872 Mobile : +2783 646 8080 Website : http://www freeplayenergy.com 125 ANNEX 9: LIST OF SCHOOLS / LEARNING GROUPS OFFERED FOR OBSERVATION Location / County Schools / Groups Chosen by Consultant Observed but with no choice Juba Learning Village (model): United Believers Basic School, Juba Model P/S, Libya Basic School, Sadaka Basic School, Atlabara Basic School Juba Model Primary School (P1, P2) Learning Village (non-model): Altabara, , St. Thereza P/S, Komuyangi P/S, Libya Basic Terbia B (model): St. Juveline ALP, Kator Bari, Gudele (not-active at the moment, facilitator away training) Kator Bari Terbia B (non-model):St. Monica ALP Maridi Learning Village (model): Kasana P/S, Adia P/S, Maridi Girls P/S, Mabrindi P/S. Haddow P/S (P1) Learning Village (non-model): Mabrindi P/S. Mabrindi (P2) is not an SSIRI model teacher/class Terbia B: New Sudan Women’s Federation, Maridi Girls Primary (ALP), Kanzara 1, Bakindo, Mabrindi Maridi Girls Primary (ALP) Kajo Keji Learning Village (model): Kajo Keji Town P/S, Beliak P/S, Tipere P/S, Loopo P/S, Wudu P/S Wudu P/S Learning Village (non-model): St. Daniel Comboni P/S, Logili P/S, Leikor P/S Leikor P/S Terbia B: Bibi ALP, Loopo ALP, Wudu ALP, Lire ALP, Mundari SPC Mundari Pentecostal Church ALP Terbia A: Kajo Keji Secondary School Kajo Keji Secondary School Yei Learning Village (model): Ronyi P/S, Payawa P/S, Longamere P/S, Lutaya P/S, Marakonye P/S Ronyi P/S, Learning Village (non-model): Kinji P/S, Immanuel Model P/S, Kanjoro P/S, Jigomoni P/S Kinji P/S Terbia B: Ronyi ALP, Yangkonye ALP, Kanjoro ALP, Jigomoni ALP, Kinji Adult School Kanjoro ALP Yambio Learning Village (model): Pazuo P/S, Basidi CGS, Kpirabe CGS, Nabaguu P/S Kpirabe CGS, Learning Village (non-model): Daystar P/S, Maingbangaru, Nakpangau Terbia B: Yambio town APL (name?) Yambio Town ALP Nzara Learning Village (model): Nzara 1 P/S, Nzara 2 P/S, Yabua P/S, Namaiku P/S Nzara 1 P/S Kauda Learning Village (model): Kauda Development Center P/S (KDA), Kauda Model P/S, Upper Kumo P/S, Lower Kumo P/S, Kudi A P/S KDA KDA P/S, Kudi P/S, Upper Kumo P/S Terbia B: Hillman Terbia B Center Hillman Center Wau Learning Village (model): Billfam Model P/S, Machakos P/S, Liberty P/S Billfam Model P/S, Machakos P/S (only option on last day) Learning Village (non-model): Bullick Basic School Terbia B: Windell Trust 1 Group, Machakos Jor River Learning Village (model): Bar Yar B P/S Learning Village (non-model): Zoglona Zoglona Terbia B: Bar Yar Bar Yar 126 ANNEX 10: IMPLEMENTATION OF THE SSIRI COMMUNICATION PLAN November 2008 Charlton Doki, Communication Specialist The current communication (promotional) plan was designed to boost support for and participation in IRI programs following the findings of the May USAID Mid-term Evaluation. To date, the following activities have been implemented. ACTIVITIES UNDERTAKEN Launching Events: The biggest promotional activities implemented so far have been the following launching events: IRI Launch in Yei: A successful launching event was held in Yei on August 11, 2008. The launch was attended by over 500 people including the County Commissioner; County Education Director; Deputy Director for AES at the MoEST in GOSS, Nelson Odur; Supervisor for AES in CES, Satimon al￾Haj; and education partners in Yei County as well as payam education officials. County education officials, NGOs, teachers from secondary and primary schools, pupils, students, NGOs, police as well as military officials also attended the event. The activities to mark the launch included the distribution of promotional materials (T-Shirts and flyers), demonstration of an IRI lesson, performances by pupils and students, songs by a local music group, and speeches by several officials regarding education in general and IRI in particular. IRI Launch in Kajo Keji: SSIRI was officially launched in Kajo Keji County on October 1, 2008 at a celebration with over 800 participants including the County Commissioner. The occasion was made colorful by traditional dance, songs from school children and secondary school students, all praising SSIRI as a good program. The County Education Director declared that SSIRI programs be included in the school timetable. Spot Announcements: Several spot announcements promoting IRI programs have been aired on various stations. The promos encourage pupils, teachers, facilitators and individual listeners to tune and listen to IRI programs. The promos also encourage teachers to get further details regarding SSIRI programs from their respective county education offices. These promos have been broadcast on Bakhita Radio in Juba, Miraya FM (which broadcasts to Juba, Wau, Malakal, Yei, Yambio, Maridi and Torit among other places), Spirit FM in Yei, Grace FM in Kajo Keji and Naath FM in Leer. To date the spot announcements are being aired on all the above stations. Whenever, there are any changes such as new broadcasting times, a new promo is recorded and sent to the specific radio station in order to keep listeners informed about the changes. Promotional Materials: Flyers: 127 Flyers have been produced and distributed in Juba, Yei, Leer and Yei. The flyers provide basic information about the SSIRI programs and information on the radio stations being used in specific locations, the stations’ frequencies as well as broadcast times of the programs. Now that arrangements to broadcast on stations in Torit, Wau and Malakal, we will produce and distribute flyers in these areas. Maridi, Yambio, Kumuk and Kauda will also receive flyers next year now that digital devices have been delivered to those locations are IRI programs are being implemented. T-Shirts: T-shirts bearing labels such as “TERBIA”, “Improve your English”, “Learning Village and TERBIA”, have been produced and distributed in Yei, Kajo Keji, Juba (Central Equatorial),Yambio, Nzara, Mundri, and Maridi (Western Equatoria), Kauda (Southern Kordofan), Malakal (Upper Nile), and Wau (Western Bahr el Ghazal). The T-shirts are distributed mostly to IRI teachers, facilitators, and County Education Officials to motivate them to support and participate in IRI programs. Unpaid Advertising: Radio: Outreach staff in Yei conducted unpaid advertising by participating in a talk show on Spirit FM. An education official from the County Education Office also participated in the talk show, which was aimed to promote IRI programs. During the IRI launch in Yei, the two FM stations - Liberty and Spirit - covered the event in their news briefs, which was another form of unpaid advertising. Grace FM in Kajo Keji was also willing to grant time for a free interview or talk show regarding SSIRI programs. Plans are underway to follow it up with the station’s management. It has not been possible to do unpaid advertising on other radio stations because they demand payment to conduct talk shows and interviews regarding SSIRI programs. Newspapers: A feature article on the Terbia program was written and submitted to newspapers and magazines in June this year. Press Releases: A press release was written and distributed to electronic and print media houses to announce the launch of Terbia broadcasts in May-June. At the moment no media house showed any interest in covering the story. Before the launch in Yei and Kajo Keji, only media houses in the two areas were invited and they attended the functions. In Yei, a TV journalist from Juba covered the event. It’s not clear if that video was broadcast on the Southern Sudan TV in Juba. Paid Advertising: SSRIR ran a paid advertisement in The Sudan Mirror newspaper in June to announce the launch of Terbia broadcasts in May and June. 128 However, SSIRI did not advertise its programs in the other newspapers operating in southern Sudan because they did not respond to our requests for information on advertising charges. In any case, some of the papers are read by only a few people and would not be a good medium through which SSIRI can reach its target audiences. Public Gatherings: SSIRI outreach staff have used public events to promote IRI programs. In Kajo Keji and Yei outreach staff used the Girl Child Education Day celebrations to promote SSIRI programs. In Juba, staff participated in celebrations to mark the World Literacy Day and used the occasion to promote SSIRI programs. Staff have also used the periodic education coordination meetings with MoEST officials to promote SSIRI programs, for example, by pushing for the incorporation of IRI programs in schools’ timetables and call for support and joint coordination in the implementation of SSIRI programs. ACTIVITIES NOT IMPLEMENTED IRI launching event in Yambio: The planned IRI launching event in Yambio was not under taken because we needed to the launch where IRI lessons broadcasts are heard on a local FM station. Up until recently, SSIRI was relying on shortwave to broadcast Terbia in Yambio, yet shortwave doesn’t work well. Now that Terbia groups can access IRI lessons on digital devices, we will organize a launching event there next year. IRI launching event in Torit: The promotional event in Torit has been postponed due to SSIRI’s financial position. The event will be organized as soon as funds are available Town Crier Announcements: The Town Crier Announcements was used to announce the launching events in Kajo Keji and Yei. Letters to NGOs (Partners): Letters to NGOs as a means of promoting SSIRI programs have not been written. However, SSIRI has hired an Outreach Advisor who will assume responsibility for expanding our work with partners in 2009. 129 ANNEX 11: SSIRI OUTREACH, M&E, AND IT STAFF DEADLINES FOR KEY ACTIVITIES 130 131 132 133 ANNEX 12: VIDEO CONSULTANT’S REPORT - END OF TOUR REPORT FOR 14 JUNE TO 6 JULY 2008 Training Activities in Maridi, Western Equatorial, South Sudan Richard M. Cavagnol Education Technology Advisor SITUATION Education Development Center, an innovator in Interactive Radio Instruction (ISI) was contracted by USAID to introduce and continue to build educational and training technologies to one or more curriculum development centers in South Sudan. Mr. Richard M. Cavagnol was hired as the Education Technology Advisor to carry out the tasks of the statement of work (SOW). This will involve him going to Maridi, the key development center in South Sudan to work with the curriculum developers, IT personnel, and the tutors. MISSION Richard’s primary mission was to proceed to Maridi to meet with the IT staff, tutors, and curriculum development staff. Maridi is the site of the National Teacher Training Institute (TTI) and the primary center in South Sudan for the development of teacher training and IRI learning materials development. These materials are used internally and distributed to other teacher training institutes in South Sudan. His task was to assess the current technological awareness of the key staff at the National Teacher Training Institute (TTI). The purpose of this assessment was to determine their readiness and capability to work with, and use, the technologies for the development of learning aids. Based on his audience assessment, Richard was to design a plan to provide hands-on training of key personnel on the technologies. The goal was to develop the skills in the teams to allow the key personnel to carry on the development work using these technologies after Richard had departed. Richard was tasked with introducing two educational technologies; video production improving internet skills of the personnel. He was to train key members of the staff on the digital video production of short educational videos designed for use in teacher training and for eventual use with students. This would be accomplished by having the “video production team” create an actual video. At the same time, Richard would work to improve the computer skills of tutors and curriculum developers to conduct internet searches for teacher resource materials, download the materials, and print them out for organization, storage and reference in binders. At completion of these two tasks, Richard would create a CD with acquired teacher resource material and the video created by the team so all curriculum development and IT personnel and the tutors could load the materials onto their computers for reference. EXECUTION 134 Equipment Assessment Richard observed that the Maridi TTI has a computer lab that was equipped with 8 HP desktop computers and a Dell Optiplex 745 that was used for video editing work. All of the computers were linked thru a wireless repeater to the VSAT satellite for internet access. Many of the school staff had desktop and laptop computers and when all were running, a number of computers in the lab would not boot. The school is powered by a single small generator that runs from 9 to 1 and from 2 to 5 Monday thru Friday. It is clearly underpowered and to do work on the Dell computer, all the other computers in the lab had to be shut down. While there, Richard encouraged Amia john, the senior IT person at the school, and Beneth Surur, a new member of the IT staff, to get with the head of the school and lay out plans for where they will put the new computer. They need a secure location centrally located and must build a tukul to protect it. Richard also suggested they create a wiring diagram of the school to determine how much wire is required to completely wire all of the key buildings. This will also give an estimate of the cost. Clearly, they cannot continue to limp along on the existing system given the increased power requirements. The Dell computer is equipped with Pinnacle Version 11 software that provides the capability for video and audio editing. The result is the creation of .MPG or .AVI videos that may be dropped onto CD and played in teacher training classes. What was lacking was audio recording and editing software such as SoundForge or Audacity. Amia John has developed a strong capability to use Pinnacle and Beneth Surur has started to work with the tool. Peter Bay, one of the tutors and the current science coordinator, has also begun to work with the tool. The school has two Sony digital video cameras and two Klieg lights for illuminating shooting subjects. The current cameras are more than adequate for the current video production requirements and are able to transfer video clips from the camera to the computer via USB 2.0 port. Richard recommends that EDC purchase two good microphones for use if voice recording while shooting is required. The curriculum developers are using desktop computers with an HP 4250 printer and an HP 2400 Scanner to create the draft learning materials. Personnel Readiness Assessment OBSERVATION: Richard met with the tutors in the Computer Lab to determine who was both interested and capable of producing, narrating, and editing videos. Five people raised their hands and this became the production team. Others wandered in and out of our meetings and discussions but there were five people who actually participated in the creation of the first video. ACTION: Provide a number of hands-on training sessions in script development, lighting, camera work, framing and lighting subjects, and editing techniques. Amia John is very proficient with the Pinnacle Video Editing software and has learned how to use a trial copy of SoundForge. Beneth Surur has started to work with Pinnacle and has experimented with several audio reorders that he has downloaded on his own from the web. Peter Bay has started to learn how to do basic editing on Pinnacle and has narrated the first video using SoundForge. When Amia John went on vacation and took 135 his computer with the only copy of SoundForge, he went to the internet, downloaded and tried several audio recording and editing software packages, and selected one for use. OBSERVATION: After discussing the instructional design capabilities of the curriculum development team with Kutiaboro James, Richard learned that the team was not well grounded in good instructional design techniques, the creation of effective learning objectives, and the ADDIE model of courseware development ACTION: It was decided that Richard would give a class on learning objectives and basic instructional design to the curriculum developers. All of the IT personnel, curriculum development people, and the tutors are computer literate from the perspective of internet access and the ability to find and download materials. A few of the curriculum developers had downloaded materials as ideas for their work creating the Learning Village, IRI materials and other learning items. ACTION: Richard would review materials on the web, select, and downloaded for future reference various lesson plans, activities, and curriculum development papers as well as a list of URLs that the curriculum development staff, teacher trainers, and teachers may se to access and download more materials. OBSERVATION: It appears that all of the personnel have some knowledge of Microsoft Word, but only one person out of a class of 15 raised her hand when Richard queried them about the experience working with PowerPoint and Excel. ACTION: Based on this response, Kutiaboro James and Richard decided that Richard would give a one-hour overview of PowerPoint to the curriculum developers. Video Production Activities Richard held several meetings that also doubled as training sessions to help prepare the video production team for pre-production activities. In this phase of video production, the team accomplished the following: • Reviewed several existing videos to observe and discuss good and bad points of video production • Listed several possible topics that are suitable for video production • Selected one of the listed topics (How to Properly Brush Your Teeth) and begin brainstorming and conceptualizing how to produce the topic • Developed critical learning points to be covered in the video • Established a rational for the video • Wrote learning or performance objectives for the video • Created a draft script with narration using the left-side right-side format for review and revision • Reviewed the script and commented to the team about what was good and what needed work • Created storyboards that depict the various scenes in the video 136 • Finalized the script and the storyboards, including proofreading and copy editing, in preparation for production Peter bay took the lead in writing the script which Richard commented and worked with him to include all of the key elements in both the left-side video shooting directions and in the content of the narration. Amia John acted as producer, assigning different jobs to different members of the team, including securing props. Once the script was completed and approved, the team engaged several locals as “actors’ to portray the critical activities of brushing your teeth. The team set up the props (toothbrush, toothpaste, cup, and bottle of water and shot about 30 seconds of video. Amia brought the video into Pinnacle to check the quality. He also superimposed text over the video to identify each of the critical items. During the production phase, the team accomplished the following: • Created a shot list to group shots at different locations • Collected and set up props • Met with the actors and had them read the script to learn their activities • Set the scenes • Shot the video Peter Bay took the camera over the weekend and shot the rest of the video scenes. He did a very good job of framing the subject, using close-ups when required, and provided proper lighting. During the post-production phase, the team accomplished the following: • Downloaded digital video from the camera to the production computer • Checked each video scene for production quality • Created the text screens • Pulled the text screens and video segments onto the editing timeline • Pulled the audio segments onto the timeline • Edit the length of the video to match the audio voice-over Amia John downloaded all of the video into Pinnacle and created a number of text screens for the Title, rationale, and objectives. Peter completed the narration and this was integrated with the video as the voice-over. The video was completed in time for the USAID audit. Amia and Beneth showed the video to the team and explained how it was produced. They were impressed. The team then listed several topics for a second video, this one of Farm Animals in the Maridi Area. It should be a useful topic both to continue to develop video production skills and to provide teachers in training with content other than print. When Richard left Maridi on July 5th, the video had been shot, the audio narration completed and Beneth was beginning to build the text slides and pull the entire production together. Richard 137 predicts that with continued support and encouragement, and getting the production team with the curriculum team, good videos can be produced at Maridi. Curriculum Development and Teacher Training Resources There was limited opportunity to interact with the curriculum development team. They appeared to be engaged in developing materials for the incoming class of teachers to be trained or in the revision of curriculum materials based on directives from the Ministry of Education Science and Technology (MoEST). Richard held one class on how to write learning objectives that included having the group create objectives for various topics and then identify the action, condition and standard. They said they found this very helpful. The ADDIE model was discussed as a process for creating learning materials with each of the five phases discussed and their activities for that phase described. I said that I would provide a list of teacher resources that would pull from the internet and I would provide them with papers and PowerPoint presentations I had either developed or acquired during my career. I said it would put the materials on CD and make them available to everyone. The team seemed pleased and anxious to receive the resource material. I made four CDs and gave one to Friday Oka, one to Beneth, and two to Kutiaboro James for distribution to the teams. Additional Training Activities Two other technologies were introduced to the IT team. They included a web-based training development tool called Lectora, still-frame capture program called SnagIt (www.techsmith.com) and a motion video capture program called Camtasia, also from TechSmith. While web-based education development using a tool such as Lectora, and the subsequent deployment of the web-based education lessons training to computer- and internet equipped schools is some years in the future, it was nevertheless thought to be worthwhile as a motivator and for local development experience. ACTION: Richard loaded Lectora on the computers of Amia John and Beneth Surur and left them with the CD and the learner guide. He also provided them with a one-hour lesson how to use the basic functions of Lectora. The two screen capture tools have immediate application in the creation of short software application videos that may be used for the primary instruction for Microsoft Office products, and as a help or review for refresher training of teachers. ACTION: Richard used Camtasia to create two movies on the basics of how to access PowerPoint, how to create a Master Slide, how to create a text frame, and how to import graphics and create a graphic frame. These are to serve as models for Amia John and Beneth, both of whom have downloaded trail copies of the software and have worked with Camtasia. The current SUMMARY AND RECOMMENDATIONS The time spent at Maridi was a worthwhile experience and it pointed out a number of areas where small activities can result in substantial improvements. 138 • The staff at Maridi should receive continual support and communication in the form of new ideas and challenges to keep them engaged • Purchase several microphones for digital audio recording • Purchase SoundForge audio recoding and editing software • Encourage the curriculum developers and tutors to search the internet for downloadable materials • Purchase the SnagIt/Camtasia Studio for the Maridi IT team and encourage them to create a library of short Microsoft application videos on how to use the various programs • Purchase a variety of laboratory and learning kits for use by the teacher trainers at the TTI to encourage teachers to develop their own training aids • Take these ideas to the Minister of Education, Science and Technology and have the ministry office disseminate these ideas around South Sudan • Place an education technology consultant with the Ministry of Education for a short period of time to present, discuss, and work out details on a technology implementation plan for schools.