July 2013
This publication was produced at the request of the Africa Bureau of the United States Agency for International 
Development. It was prepared independently by James Wile (Team Leader), Carol Benson, Richard Chediel, 
and Cathleen Sekwao under Task Order AID-OAA-TO-12-00054 awarded to International Business & 
Technical Consultants, Inc. (IBTCI). The views expressed in this report do not necessarily reflect the views of 
the United States Agency for International Development or the United States Government. 
EVALUATION REPORT
Evaluation of the USAID-Funded Textbooks and Learning Materials Program (TLMP) 
in Ethiopia, Malawi, Senegal, South Africa and Tanzania 
TLMP in Tanzania
Intentionally left blank
EVALUATION OF THE USAID-FUNDED TEXTBOOKS 
AND LEARNING MATERIALS PROGRAM (TLMP) IN 
ETHIOPIA, GHANA, MALAWI, SENEGAL, 
SOUTH AFRICA AND TANZANIA 
TLMP IN TANZANIA
Prepared by:
James Wile, Team Leader
Carol Benson
Richard Chediel
Cathleen Sekwao
July 19, 2013
The authors’ views expressed in this report do not necessarily reflect the views of the United States Agency 
for International Development or the United States Government.
Table of Contents 
ACKNOWLEDGEMENTS............................................................................................... iii
ACRONYMS...................................................................................................................... iv
EXECUTIVE SUMMARY.................................................................................................. v
Evaluation Purpose and Evaluation Questions....................................................................................... v
Project Background ..................................................................................................................................... v
Evaluation Questions, Design, Methods and Limitations.................................................................... v
Findings and Conclusions: .........................................................................................................................vi
Lessons Learned............................................................................................................... vii
Recommendations............................................................................................................ix
Materials Improvement ..............................................................................................................................ix
Systems Improvement................................................................................................................................ix
1. EVALUATION PURPOSE, QUESTIONS AND OBJECTIVES................................ 1
1.1 The Structure of TLMP........................................................................................................................ 1
1.2 Evaluation Objectives........................................................................................................................... 1
2. TLMP PROJECT BACKGROUND ............................................................................. 2
2.1 SCSU Responsibilities........................................................................................................................... 2
2.2 MOEVT and the Educational System in Mainland Tanzania......................................................... 4
3. EVALUATION QUESTIONS, METHODS & LIMITATIONS................................. 6
3.1 Research Conducted at SCSU............................................................................................................ 6
3.2 Field Research Activities...................................................................................................................... 6
3.3 Limitations of the Research ................................................................................................................ 7
4. FINDINGS AND CONCLUSIONS ............................................................................ 8
4.1 SCSU Achievements and Challenges................................................................................................ 8
4.2 TLMP Output......................................................................................................................................... 9
4.3 Project Management and Partnerships.......................................................................................... 10
4.4 Project Implementation .................................................................................................................... 10
4.4.1 Materials Development............................................................................................................. 10
4.4.2 Materials Distribution and Storage......................................................................................... 12
4.5 Assessment of TLMs by Various Stakeholders............................................................................ 14
4.6 Use of TLMs and Teacher Observations...................................................................................... 15
4.7 Textbook Review............................................................................................................................... 18
4.8 Conclusions......................................................................................................................................... 19
5. LESSONS LEARNED................................................................................................. 19
6. RECOMMENDATIONS..............................................................................................21
6.1 Materials Improvement..................................................................................................................... 21
6.2 Systems Improvement....................................................................................................................... 21
ANNEX A. SCOPE OF WORK ....................................................................................23
ANNEX B. QUESTIONS POSED OF SCSU PROJECT TEAM................................26
Evaluation of TLMP – TLMP in Tanzania ii
ANNEX C. SCHEDULE OF DATA COLLECTION ACTIVITIES ...........................28
ANNEX D. DATA COLLECTION INSTRUMENTS..................................................30
ANNEX E. MATHEMATICS EXAMPLE OF ADAPTATION PROCESS................41
ANNEX F. TEXTBOOK REVIEWS .............................................................................42
ANNEX G. CONFLICT OF INTEREST STATEMENTS...........................................43
Evaluation of TLMP – TLMP in Tanzania iii
ACKNOWLEDGEMENTS
The in-country team members would like to express our appreciation for the willingness of 
Ministry of Education and Vocational Training (MOEVT) officials to provide documents and 
information as well as ensure that we could access targeted schools, teachers, and students. 
The international members also note the extremely important contributions made by national 
team members Cathleen Sekwao and Richard Chediel, whose experience and contacts greatly 
facilitated our access to key informants and information. 
Evaluation of TLMP – TLMP in Tanzania iv
ACRONYMS
AEI African Education Initiative
AERA American Education Research Association
CA Cooperative Agreement
DEO District Education Office/Officer 
EGRA Early Grade Reading Assessment
EMAC Educational Materials Approval Committee
ESL English as a Second Language
IBTCI International Business & Technical Consultants, Inc. 
INSET In-service Teacher Training
MRALG Ministry of Regional Administration and Local Government
M&E Monitoring and Evaluation
MOE Ministry of Education
MOEVT Ministry of Education and Vocational Training
MSI Minority Serving Institution
NECTA National Examination Council of Tanzania
PAC Project Advisory Committee
PMP Performance Monitoring Plan
OUP Oxford University Press
REO Regional Education Office/Officer 
SCSU South Carolina State University
TIE Tanzania Institute of Education
TL Team Leader
TLM Textbooks and Learning Materials
TLMP Teaching and Learning Materials Program
TTC Teacher Training College
UM Uhuru Media
USAID United States Agency for International Development
USAID/T United States Agency for International Development/Tanzania
USAID/W United States Agency for International Development/Washington
Evaluation of TLMP – TLMP in Tanzania v
EXECUTIVE SUMMARY
Evaluation Purpose and Evaluation Questions
The purpose of this evaluation is to assess the impact of the Teaching and Learning Materials
Program (TLMP) as it was implemented in mainland Tanzania from 2009 to 2012. A research 
design was developed to explore the way textbooks and learning materials (TLM) were created, 
distributed and used in Tanzanian classrooms. The operational assumption was that the 
process of developing and disseminating quality teaching and learning materials for secondary 
school math and science would increase students’ access to texts, help to improve instruction
and lead to improved student learning outcomes. It was also hypothesized that this activity 
would build capacity of the contractor—South Carolina State University (SCSU)—as well as the 
Ministry of Education and Vocational Training (MOEVT) in Tanzania, local publishers/printers, 
and classroom teachers.
Project Background 
SCSU had facilitated the development of TLMs for secondary school math and science for 
Zanzibar from 2005 to 2008. The MOEVT determined that the TLMs should be adapted and 
revised for use on the Tanzanian mainland, where they were intended to be distributed to 
approximately 1500 government secondary schools. SCSU worked in collaboration with the 
MOEVT and its curriculum specialists at the Tanzanian Institute of Education (TIE), the 
Educational Materials Approvals Committee (EMAC), and local publishers to produce and 
distribute over 1.25 million textbooks and accompanying Teacher Guides to schools across 
mainland Tanzania, ultimately reducing the pupil-to-textbook ratio to 1:3 in some subject areas 
and 1:1 in some grade levels by the end of the project on December 31, 2012. 
 
Evaluation Questions, Design, Methods and Limitations
The evaluation was framed by a series of research questions. These include the following: 
• How well did the intervention achieve its program goals and expected impacts?
• What were the results achieved for each partner country in relation to intended program 
targets, as well as to standardized and variable indicators, by measuring quantitative and 
qualitative impacts of TLMP in terms of local capacity building (i.e. U.S.-based MSIs, in￾country institutions, the MOEVT, student achievement and teacher performance?
• To what extent were in-country institutions (with support from U.S.-based MSIs) able to 
deliver services effectively in terms of coordinating materials design, curriculum alignment, 
production, and distribution?
• What was the overall efficiency of allocated USAID funding in terms of usage and overall 
cost effectiveness?
• What were the specific program accomplishments/outcomes as a result of the partnership 
between SCSU and its counterparts in Tanzania?
• What lessons can be learned from this intervention program, and what recommendations 
can be made for potential program scale-up and/or replication as related to the new USAID 
Education Strategy?
The design of the evaluation study consisted of three main components: 1) an extensive review 
of documents including project formation, internal (SCSU) operational documents, formal and 
Evaluation of TLMP – TLMP in Tanzania vi
informal reports, and the TLMP products; 2) interviews conducted at SCSU; and 3) a two-week 
field study in which two teams of one international and one Tanzanian education expert
reviewed the processes, products, and outcomes of the project. The logic of this approach was 
that by triangulating these perspectives the data that would emerge would be both accurate and 
nuanced. 
The methods used to collect data combined structured interviews and structured performance 
observations. The evaluation team followed a comprehensive set of interview questions tailored 
to the key informants at SCSU, the education institutions and schools in Tanzania, publishers, 
and other stakeholders. The performance observations consisted of structured classroom 
observations of teachers using the TLMs and an informal reading assessment of students’ ability 
to read and understand the materials. Other data collection techniques included focus group 
discussions with classroom teachers and students. The team visited regional and district 
education offices as well as school storage rooms and libraries to see how books were stored 
and distributed. Additional data were collected through a rigorous review of documents 
pertaining to the inception, implementation and outcomes of the TLMP activity. 
As with any research study, this evaluation was confronted by several limitations. Chief among 
these was the challenge of meeting with key people involved in the project. Some had died, 
and some were unavailable because they had retired or been relocated, or were simply 
unavailable at the time of the study. In such cases, the team collected data from deputies, 
assistants, or individuals who were new in their positions and had no personal experience with 
TLMP. A second limitation was that much of the data relied on personal recall of meetings, 
workshops, and events that had taken place 12-24 months earlier. The lack of agenda records, 
meeting notes, and similar documents proved challenging when attempting to re-create the 
process in which TLMs were developed. A third limitation was the lack of any pre-intervention 
assessments or baseline data to compare/contrast against the final project outcomes. 
Classrooms could be observed to see whether teachers were using the TLMs appropriately, 
but except for teachers’ self-reports there was no way to compare the teaching/learning 
dynamic before and after the introduction of TLMs. Finally, as educators in Tanzania pointed 
out, it was too soon to be able to inquire about the impact of TLMP textbooks on student 
learning, because the materials had not been in use long enough to conclusively support such 
findings. 
Findings and Conclusions: 
The team found that TLMP on the mainland was generally successful in producing good quality 
textbooks that were relatively well aligned with the curricula for secondary math and science 
and that these TLMs met Tanzanian standards for cultural and political appropriateness. 
Although budget cuts eventually prevented SCSU from reaching the stated production target, in 
many cases the intervention dramatically reduced the pupil-to-textbook ratios at the recipient 
schools, and in some instances brought about a 1:1 pupil-to-textbook result.
The team also found that SCSU made good use of the educational systems in place in Tanzania. 
Relying on the MOEVT’s subject area and curriculum experts at TIE, along with the adoption of 
MOEVT’s procedures, the program was able to achieve its objective of producing 
approximately 1.25 million TLMs. This process was aided by engaging the expertise of local 
Evaluation of TLMP – TLMP in Tanzania vii
commercial publishers and printers, in this case Uhuru Media (UM) and Oxford University 
Press (OUP). However, USAID budget cuts in the last two years prevented the project from 
proceeding with its plans to put a textbook in every pupil’s hands.
Data and self-reports confirm that capacities were developed by all participants involved in the 
production of the TLMs, including SCSU faculty, MOEVT staff, and the staff at the commercial 
publishers. 
The team found that the TLMP textbooks and teacher guides were not having the degree of 
impact on teaching and learning desired. Although it may be too soon to conclude how 
teachers will eventually use these resources, the results of observations, interviews, and focus 
group discussions suggest that many secondary school teachers consistently use traditional 
lecture methods to transmit content information to students. No orientation or training was 
given to teachers with regard to using the TLMs. The research indicates that intensive and 
continuous professional development would be required to help teachers manage the paradigm 
shift from traditional teaching methods to critical thinking, problem-solving, and application 
(which TLMs appear well-designed to support). 
The team found that the USAID/W-based management strategy used for TLMP implementation 
was not the most efficient or effective. The centralization of management in Washington in 
tandem with the relative inexperience of SCSU in international development did not guarantee 
optimal outcomes. Cultural and geographic distances between Washington, Tanzania, and 
SCSU created insurmountable challenges. Inefficiencies in communication and monitoring led 
to misunderstandings and inefficiency, particularly in the case of materials development. There 
were missed opportunities for utilizing the flexible nature of the collaborative agreement 
structure to build new relationships, strengthen local capacity, and to promote mathematics 
and science knowledge on the part of authors and teachers. Had USAID/Tanzania (with its 
extensive knowledge of local context and its personal and professional relationships with 
potential partners) been bought into the planning and project design more significant outcomes 
could have been achieved.
Lessons Learned 
• Politics, culture, and international development: Politics is endemic to the entire 
process, and this had to be negotiated all along the way. One SCSU interviewee 
commented, “A full-fledged country study was required before we moved ahead on the 
proposal so as to be prepared for what we would see when we actually visited. In 
preparation for such a trip, we needed cross-cultural training so that we could interact with 
people locally with a degree of cultural sensitivity. Some training in language would also 
have been good.”
• SCSU Leadership: Instability in SCSU leadership over the life of a multi-year program 
created a context in which, as one informant responded, SCSU TLMP leadership was
constantly explaining the purpose of the project and the rationale for the college to support 
it. Without this stability, working with USAID was not fully institutionalized and the close 
working relationships with the MOEVT on the mainland were not established as they had 
been in Zanzibar.
Evaluation of TLMP – TLMP in Tanzania viii
• Confusion in Authorship and Copyright: TLM authorship was to have been a joint 
activity between designated individuals of the MOEVT/TIE and SCSU; however, the Project 
Coordinator’s name appears as the author on one of the textbooks, and the copyright was 
maintained by SCSU, making reproduction of the materials extremely difficult. 
• Operations: Having an on-site, in-country coordinator would have facilitated key 
relationships. In addition, according to one interviewee it “would have been better” to have 
created the original texts for the mainland and then move to Zanzibar. 
• Pedagogy: Teachers were not very prepared to teach their subjects at the assigned grade 
level using the TLMs as they called for a different approach to teaching. Students are now 
learning from 21st Century textbooks; however, science kits are needed, teacher evaluations 
of the use of the books are needed, and a plan for the local team needs to be developed to 
carry on with the process.
• Language: Teachers and students do not have the English competence to work in the 
level of English required by each of the textbooks. An English as a second language (ESL) 
approach may have been more effective rather than the assumption that students would be 
able to work in English at the same level of native speakers.
• The value of systems: TLMP in Tanzania made great use of the existing systems within 
the national education context. Rather than operating as a project outside the national 
system, the mainland portion of TLMP was managed and supported largely by well￾established Tanzanian systems for textbook revision, approval and distribution. This 
resulted in a cost savings and provided a measure of ownership for MOEVT including at the 
regional and district levels. 
• The value of professional publishers: TLMP benefited from the contributions of 
professional commercial publishers. The success of the project seems in large part because 
of the appropriate use of publishers’ local technical expertise. In fact, had the publishers’ 
standard practices for piloting and training been followed, the TLMs might have been better 
adapted to students’ needs, and teachers better prepared to use the TLMs.
• International publisher practices: OUP is a multinational corporation and as such can 
take advantage of resources available in other countries. OUP printed many of the TLMs in 
Kenya, where machinery, supplies, and electricity are more reliable. Thus, the Tanzanian 
printing industry, per se, did not benefit from TLMP.
• The limitations of textbooks as change agents: Although SCSU and the MOEVT may 
have felt that the level of innovation incorporated in the TLMs was modest most teachers 
observed seemed reluctant to change teaching styles. In this case, teachers were familiar 
with lecture and recall of terms and concepts, and used the TLMs to highlight facts rather 
than to engage students in a discussion of the implications and applications of factual 
information. 
• The limitations of centrally-managed international development programs:
USAID supervision from Washington was not the most efficient. The USAID/W office 
appeared to be too far removed from the SCSU and the MOEVT, despite the presence of 
the USAID office in Dar es Salaam. Critical oversight regarding budget, partnership 
development, and product review seemed to be lacking or poorly timed. 
Evaluation of TLMP – TLMP in Tanzania ix
Recommendations
Materials Improvement
• Address the lower levels of English language capability of secondary school 
students: Student difficulty in understanding the vocabulary and more complicated 
sentence structures used in the TLMs requires reconsideration of the level of English used 
in the textbooks. Future textbooks in any/all subjects must be solidly based on an English 
language assessment to guide the level of English competency required.
• Conduct a more comprehensive review of the usability of TLMs: Since teachers 
themselves have varying levels of subject matter knowledge and know-how in building 
competencies in their students, a more comprehensive assessment of their ability to 
understand and teach the range of concepts incorporated in the TLMs should be 
undertaken to fulfill two goals: 1) improve instruction at TTCs; and 2) improve classroom 
delivery of lessons. As noted above, textbooks don’t teach; teachers do.
Systems Improvement 
• Link TLMP to teacher professional development: To maximize use of the TLMs, an 
investment should be made to build on the MOEVT’s own initiatives to implement a 
professional development support program on teaching with textbooks.
• Address copyright issues: An investigation should be undertaken to determine who 
actually owns the copyright to the TLMs so that, if so desired, they can be reprinted.
• Support a large-scale evaluation of the impact of TLMs on student learning:
USAID might consider supporting MOEVT and Tanzanian research universities to conduct 
a national assessment of the impact of TLMP and other textbooks on teaching and learning. 
• Create links between teacher education institutions and schools: To maintain 
continuous professional development and to further professionalize the teaching 
profession, a link between SCSU science and math educators and secondary school 
teachers in Tanzania via shared technological resources, particularly through USAID￾sponsored Teacher Resource Centers in Tanzania, should be considered. This pairing might 
be extended to U.S. secondary schools with which SCSU has a relationship through student 
teaching and in-service programs. 
• Build school-based management of resources: USAID might consider developing 
support programs that build on the MOEVT’s school-based decision making and 
decentralization efforts. The formation of school-based textbook review committees with 
objective criteria for evaluating potential textbooks for adoption is one critical function. 
USAID’s support for secondary school directors and academic leaders might help the 
MOEVT to capture these exemplars and create leadership development models around 
them for future school-based decision making. 
• Develop a strategy to support secondary math and science: All TLMs should be 
provided to TTCs so that they might be included in the curriculum in training new 
teachers, thus providing input on how to teach using a 21st Century textbook. Without 
such an input it is likely that students will not elect to study the sciences at a higher level.
• Link to other secondary education programs: The TLMP experience – processes 
and outcomes – should be linked to other donor secondary school interventions.
Evaluation of TLMP – TLMP in Tanzania x
• Insure quality and accountability of textbooks at the school level: The same level 
of quality and accountability in the choice of textbooks in evidence at EMAC should be 
required at the school level. This will mean that the appropriate MOEVT divisions should 
provide training at schools (including directors and teachers) on how to make textbook 
choices.
• Systematize the allocation of textbooks to schools that need them: As part of 
the process of identifying the number of textbooks needed in a particular subject at a 
particular grade level, the MOEVT should determine whether the subject is taught at that 
school and at that grade level so as not to provide books to a school where they are not 
needed. 
• Share TLMP evaluation processes and results with early grade reading project 
awardees: The lessons learned by SCSU in developing the TLMs should be shared with 
early grade reading project leaders. Even though the subject matter is different, there are 
useful elements emerging from the project that can be learned by others (e.g., copyrights, 
publishing, textbook review, etc.) . 
USAID/W Improvement 
• Consistently apply performance monitoring procedures: USAID should require 
that all funded projects prepare a performance monitoring plan (PMP) and consistently 
conduct monitoring on project progress. In reviewing monitoring reports, included in 
quarterly reports, USAID should determine if the project is “on track” or whether 
approaches and activities need to be adjusted to obtain the desired results. 
• Build institutional capacity for less experienced US-based implementers: If 
USAID/W intends to earmark funds to expand capacities at U.S. institutions that do not 
have extensive experience in international development, it should anticipate the need for 
even greater oversight of their activities and ensure that such institutions are mentored so 
that capacity is gradually built over time. 
1. EVALUATION PURPOSE, QUESTIONS AND OBJECTIVES
1.1 The Structure of TLMP
The Textbooks and Learning Materials Program (TLMP) was launched by the United States 
Agency for International Development in Washington, DC (USAID/W), in 2005 as a part of the 
President’s African Education Initiative (AEI). It contributed directly to USAID’s effort to 
improve the management capacity of education sector personnel in a number of countries in 
Sub-Saharan Africa. TLMP was extended for another three years in 2008/09 to 2012. The 
overall aim of TLMP was to provide quality textbooks to African students. According to a
TLMP White Paper produced by USAID, well over 25 million children have gained access to the 
textbooks produced through the project, which consist of more than 500 titles in 13 languages. 
According to the Cooperative Agreements (CA) awarded to each MSI, the materials produced 
were to be fully aligned with national curricula, to focus on primary education, to be culturally 
relevant and to integrate important cross-cutting themes such as HIV/AIDS, gender sensitivity 
and equity, hygiene and youth leadership. Each Minority-Serving Institution (MSI) was 
responsible for managing and implementing the TLMP in a specific country and with achieving 
specific outputs and results in accordance with its respective CA (the University of Texas San 
Antonio had been awarded two CAs). 
Originally a program to develop TLMs for primary schools, initial needs assessments conducted 
by each MSI determined that in three countries MOE priorities were for middle and secondary 
school TLMs. South Carolina State University (SCSU) was chosen as the MSI to develop 
textbooks in secondary school math, biology, chemistry and physics in English in Zanzibar from 
2005 to 2008 and for mainland Tanzania from 2009 to 2012. This evaluation covers the TLMP 
implemented only in mainland Tanzania.
1.2 Evaluation Objectives
This performance evaluation was intended to satisfy the following objectives (see ANNEX A – 
SCOPE OF WORK): 
• Validate stated program goals and impacts;
• Assess the results achieved for each host partner country in relation to intended program 
targets measuring quantitative and qualitative impacts of TLMP in terms of local capacity 
building (i.e. U.S.-based MSIs, in-country institutions (ministries of education, etc.), student 
achievement, teacher performance, amongst other criteria, in each host partner country;
• Determine if in-country institutions (with support from U.S.-based MSIs) were able to 
deliver services effectively in terms of coordinating material design, alignment, production, 
and distribution;
• Highlight specific program accomplishments per MSI-host country partnership; and
• Document lessons learned and provide recommendations for potential program scale-up 
and/or replication as related to the New Agency (USAID) Education Strategy. 
Evaluation of TLMP – TLMP in Tanzania 2
2. TLMP PROJECT BACKGROUND
2.1 SCSU Responsibilities
SCSU was the recipient of two TLMP awards, the first addressing the need for secondary 
school textbooks in biology, chemistry, physics and mathematics in Zanzibar (2005-2008), and 
the second (2009-2012) adapting the Zanzibar materials for use on the Tanzanian mainland 
because of differences in culture, level of vocabulary/language, selected changes in the 
curriculum/syllabus, and incorporation of a competency-based approach. It was originally 
planned that during this period the expansion of materials for early childhood education in 
Zanzibar and in Mainland Tanzania would also take place, but USAID/W budget cuts prevented 
the production of early childhood TLMs. SCSU was the only MSI of the five to focus solely on 
the development of secondary school textbooks.
 
At SCSU, the team was comprised of a Program Director, a Program Coordinator and 
Assistant Coordinator, and subject matter specialists in language arts, biology, chemistry 
physics, math, and education assessment and evaluation. The team was supported by a fiscal 
analyst who worked with the different SCSU departments in training and making logistical 
arrangements.
Programmatic responsibilities of the leadership team included the following:
• To establish professional teams (at SCSU and in Tanzania) to guide and supervise the 
development and production of TLMs for mainland Tanzania.
• To select and supervise the professional development teams and other personnel that will 
develop and produce the TLMs.
• To engage, support and supervise relevant and appropriate program activities and other 
activities associated with the development, production and delivery of the TLMs.
• To accomplish the development and production of the TLMs so that the printing and 
distribution can take place within the timeframe.
• To be responsible for the quality assurance of the TLMs. 
• To obtain approval from MOEVT in Tanzania and USAID in Washington, DC for the 
printing of the TLMs for secondary schools in the United Republic of Tanzania. 
• To approve and supervise the printing of the TLMs developed.
• To monitor the delivery, storage and distribution of the printed TLMs. 
• To provide the appropriate and timely documentation, periodic reports, invoices, etc. 
associated with the development, production, printing, and delivery of the textbooks and 
other learning materials, as well as requesting timely reimbursement of expenditures of 
funds. 
The mainland adaptation program was a collaborative activity in which Tanzanian systems and 
procedures guided and directed the development, printing, quality control and dissemination of 
TLMs rather than the activities of SCSU. SCSU’s role was mainly to organize a local advisory 
committee, co-facilitate several adaptation workshops, and manage the program budget. 
Zanzibar (2005-2008) – We present the following achievements for the Zanzibar portion of
TLMP to illustrate that the bulk of the production work was accomplished during this period:
Evaluation of TLMP – TLMP in Tanzania 3
• Development, publishing and distribution of 1,140,000 books: 
• 10 secondary student textbooks (some covering two grades)
• 10 teachers’ guides
• 3 laboratory/practical manuals
• 20 titles of supplementary materials (posters and wall charts)
• Strengthening of US/Tanzania education development partnerships
• Achievement of a textbook-to-student ratio of 1:1 in Zanzibar 
• Training of 1,250 Zanzibari science and mathematics teachers 
• Awarding of Presidential scholarships at SCSU to 2 deserving Zanzibari students 
Project documents produced by SCSU assert that national test score averages in Pemba and 
Unguja (in Zanzibar) increased by over 30% since 2005, with TLMP materials a presumed factor.
Mainland Tanzania (2009-2012)
The MOEVT determined that the TLMs developed for Zanzibar could be of better use on the 
mainland if they could be adapted to conform to the competency-based curriculum that had 
recently been implemented in secondary schools, and reflected a more inclusive culture.
Adaptations were completed through a series of workshops that were co-facilitated by SCSU, 
the Tanzanian Institute of Education (TIE) and the Educational Materials Approval Committee 
(EMAC). Editorially-oriented workshops were facilitated by MOEVT/EMAC and the 
commercial publishers, Oxford University Press (OUP) and Macmillan/Uhuru Media (UM). The 
nature of these collaborations and the contexts in which they took place are detailed below. 
Evaluation of TLMP – TLMP in Tanzania 4
2.2 MOEVT and the Educational System in Mainland Tanzania
The formal educational structure on mainland Tanzania consists of two years of pre-primary 
education (KG1-2); seven years of primary school; four years of junior secondary (“O” levels); 
two years of senior secondary (“A” levels); and three or more years of tertiary education. 
Education is classified in three levels: basic, secondary and tertiary.
Two ministries manage and coordinate the education sector: the Ministry of Education and 
Vocational Training (MOEVT) and the Ministry of Regional Administration and Local 
Government (MRALG) (which manages basic education). At the primary and secondary levels 
quality assurance is the responsibility of school heads, Ward/District Education Offices, and 
School Inspectors. 
TIE, a quasi-independent education agency, was responsible for producing all instructional 
materials up until legislation in 1995 when its operating mandate changed. Prior to that time 
(and perhaps more recently), TIE developed detailed curricula based on the goals expressed in 
the national education improvement plan. These curricular goals now serve as content 
guidelines for commercial textbook publishers to develop all manner of textbooks. The 
MOEVT provides further publishing standards through the criteria developed by the EMAC. 
Tanzania has a robust and competitive education textbook market, particularly in secondary 
education, where English is the current language of instruction. International English-language 
publishers (OUP and UM) maintain sales representatives and offices in Tanzania and in 
neighboring Kenya. Both of these publishers were responsible for producing the TLMs for 
Zanzibar and mainland Tanzania.
The MOEVT has been engaged in a change process that includes four critical features:
• Move to Competency-Based Curriculum: In an effort to move instruction from a 
tradition of lecture, note-taking and content recall, the MOEVT, TIE and the National 
Examination Council of Tanzania (NECTA) organized a competency-based curriculum with 
performance objectives accompanying content. Students are expected to learn not only the 
content of science and mathematics, but also to understand how to apply subject-matter 
concepts to real-life situations. Student evaluation is based on performance criteria set by 
the curriculum. This dramatic shift in the teaching/learning dynamic has left many secondary 
school teachers in Tanzania unprepared, but the math and science departments at MOEVT 
are working on ways to help them, including developing a manual on how to use local 
materials for Chemistry and a DVD about using textbooks for teaching, both of which refer 
to the TLMs. Many of the existing textbooks in schools did not reflect this competency￾based curriculum; the TLMs were specifically designed to fill this need to support activity￾based opportunities in which students can demonstrate competencies. 
Evaluation of TLMP – TLMP in Tanzania 5
Teacher self-guides referring to TLMs DVD self-guide for teachers 
referring to TLMs
• Decentralization of Decision-Making: An MOEVT priority is the decentralization of
educational decision-making to the district and school levels. MOEVT no longer mandates 
the use of a specific textbook or instructional resource. Instead, the policy calls for school 
directors and classroom teachers to decide on a main and supplementary textbook(s) for 
each course at each grade level from among available materials (including TLMs and those 
developed by commercial publishers or by TIE). Schools are given budgets to buy their own 
materials, so the donated TLMs have been well appreciated in the designated schools. 
Secondary schools rarely have the funds to buy enough textbooks for all students, but 
TLMP tried to provide a sufficient number for each student to have one textbook each. For 
supplementary materials, some teachers use the TLMs, some use textbooks donated from 
the U.S. or other English-speaking countries, and some use materials they have created 
themselves. Others do not have any textbooks, relying on the lecture method to transmit 
key subject content. 
• Obtaining Funding for Materials Dissemination: The MOEVT does not have a 
budget for disseminating instructional materials to schools (via regional and district 
education offices) as this is now the responsibility of publishers (TLMP leadership assumed 
MOEVT would deliver what the publishers had produced). The geographic expanse of the 
mainland presents enormous logistical challenges, especially during the rainy season when 
school principals had to travel up to 100 kms to pick up their TLMs. 
• Commercial Professional Development: The MOEVT has limited financial capacity to 
provide in-service professional development programs for teachers (INSET). Typically, 
INSET is funded through external donors. Commercial publishers provide INSET on the 
use of the texts they have produced to those who purchase them. 
• No Education Portfolio at USAID/T: At the inception of TLMP USAID/T did not have 
an education portfolio or Education Director to coordinate potential linkages between 
other USAID-supported education activities and TLMP. Hence, the MOEVT did not have 
an opportunity to discuss TLMP and its ramifications in other educational development 
activities supported by USAID/T.
The MOEVT had prioritized the need to enhance services to nearly 2000 “new” government 
secondary schools (established since 2006). Many of these schools were operating with 
Evaluation of TLMP – TLMP in Tanzania 6
donated materials from the U.S. or Europe, outdated textbooks produced by TIE, or no 
textbooks at all. These needs provided the direction for TLMs. 
3. EVALUATION QUESTIONS, METHODS & LIMITATIONS 
3.1 Research Conducted at SCSU 
The research design included initial data collection on the program from the TLMP leadership 
team and faculty textbook developers at SCSU (see ANNEX B – QUESTIONS POSED 
OF SCSU LEADERSHIP TEAM). 
3.2 Field Research Activities
The evaluation team conducted site visits in the capital and three regions: Dar es Salaam
(Temeke District), Coast (Kibaha Town), Arusha (Arusha, Arusha City, Meru and Monduli 
Districts), and Morogoro (Morogoro Town and Rural Councils). These regions were specifically 
selected to represent urban, peri-urban, and rural, remote schools. Data were collected from 
MOEVT officials at the national, regional and district levels and at schools from directors, 
teachers, and students. Teachers were also observed using the TLMs in the classroom, and an 
informal assessment of student reading ability was undertaken. The following is a summary of 
research activities (for a full schedule of research activities, see ANNEX C – SCHEDULE 
OF RESEARCH ACTIVITIES):
• 18 secondary schools visited
• 22 school administrators interviewed 
• 19 classroom observations
• 33 students given an informal reading assessment
• 56 students participated in focus group discussions
• 49 teachers participated in structured focus group discussions
• 5 teachers/writers participated in structured focus group (Dar es Salaam)
• 3 Regional Education Officers (REO) interviewed
• 10 District Education Officers (DEO) interviewed
• 6 MOEVT officials interviewed
• 2 Tanzanian Institute for Education (TIE) (Curriculum) officials interviewed
• 3 USAID-Tanzania Education Officers interviewed
• 4 Publisher’s representatives interviewed
A final component called for a team review of the TLMs in terms of their structure, readability, 
and coverage of the cross-cutting issues (see APPENDIX D – DATA COLLECTION 
INSTRUMENTS). 
Evaluation of TLMP – TLMP in Tanzania 7
Strong room for 
safekeeping of TLMs
During the second week of field research, the team divided to ensure that each unit had one 
national expert to manage local contacts and language (Kiswahili) and to provide valuable 
cultural/historical/educational contexts where appropriate. The Monitoring and Evaluation
(M&E) expert from USAID accompanied the Morogoro unit, while the Arusha unit was
accompanied by a member of the Arusha DOE. Their roles were largely to observe.
In Dar es Salaam, the entire evaluation team met with the MOEVT and TIE representatives 
directly involved in the project, and with the senior representatives of OUP and UM, the two 
commercial publishers who produced the TLMs. 
Visiting a makeshift school 
library
Observing a chemistry 
lesson in Arusha
3.3 Limitations of the Research
A number of contextual factors presented certain limitations on data collection. These included:
• Changing personnel: Many MOEVT and TIE officials (as well as those in OUP and UM) 
who had participated in TLMP were not accessible as they had been replaced due to death, 
retirement, or change in sector. 
• Textbooks recently introduced in classrooms: Several TLMs had not been in the 
schools long enough for the team to collect meaningful data on impact on student learning 
outcomes; materials had not always been delivered to schools in a timely manner, although 
most deliveries had been completed by 2011. 
• Lack of baseline assessment: No baseline studies had been undertaken on the status of 
teaching and learning prior to the introduction of the TLMs. The team thus engaged 
informants in recollecting their perceptions of the materials when they were introduced and 
how these compared with their previous texts.
• Passage of time: Much of the information obtained from informants regarding the 
processes of developing, editing, reviewing/revising the TLMs was based on remembering
Evaluation of TLMP – TLMP in Tanzania 8
events that happened in 2009-2010, so it is understandable that some data could be missing 
from their recollections. 
• Logistical challenges: The Morogoro unit lost a day of travel due to their vehicle 
breakdown. While these meetings were important in providing key information, because
sub-teams had to visit the respective Regional Education Officer (REO), District Education 
Officer (DEO), and the school principal/head teacher before visiting the classrooms, the 
amount of time available for observations was limited. This was especially challenging 
because mathematics and science are normally taught during the first periods of the school 
day. 
• Time/Sampling limitations: In the time available, it was not possible for the evaluation 
team to make a comprehensive evaluation of the schools that received the TLMs. 
4. FINDINGS AND CONCLUSIONS 
4.1 SCSU Achievements and Challenges
SCSU has faced many institutional leadership challenges over the past several years. Despite 
internal setbacks, the TLMP management team moved ahead in addressing changes in the 
Zanzibari materials to make them more suitable for use on the mainland.
TLMP was challenged with ongoing communication problems related to social/cultural factors 
as well as technical misunderstandings with their Tanzanian partners. The inability of SCSU to 
have its own permanent in-country field manager contributed to the challenges it faced in 
program management. The program also suffered from changes in Tanzanian mainland 
counterparts. For example, the Director of the TIE, the Permanent Secretary of Education and 
Chair of the Project Advisory Committee (PAC), and the Director of EMAC were all replaced 
during TLM production. 
With the exception of a series of initial writer’s workshops co-facilitated with MOEVT and TIE, 
SCSU activities during the mainland phase appear to have been largely managerial and 
supervisory. Additionally, OUP and UM assumed greater responsibility for revising their 
respective textbooks.
Despite the administrative and budgetary challenges, the following were important 
achievements identified by SCSU: 
• International procurement: A new wire transfer policy was developed and 
implemented by the Controller’s office, including a Wire Transfer Request Form that 
enabled disbursement of funds to faculty and to pay invoices in a timely manner. The state 
procurement system was also revised so as to allow procurements in Tanzania and to 
exceed $5,000 without major university review. 
• Travel: Forms, advances and approvals times were simplified and shortened to
accommodate a quick turnaround.
• Internationalization plan: The current board chair has requested that the Director of 
TLMP develop a plan for enhanced international activities beyond Fulbright research grants
and a limited education abroad program currently funded under Title III. 
Evaluation of TLMP – TLMP in Tanzania 9
• Exposure to African culture: SCSU students have met Tanzanian students and learned 
from them, as has faculty. 
• Teaching and Learning: Faculty participants have incorporated different teaching and 
learning concerns in their classrooms and in their teaching practicums.
4.2 TLMP Output
The original goal was to produce 2.25 million textbooks for distribution to over 1,500 
government secondary schools in all regions of the mainland. USAID funding cuts resulted in a 
shortfall; the final publication tally was 1,560,400 according to MOEVT. Production numbers 
were reportedly affected by textbook configuration: the Zanzibar science materials combined 
Forms 1 and 2 into one book and Forms 3 and 4 into another, but the mainland science 
materials separated the Forms 3 and 4 textbooks, adding content according to the mainland 
curriculum. The creation of two titles from one should have raised the number of copies, but 
did not. Mainland production and distribution was as follows: 
Table 4.2.1 – Production and Distribution of TLMs
Title Form # copies Year Publisher
Mathematics 1 160,000 2010 UM
Mathematics 2 160,000 2010 UM
Mathematics 3 120,000 2011 UM
Mathematics 4 120,000 2011 UM
Biology 1&2 160,000 2010 OUP
Biology 3 70,000 2012 UM
Biology 4 70,000 2012 UM
Chemistry 1&2 160,000 2010 OUP
Chemistry 3 120,000 2011 OUP
Chemistry 4 120,000 2011 OUP
Physics 1&2 160,000 2010 OUP
Physics 3 120,000 2011 OUP
Physics 4 120,000 2011 OUP
Teacher Guide Math 1 8,000 2010 UM
Teacher Guide Math 2 8,000 2010 UM
Teacher Guide Math 3 6,000 2011 UM
Teacher Guide Math 4 6,000 2011 UM
Teacher Guide Biology 1&2 6,700 2010 OUP
Teacher Guide Biology 3 3,500 2012 UM
Teacher Guide Biology 4 3,500 2012 UM
Teacher Guide Physics 1&2 6,700 2010 OUP
Teacher Guide Physics 3 6,000 2011 OUP
Teacher Guide Physics 4 6,000 2011 OUP
Total 1,560,400
Evaluation of TLMP – TLMP in Tanzania 10
4.3 Project Management and Partnerships
TLMP was largely managed from SCSU. SCSU established a Program Advisory Committee 
(PAC) chaired by the Permanent Secretary of Education with membership including curriculum 
specialists from TIE and subject area teachers from universities and secondary schools. SCSU 
also established a relationship with the Director of EMAC who served as the local program 
coordinator, because revision of textbooks fell under the normal purview of that office. In this
way, local structures were respected and utilized effectively by TLMP. USAID/T assigned a 
person on the PAC to monitor progress.
 
Partnerships were established only with the publishers, OUP and UM, but several issues arose – 
one involving turnover and the other the ownership of the TLM copyright. Key figures at OUP 
had been replaced due to an alleged corruption scandal involving the World Bank,1 and the new 
Is this Managing Director had been in office only two weeks prior to our meeting. The 
publisher of the Zanzibari materials, Macmillan, had sold off the TLMP contract to a mainland 
partner, Uhuru Media. Ownership of the copyrights was in dispute (although the CA required 
that ownership be retained by the MOEVT or its designate) up until the last visit of the SCSU 
TLMP leadership team in December 2012. At that time OUP reported that the SCSU team 
demanded the entire set of digital textbook files and required OUP to sign over the rights that 
they previously had to re-publish the TLMs for sale. 
4.4 Project Implementation
4.4.1 Materials Development 
The two individuals most directly involved with the adaptation of the Zanzibari TLMs were the 
Director of Secondary Education Science and the Coordinator of EMAC. Their respective
teams estimated that 50% of Zanzibar TLMs required revision or replacement due to cultural 
references (e.g., depictions of mosques or Muslim clothing), curriculum alignment, editorial 
modifications, and errors.
Adaptation of the TLMs was largely led by the EMAC Coordinator who began with the 
organization of a Review Board to evaluate the Zanzibari materials. The Board consisted of 
subject matter specialists from the TIE and exemplary secondary school teachers from around 
the country. The Board used such review criteria as degree of alignment with the national 
curriculum, cultural and gender appropriateness, and paper and print quality. Board 
recommendations were submitted to EMAC. When all requests for revision/correction had 
been addressed and the materials were approved by EMAC, the TLMs were then sent to OUP 
and UM who conducted a final editing (with the assistance of subject matter specialists and 
editors) and then had the TLMs printed either in Tanzania (OUP) or Kenya (UM). For a review 
of the adaptation process followed for Basic Mathematics, Forms 3 and 4, see ANNEX E – 
MATHEMATICS EXAMPLE OF ADAPTATION PROCESS. 
1 See http://www.worldbank.org/en/news/press-release/2012/07/03/world-bank-sanctions-oxford-university-press￾corrupt-practices-impacting-education-projects-east-africa) for the World Bank’s discussion of the action it took
on debarment of OUP East Africa and OUP Tanzania.
 
Evaluation of TLMP – TLMP in Tanzania 11
Several authors/adapters felt that their contributions were not accepted by the publishers. For 
example, a mathematics specialist told the team that the context-appropriate suggestions he 
and his colleagues gave for the math book cover illustrations were not used, and there was no 
explanation from the publisher. They found the final design (an abstract head with numbers 
floating around in all directions) confusing and culturally inappropriate.
One constraint in revising the TLMs was time, which placed a burden on the reviewers, writers, 
and editors. For example, the publication of the Forms 3 and 4 biology books by UM was 
delayed until February 2013, after our research was concluded. SCSU and USAID officials were 
aware of this delay, and the MOEVT confirmed that the books were awaiting distribution. The 
problem was a combination of timing and funding and the need to add content when the two 
textbooks were separated into two texts. Additionally, the books had apparently arrived 
during examination time and both funds and time were lacking to distribute them. 
Subsequent to the revision of the TLMs, the Director of Secondary Education Science
supervised the development of a self-guided in-service DVD and manual for secondary school 
teachers on “teaching with textbooks.” While not specifically a training program in how to use 
the TLMs, these self-study resources are intended to support the transition from content-based 
instruction (lecture and memorization) to competency-based teaching—and they depict the 
TLMs among other available textbooks. These self-study professional development materials 
were waiting to be distributed to schools at the time of this evaluation. 
There were both positive and negative aspects in the development of the TLMs. According to 
representatives of UM, an important outcome of this process was the capacity development of 
its cadre of textbook writers and editors. UM also mentioned the positive impact the project 
had on enhancing their capacity to deliver high-quality products on a tight time schedule, which 
skill will be applied to future textbook production activities. 
Informants interviewed could not easily describe the role SCSU technical experts played in the
revision and adaptation process. One Tanzanian mathematics specialist pointed to a (U.S.) 
name in the credits and said he never had contact with the man. There is no evidence of 
analytical input by SCSU to ensure achievement of program objectives; rather, the process 
followed parameters established by EMAC. Moreover, according to informants from both
publishing companies, the piloting of new textbooks and orientation/training teachers to the
new materials were never conducted due to budgetary restrictions. One workshop co￾facilitated by EMAC and TIE (with representatives from SCSU present) was provided to 
teachers, but when the team asked for the curriculum, it was not provided. 
USAID/W funding was cut in the last two years of TLMP, resulting in a shortfall in the total 
printing of TLMs. This meant the project goal of providing one book for every student was 
compromised. The result was that USAID/T was seen as over-promising and under-delivering. 
Public assurances were made by the Mission Director to MOEVT, but the relationship between 
USAID/T and MOEVT has been strained. 
Evaluation of TLMP – TLMP in Tanzania 12
Copyright page Back cover
The most critical finding pertaining to the development of the TLMs is the issue of copyright
ownership and intellectual property. That SCSU and the MOEVT both claim copyright 
ownership is a matter of deep concern as without local ownership of the copyright, the 
materials cannot be reprinted. Ownership was apparently an issue beginning with the 
production of the Zanzibar TLMs, as the name of the Project Coordinator appears on some of 
the books as if he were the author. TLMs published by OUP for Zanzibar cannot be reprinted 
for the mainland until copyright ownership is clarified. This unfortunately means that some of 
the (inappropriate) Zanzibari books are being sold in the mainland market. UM did not 
experience these difficulties and to their knowledge they have the right to re-publish and sell 
any TLMs after 2012. It is not clear whether the mainland versions of the textbooks should be 
considered new books or edits of the Zanzibar publications, a further complication. We left 
USAID/T with this information so that the legal issues could be clarified. 
4.4.2 Materials Distribution and Storage
The Director of Secondary Education Science also managed the TLM distribution. Upon 
receiving the printed books from the publishers, the different titles were broken into pre￾assigned 80 unit blocks and repackaged for specific schools. These schools were targeted 
because they were “new” schools, i.e. started since 2006, most of them relatively rural and 
remote. A few “old” schools were added to the list because they had not received material 
assistance through other programs. The TLMs were distributed to REOs, then -sorted and sent 
to DEOs, and finally retrieved by school principals. 
TLMs were distributed in the following manner:
• Notification was sent to the MOEVT and the Ministry of Regional Administration and Local 
Government (MRALG) in the Prime Minister’s office to alert them that the TLMs were 
printed. 
• The MRALG contacted the Regional Administrative Secretaries to say that the TLMs would 
be delivered to the MOEVT warehouse.
Evaluation of TLMP – TLMP in Tanzania 13
• The REO at the Regional Administrative Office informed the target secondary schools 
through their principals to expect a specific number of science and math textbooks and 
teacher’s guides for forms 1-4 on a specific delivery schedule. Targeted schools included 
community schools registered in 2007-2010; all secondary schools designated by MOEVT as 
training centers for science and mathematics teachers; old established secondary schools 
with instruction ending at “O” level; and selected community schools registered in 2005/06 
that had not received teaching materials from MOEVT.
• The number of books distributed to schools was 160 copies per title for Forms 1 and 2 and 
80 copies per title for Forms 3 and 4. (For biology, 1,500 schools received the books at an 
average of at least 46 copies per school). (This uniform distribution did not take into 
account the specific needs of individual schools or the number of students studying specific 
subjects in the upper forms, where sciences were elective. This meant that some TLMs 
occasionally ended up where there was little or no demand for them. There did not seem 
to be any mechanism that would have enabled DEOs or principals to take initiative to re￾distribute the TLMs according to actual needs.) 
According to MOEVT officials, delivery of the final Form 3 and 4 biology books has been 
delayed because DEOs and school principals are waiting for the next quarterly tranche of 
budget funds to be released by MOEVT. Such TLMs are unlikely to be used until the 2013-14 
academic year.
There appeared to be a good accountability system at each distribution point as the numbers of 
books delivered to each school – and to each point along the channel - was documented. 
SCSU did not appear to have a mechanism for monitoring this distribution. When USAID/T
conducted spot monitoring in Morogoro, cartons of TLMs were found in the DEO’s office that 
had been there for at least three months since their receipt from the REO. In other cases the 
team found TLMs that had been distributed to schools but not to teachers or students. Overall 
the situation should have been more carefully monitored. 
At the school level, the team saw a range of strategies for distributing or maintaining the TLMs. 
In most cases TLMs were logged into the school system and individual copies were labeled with 
a control number. Where there were enough textbooks to assign one copy per student, these 
were recorded. Most schools had a policy that required students to pay a replacement fine for 
any loss of books assigned to them. When students get the books for the whole term or 
school year, they are required to sign for them and cover them with protective book jackets. 
When there was an insufficient supply of textbooks to provide each student a book, they were 
kept in the library, a storage room, or were given to the subject head, who disbursed the books 
when teachers and/or students signed them out. In these cases, schools typically maintained 
sign-out logs. 
 
Evaluation of TLMP – TLMP in Tanzania 14
We saw many lists like this 
indicating delivery
EMAC approval page and 
book no. 54/78
Library borrowing record 
at Kipok Girls’
TLMs were not distributed to students at several schools. The explanation given was that the 
school did not want to risk loss or damage of the materials. These materials were often kept in 
secure storage rooms, but it seemed unlikely that students would be able to access them. 
4.5 Assessment of TLMs by Various Stakeholders
The TLMs were generally felt to be useful and of high quality at all levels, including USAID/T. 
The publishers pointed out that the quality of the paper, ink, printing and binding make the 
TLMs attractive and durable and provide good value for cost. They appeared to be durable, 
even when (over)used or apparently soaked through.
 
Unused math books, Ilboro 
Boys’ SS
A very used math book Used biology books, 
some covered
Studying at the library, 
Toangoma SS
The most important observation was that the TLMs were adapted to be consistent with the 
2005 competency-based curriculum currently in effect. Teachers like that the TLMs are 
sequenced, i.e. written to build on information that has previously been introduced so that 
content becomes gradually more complex. Teachers feel that TLMs are of good quality and 
comparable to the materials used in private schools. Members of the Education Team at 
USAID/T had a similar opinion, and Peace Corps Volunteers using the TLMs reportedly had 
high praise for the books. 
Evaluation of TLMP – TLMP in Tanzania 15
Students reported they liked the language in the texts, the photographs and illustrations, along 
with the chapter summaries, key vocabulary, exercises and activities. Students found that the 
activities and some examples reflected their daily lives. They also reported the format enabled 
them to use the TLMs for self-study, which is important given high teacher absenteeism, 
particularly in rural areas. Students commented on the list-like nature of the presentation of 
content. This made the identification of key information easier, especially for second-language 
users. Students generally perceived the quality of the textbooks produced by TLMP to be high. 
They commented that the color illustrations and use of graphics was an improvement over the 
older textbooks from TIE.
Some teachers perceived that the TLMs were not uniform in quality in each of the subjects. 
Some preferred the materials developed by TIE, whether or not their departments had done a 
comparison. Some math teachers perceived the books as providing too shallow a treatment of 
some content areas, but an expert math teacher told us that those teachers probably do not 
know the books well because the sequential approach means delving more deeply into some 
topics later in the books. Teachers who lacked understanding of the competency-based 
curriculum were more comfortable with the older textbooks, with which they themselves had 
learned and which fit the thematic content of the previous curriculum. Others wished for 
designated sections in the TLMs on performance evaluations, which are required for a 
competency-based curriculum. Both authors and teachers mentioned that some small errors 
should be corrected if the books were to be reprinted. 
Some teachers commented positively on the structures used to present information. The 
textbooks highlighted key terms and concepts. Teachers frequently commented that even if 
they did not use the textbooks in their classes, the TLMs helped them develop their lesson 
plans. Few teachers seemed to be aware of the planning resources available in the teacher’s
guides accompanying each textbook, and indeed, not all were aware the teacher’s guides 
existed—until colleagues went and found them. 
School principals and head teachers were generally positive in their perceptions of the quality of 
the textbooks. Typically, head teachers had reviewed the textbooks informally and made the 
decision to use them, either as supplementary texts or (more often) as main texts. All 
informants expressed the opinion that the materials were consistent with the subject area 
curriculum. A minority of head teachers seemed to be aware that the textbooks were meant
to support the competency-based approach being implemented nationwide. Informants felt 
that using the textbooks would lead to improved student learning by virtue of color graphics, 
clarity of information, organization, and features that support student learning. 
Some principals and head teachers felt that increasing the number of textbooks in the 
classroom would have a positive impact on student learning. However, our observations 
indicated that simply putting textbooks in learners’ hands was only one part of the puzzle; 
teachers often copied text on the blackboard, even with the books in front of the pupils. 
4.6 Use of TLMs and Teacher Observations
4.6.1 Teacher Observations
Evaluation of TLMP – TLMP in Tanzania 16
The evaluation team observed over 60 teachers at 18 secondary schools. The use of the TLMs
in these settings varied considerably. Approximately 15 percent of teachers observed appeared 
to use the TLMs as they were intended, i.e., they were the main instructional resource used 
during the lesson, teachers generally followed the teacher’s guide, and the teacher used the text 
to guide student reading during class. 
About 40 percent of the teachers observed used the TLMs as supplementary resources. This 
was also reinforced by comments from focus groups. For example, in one math classroom a 
teacher was observed assigning a particular math concept and then directing students to draw 
on resources in the school library (textbooks) for sample exercises. Teachers also assigned the 
TLMs as resources for independent study and especially for the use of graphs, diagrams, and 
charts. The efficacy of this approach seems questionable as the texts appeared to be generally 
beyond most students’ independent reading level. Interestingly, nearly all of the teachers who 
said they used the TLMs as a resource commented that they also used them as a resource for 
themselves. The textbooks helped identify key concepts, vocabulary and high value content. 
However, none of the teachers in this group appeared to use the TLMP resources as a guide to 
innovative pedagogy. 
Approximately 25 percent of teachers observed did not use the TLMs in any manner. Some 
used texts previously published by TIE. These teachers seemed to use a traditional lecture 
mode in which students copied information from the chalkboard.
Approximately 20 percent of teachers observed appeared to use the materials in an 
inappropriate manner. For example, teachers were observed lecturing students using the 
content of the text (which the students had in front of them) or teachers reproduced diagrams 
and charts that were already produced in the students’ texts.
The most significant observation on the use of TLMs is that teachers seemed to be unable to 
use these resources to stimulate critical thinking, scientific reasoning or opinion-formation. 
Instead all teachers observed, whether they used the TLMs or not, focused instruction on 
information that could be easily listed, memorized and repeated. 
The MOEVT is generating a professional development program that focuses on the topic of 
teaching with textbooks as teachers appear to be unable to incorporate instructional materials 
into their pedagogical repertoire. It may actually be the case that because textbooks convey 
information, the traditional role of the classroom teacher is becoming redundant. Teachers 
clearly need re-training on how to build on text-based information.
The variation in the use of the textbooks may be attributable to several factors, none of which 
appeared to be related to the essential quality of the textbooks: 
• Textbook policy: The current decentralized policy empowers the leadership of each 
school to select commercially-published texts from an MOEVT/TIE-approved list, i.e.
materials that have passed the EMAC approval process. Whereas materials from TLMP 
were once the only books in schools, and indeed still are in some rural areas, now varied
instructional resources are available. In addition, school leadership is encouraged to use 
Evaluation of TLMP – TLMP in Tanzania 17
more than one resource for content information. Some books are used textbooks donated 
from English-speaking countries; others were published by TIE with copyrights from 1992. 
Other secondary science texts produced with USAID funding were also found in the 
schools. It is not at all clear why USAID/W provided the funding for yet another set of 
texts, essentially duplicating what USAID/T had already done in the recent past.
• Competition from commercial publishers: Most schools had a mix of commercial 
materials either purchased or donated. Some of these commercially prepared materials 
were nearly 20 years old while others were apparently quite new (TIE had engaged in a 
partnership with Pearson Publishing in 2009 for the production of science textbooks). 
• Teacher understanding of TLMs: In most cases, it appeared that the TLMs arrived in 
schools without any formal review process. Some teachers were unaware that the TLMs
reflected the new national competency-based curriculum in contrast to the older TIE 
materials. Others were unaware that there were teacher’s guides to accompany each TLM, 
although they were in school libraries or stores. Some principals commented that teachers 
were more likely to use the materials with which they were familiar. 
• TLMs as a resource: Teachers reported using the TLMs to prepare their lessons, 
especially to identify key terms and learning objectives. Others reported using the activities 
or exercises with their students. Both teachers and students said that the science books 
were useful for self-study during study time, when doing homework, or for supporting 
learning when teachers are absent. In many cases, students used TLMs on loan in the 
school library. Most schools had well-functioning check-out systems from libraries, subject 
department heads, or their teachers (who had signed them out from their departments).
• Use of traditional teaching methods: When asked if the TLMs had changed their 
teaching methods, teachers often said yes. However, according to observations in 62 
classrooms, teachers still appeared to rely heavily on traditional teaching methods of 
lecturing, note-taking and literal recall. Students are also instructed to copy content from 
the chalkboard, even when the same information was in the TLMs. One reason for this 
practice seems to be the strong tradition of fact learning and recitation, reinforced by an 
assessment tradition that rewards recall of factual information in flawless English (which is 
nearly impossible, as English is a foreign language for most learners). Related to this is the 
difficulty of comprehending complex technical content and critical analysis in a foreign
language, so that remembering key terms and lists of attributes already taxes the linguistic 
proficiency of teachers as well as students. To meet their needs, not only would the 
materials need to be written at an appropriate level of English, but approaches to teaching 
and learning English across the curriculum would need to be built in, and strategies 
developed to support both linguistic and cognitive development. The point is that 
textbooks alone cannot transform the teaching/learning dynamic. 
• Lack of equipment: Officials at the Secondary Education Department at MOEVT 
indicated that it would be extremely useful to have adequate instructional materials and 
laboratory equipment (e.g., microscopes, voltmeters or oscilloscopes) referred to in the 
texts. Without appropriate equipment, teachers omit activities and focus on memorization
at the expense of developing scientific reasoning, hypothesis, observation, and analytical 
thinking. 
• Lack of students/Lack of teachers: It is not uncommon in Tanzania for there to be no 
students studying a particular subject area at a given time, especially in forms 3 and 4 
science. One school official commented that high-achieving students in his remote school 
Evaluation of TLMP – TLMP in Tanzania 18
typically transfer out to other schools after their second year, leaving a reduced demand for 
higher level textbooks. In such cases, textbooks are stored until such time as a class of 
students is formed. In addition, secondary schools have difficulty recruiting and retaining 
qualified instructors for upper-level sciences and mathematics, especially in remote and 
rural schools. In this case textbooks are unused and stored until appropriate teachers are 
hired. 
Even though they did not always use the materials to promote independent reading for 
knowledge, teachers still commented that they wished for a 1:1 pupil-to-textbook ratio. We 
observed teachers managing this situation by having students read in pairs or assigning group 
work in which students used the textbooks as resources. We observed one math teacher who 
used the textbooks as a resource along with all of the other textbook series in the school 
library. After presenting students with a particular math concept, he would assign students to 
identify similar math problems in the math resource books, solve these problems and present 
their solutions on the chalkboard. 
In general we observed very little instruction that corresponded to the suggestions in the 
accompanying teacher’s guides. In only one biology classroom observed did the teacher have
students go outside to collect examples of living and non-living things and sort these according 
to the definitions in the textbook. 
4.6.2 Reading Assessment
One sub-team (including a Kiswahili speaker) did informal assessments of student reading. 
These were intended not as a measure of student reading performance but to determine
whether the TLMs were written at students’ instructional level (meaning the ability to identify 
90-95 percent of the words and comprehend 75-90 percent of the material). Students in forms 
1-4 participated in individual informal assessments of reading using sample passages selected at 
random from their TLMs in a location outside the classroom. The results of these sample 
assessments gave cause for concern. Although most students were able to decode words with 
a high level of accuracy, they generally had a great deal of difficulty comprehending what they 
read. Students were unable to respond to specific prompts for literal or inferential thinking. 
Even when prompts or directions were provided in Kiswahili, students remained unable to 
respond. These findings are particularly troubling given that many teachers and students saw 
these TLMs as useful outside the classroom, in other words to be used without scaffolding or 
any other form of support. As these materials did not seem to be on students’ instructional 
level (and certainly not their independent reading level), it is unlikely that students would gain 
much information without considerable reading comprehension support provided by the 
teacher (such as pre-reading, guided reading, vocabulary development, etc.). 
4.7 Textbook Review
The TLMs (textbooks and teacher guides) were reviewed according to standards for reader￾friendly instructional texts. This analysis has to do with specific elements that make instructional 
texts easy or difficult for students to use as learning materials. These elements include:
• Sentence length
• Passage organization and sentence cohesion
Evaluation of TLMP – TLMP in Tanzania 19
• Use of headings and labels
• Pre-reading and post-reading activities
A more detailed analysis of the TLMs appears in ANNEX F – TEXTBOOK REVIEW. 
4.8 Conclusions
TLMP in mainland Tanzania created a significant quantity of instructional materials for math and 
science for resource-poor secondary schools and clearly helped reduce the pupil-text ratio. 
The books were published with high quality materials. However, several troubling conclusions
emerged from this research. 
• Limited capacity at SCSU: One of the main program objectives was to build capacity at 
the MSI. Despite its experiences in Zanzibar, SCSU demonstrated that it still has limited 
capacity to manage and implement a large-scale development project. Both SCSU and the 
Tanzanian educational system would have been better served if the MSI had partnered with 
an NGO or agency with experience in educational development, so that SCSU capacity 
would have been raised. 
• No innovations or insights applied: Very few individuals from SCSU were involved and 
no visibly new approaches, activities, or strategies were introduced. 
• Redundancy in textbook support: Although Tanzania has a great need for textbooks
nationally, the need is not uniform. A cost-effective approach would demand a more 
nuanced understanding of the education context that goes far beyond simple textbook 
production to distributing them to the neediest areas and following up on their use. 
• Textbooks can’t teach: TLMP intended to improve learning outcomes for secondary 
school students in Tanzania. However, even well--designed instructional materials do not 
guarantee effective teaching nor do they automatically lead to improved learning outcomes.
Textbooks do not teach; well-prepared professional teachers do. Along with providing 
materials, TLMP should have worked with teachers to help them understand the demands 
of competency-based teaching and learning—as well as teaching a second/foreign language 
across the curriculum. At the very least, there should have been an orientation to the 
teacher’s guides. 
5. LESSONS LEARNED
• Staff politics, culture, and international development: Politics is endemic to the 
entire process, and this had to be negotiated all along the way. One SCSU interviewee 
commented, “A full-fledged country study was required before we moved ahead on the 
proposal so as to be prepared for what we would see when we actually visited. In 
preparation for such a trip, we needed cross-cultural training so that we could interact with 
people locally with a degree of cultural sensitivity. Some training in language would also 
have been good.”
• SCSU Leadership: Instability in SCSU leadership over the life of a multi-year program 
created a context in which, as one informant responded, SCSU TLMP leadership was
constantly explaining the purpose of the project and the rationale for the college to support 
it. Without this stability, working with USAID was not fully institutionalized and the close 
Evaluation of TLMP – TLMP in Tanzania 20
working relationships with the MOEVT on the mainland were not established as they had 
been in Zanzibar.
• Confusion in Authorship and Copyright: TLM authorship was to have been a joint 
activity between designated individuals of the MOEVT/TIE and SCSU; however, the Project 
Coordinator’s name appears as the author on one of the textbooks, and the copyright was 
maintained by SCSU, making reproduction of the materials extremely difficult. 
• Operations: Having an on-site, in-country coordinator would have facilitated key 
relationships. In addition, according to one interviewee it “would have been better” to have 
created the original texts for the mainland and then move to Zanzibar. 
• Pedagogy: Teachers were not very prepared to teach their subjects at the assigned grade 
level using the TLMs as they called for a different approach to teaching. Students are now 
learning from 21st Century textbooks; however, science kits are needed, teacher evaluations 
of the use of the books are needed, and a plan for the local team needs to be developed to 
carry on with the process.
• Language: Teachers and students do not have the English competence to work in the 
level of English required by each of the textbooks. An ESL approach may have been more 
effective rather than the assumption that students would be able to work in English at the 
same level of native speakers.
• The value of systems: TLMP in Tanzania made great use of the existing systems within 
the national education context. Rather than operating as a project outside the national 
system, the mainland portion of TLMP was managed and supported largely by well￾established Tanzanian systems for textbook revision, approval and distribution. This 
resulted in a cost savings and provided a measure of ownership for MOEVT including at the 
regional and district levels. 
• The value of professional publishers: TLMP benefited from the contributions of 
professional commercial publishers. The success of the project seems in large part because 
of the appropriate use of publishers’ local technical expertise. In fact, had the publishers’ 
standard practices for piloting and training been followed, the TLMs might have been better 
adapted to students’ needs, and teachers better prepared to use the TLMs.
• International publisher practices: OUP is a multinational corporation and as such can 
take advantage of resources available in other countries. OUP printed many of the TLMs in 
Kenya, where machinery, supplies, and electricity are more reliable. Thus, the Tanzanian 
printing industry, per se, did not benefit from TLMP.
• The limitations of textbooks as change agents: Although SCSU and the MOEVT may 
have felt that the level of innovation incorporated in the TLMs was modest most teachers 
observed seemed reluctant to change teaching styles. In this case, teachers were familiar 
with lecture and recall of terms and concepts, and used the TLMs to highlight facts rather 
than to engage students in a discussion of the implications and applications of factual 
information. 
• The limitations of centrally-managed international development programs:
USAID supervision from Washington was not the most efficient. The USAID/W office 
appeared to be too far removed from the SCSU and the MOEVT, despite the presence of 
USAID in Dar es Salaam. Critical oversight regarding budget, partnership development, and 
product review seemed to be lacking or poorly timed. 
Evaluation of TLMP – TLMP in Tanzania 21
6. RECOMMENDATIONS
6.1 Materials Improvement
• Address the lower levels of English language capability of secondary school 
students: Student difficulty in understanding the vocabulary and more complicated 
sentence structures used in the TLMs requires reconsideration of the level of English used 
in the textbooks. Future textbooks in any/all subjects must be solidly based on an English 
language assessment to guide the level of English competency required.
• Conduct a more comprehensive review of the usability of TLMs: Since teachers 
themselves have varying levels of subject matter knowledge and know-how in building
competencies in their students, a more comprehensive assessment of their ability to 
understand and teach the range of concepts incorporated in the TLMs should be 
undertaken to fulfill two goals: 1) improve instruction at TTCs; and 2) improve classroom 
delivery of lessons. As noted above, textbooks don’t teach; teachers do.
6.2 Systems Improvement 
• Link TLMP to teacher professional development: To maximize use of the TLMs, an 
investment should be made to build on the MOEVT’s own initiatives to implement a 
professional development support program on teaching with textbooks.
• Address copyright issues: An investigation should be undertaken to determine who 
actually owns the copyright to the TLMs so that, if so desired, they can be reprinted.
• Support a large-scale evaluation of the impact of TLMs on student learning:
USAID might consider supporting MOEVT and Tanzanian research universities to conduct 
a national assessment of the impact of TLMP and other textbooks on teaching and learning. 
• Create links between teacher education institutions and schools: To maintain 
continuous professional development and to further professionalize the teaching 
profession, a link between SCSU science and math educators and secondary school 
teachers in Tanzania via shared technological resources, particularly through USAID￾sponsored Teacher Resource Centers in Tanzania, should be considered. This pairing might 
be extended to U.S. secondary schools with which SCSU has a relationship through student 
teaching and in-service programs. 
• Build school-based management of resources: USAID might consider developing 
support programs that build on the MOEVT’s school-based decision making and 
decentralization efforts. The formation of school-based textbook review committees with 
objective criteria for evaluating potential textbooks for adoption is one critical function. 
USAID’s support for secondary school directors and academic leaders might help the 
MOEVT to capture these exemplars and create leadership development models around 
them for future school-based decision making. 
• Develop a strategy to support secondary math and science: All TLMs should be 
provided to TTCs so that they might be included in the curriculum in training new 
teachers, thus providing input on how to teach using a 21st Century textbook. Without 
such an input it is likely that students will not elect to study the sciences at a higher level.
• Link to other secondary education programs: The TLMP experience – processes 
and outcomes – should be linked to other donor secondary school interventions. 
Evaluation of TLMP – TLMP in Tanzania 22
• Insure quality and accountability of textbooks at the school level: The same level 
of quality and accountability in the choice of textbooks in evidence at EMAC should be 
required at the school level. This will mean that the appropriate MOEVT divisions should 
provide training at schools (including directors and teachers) on how to make textbook 
choices.
• Systematize the allocation of textbooks to schools that need them: As part of 
the process of identifying the number of textbooks needed in a particular subject at a 
particular grade level, the MOEVT should determine whether the subject is taught at that 
school and at that grade level so as not to provide books to a school where they are not 
needed. 
• Share TLMP evaluation processes and results with early grade reading project 
awardees: The lessons learned by SCSU in developing the TLMs should be shared with 
early grade reading project leaders. Even though the subject matter is different, there are 
useful elements emerging from the project that can be learned by others (e.g., copyrights, 
publishing, textbook review, etc.) 
6.3 USAID/W Improvement 
• Consistently apply performance monitoring procedures: USAID should require 
that all funded projects prepare a PMP and consistently conduct monitoring on project 
progress. In reviewing monitoring reports, included in quarterly reports, USAID should 
determine if the project is “on track” or whether approaches and activities need to be 
adjusted to obtain the desired results. 
• Build institutional capacity for less experienced US-based implementers: If 
USAID/W intends to earmark funds to expand capacities at U.S. institutions that do not 
have extensive experience in international development, it should anticipate the need for 
even greater oversight of their activities and ensure that such institutions are mentored so 
that capacity is gradually built over time. 
Evaluation of TLMP – TLMP in Tanzania 23
ANNEX A. SCOPE OF WORK
DESCRIPTION/ RESULTS-ORIENTED STATEMENT OF OBJECTIVES (SOO)
Evaluation of Textbooks and Learning Materials Program (TLMP) in Ethiopia, 
Ghana, Malawi, Senegal, South Africa, and Tanzania
I. BACKGROUND 
TLMP contributed directly to USAID‟s effort in the development and distribution of learning 
materials to improve sub-Saharan African (SSA) host country partners‟ management capacity in 
the education sector. Each Minority-Serving Institution (MSI), based upon the provisions noted 
in their Cooperative Agreement (CA), was responsible for managing and implementing the 
TLMP in a specific country and with achieving specific output results. Each MSI was also 
responsible for providing (i.e., identifying, selecting, developing, adapting, printing, assisting with 
distributing, and training users) a minimum of 600,000 copies of quality, cost-effective education 
materials for use in primary schools in its host partner country. These materials were to be 
developed and/or adapted under the CA in partnership with the host partner country’s Ministry 
of Education (MOE) and other local specialists. The main objectives of the TLMP were to: 1) 
produce and distribute high quality, cost-effective textbooks and learning materials, in support 
of USAID’s African Education Initiative (AEI) to enhance girls’ and boys’ access to learning 
opportunities in primary schools within SSA, 2) strengthen the capacity of U.S.-based MSIs to 
build sustainable linkages with African institutions, which would enable the latter to continue 
technical assistance after the completion of the program, and 3) ensure alignment with national 
curriculum to include relevant cross-cutting themes (i.e. gender, health, etc.).
 
TLMP Cooperative Agreement History 
• TLMP Ethiopia: USAID Cooperative Agreement RLA-A-00-09-00035-00; In
coordination with local entities, Alabama Agricultural and Mechanical University 
(AAMU), over 3 million English for Ethiopia textbooks were produced and disseminated 
for grades 1, 6, 7, and 8. Over 132 teachers were subsequently trained to use the 
materials in classroom settings. 
• TLMP Ghana: USAID Cooperative Agreement RLA-A-00-09-00036-00; In 
coordination with local entities, Chicago State University (CSU) has trained 260 
teachers in using the developed materials. Over 6 million materials and textbooks have 
been created and distributed for students up to grade 3 in mathematics, environmental 
science, and English. 
• TLMP Malawi: USAID Cooperative Agreement RLA-A-00-09-00033-00; In 
coordination with local entities, University of Texas, San Antonio (UTSA) developed and 
provided over five million supplemental reading books, teachers guides and training 
materials and trained nearly four thousand teachers on methodological classroom usage. 
• TLMP Senegal: USAID Cooperative Agreement RLA A 00-09-00037-00; In 
coordination with local entities, Elizabeth City State University (ECSU) produced and 
distributed over 1.8 million materials in both French and English for grades 2-10 in 
Evaluation of TLMP – TLMP in Tanzania 24
science, mathematics, and language arts. Over 160 teachers were trained on utilizing the
materials as part of their curriculum. 
• TLMP South Africa: USAID Cooperative Agreement RLA-A-00-05-00079-00; In 
coordination with local entities, University of Texas, San Antonio (UTSA) developed and 
provided over 1.4 million materials in 11 languages for grades 4, 5, and 6, as well as 
trained over 6,000 teachers. The work was completed in 2009. 
• TLMP Tanzania: USAID Cooperative Agreement RLA-A-00-09-00034; In 
coordination with local entities, South Carolina State University (SCSU) created and 
disseminated over 1.1 million materials for secondary level usage in the fields of science 
and mathematics. Over 1,200 teachers were trained. 
The Contractor will be provided with each institution’s Cooperative Agreement by each 
individual institution, which will include the relevant scope of work. The Contractor will be 
required to obtain other pertinent documents as necessary.
II. PURPOSE 
The purpose of this SOO is to support the Evaluation of Textbooks and Learning Materials 
Program (TLMP) in Ethiopia, Ghana, Malawi, Senegal, South Africa, and Tanzania. 
III. SCOPE OR MISSION 
Task 1 – Data Collection 
Task 2 – Data Review 
Task 3 – Coordination and Management 
Task 4 – Site Visit 
Task 5 – Data Analysis 
IV. PERFORMANCE OBJECTIVES / DESIRED OUTCOMES 
The Contractor shall provide all labor, equipment, supplies and materials, and travel necessary 
to conduct Textbooks and Learning Materials Program Evaluation (PE). The PE is intended to 
satisfy the following objectives: 
• validate stated program goals and impacts; 
• assess the results achieved for each host partner country in relation to intended 
program targets, as well as standardized and variable indicators by measuring 
quantitative and qualitative impacts of TLMP in terms of local capacity building (i.e. U.S.-
based MSIs, in-country institutions, Ministries of Education (MOEs), etc.), student 
achievement, teacher performance, amongst other criteria, in each host partner 
country; 
• determine if in-country institutions (with support from U.S.-based Minority-Serving 
Institutions [MSIs]) were able to deliver services effectively in terms of coordinating 
material design, alignment, production, and distribution; 
• review allocated USAID funding in terms of usage and overall cost effectiveness; 
• highlight specific program accomplishments per MSI-host country partnership; and 
Evaluation of TLMP – TLMP in Tanzania 25
• document lessons learned and provide recommendations for potential program scale-up 
and/or replication as related to the New Agency Education Strategy 
http://www.usaid.gov/our_work/education_and_universities/documents/USAID_ED_Str
ategy_feb2011.pdf 6 
V. OPERATING CONSTRAINTS / LIMITATIONS
We anticipate that Awardee would complete one site visit per country and that the site visits 
would take no longer than 10 days each. There is not a requirement for specific key personnel 
or a combination of key personnel to complete the site visits; however consistency in terms of 
personnel for the site visits is preferred. 
The Contractor shall perform the PE in accordance with USAID ADS 203 and the new USAID 
Evaluation Policy published in January 2011. The USAID ADS 203 Performance and Monitoring 
Guidance can be found here: http://www.usaid.gov/policy/ads/200/203.pdf. The new USAID 
Evaluation Policy can be found here: ttp://www.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf 
Monthly Status Reports. The Contractor shall provide written reports to the USAID COTR or 
his/her designee on the progress of the work, contacts made, and problems encountered on a 
monthly basis. They should be submitted by the last business day of every month. 
Comment Reponses. Comments will be provided to the Contractor electronically. The 
Contractor shall prepare comment responses that clearly state the actions taken to 
incorporate the comment or show the changes in a red-line and strikeout version of the 
revised report. The Contractor may contact the reviewers for clarification. Unresolved 
technical issues shall be coordinated with the COTR. 
Evaluation of TLMP – TLMP in Tanzania 26
ANNEX B. QUESTIONS POSED OF SCSU PROJECT TEAM
Ask re Following Documents
• MOE report on the appropriateness of the textbooks developed for Zanzibar
• Matrices of changes needed to textbooks to contextualize them to the Mainland
• Mapping instrument to align textbooks sequentially and in scope with the curriculum of the 
Mainland 
Background
• Initial MOU in 2005-2008 for secondary textbooks and learning materials in biology, 
chemistry, physics and math in Zanzibar ($5.5 million); what was accomplished under AEI? 
What were lessons learned from this first phase? How did SCSU build on what was done in 
Zanzibar to continue the TLM production and distribution process for the mainland in the 
second phase under CA of 2009-2012 ($13 million)? How did the results of Phase1 
influence Phase 2?
• How did SCSU become involved in the development of TLMs at the pre-school level in 
Zanzibar? What happened when funding was not available? Was any attempt made to find 
other funding for the early childhood program?
Administration
• How did all the changes in senior leadership affect the ability of TLMP to get things done? 
What types of stumbling blocks were overcome over the period of Phases 1 and 2 of 
TLMP?
• What was the role of the Ministry of Regional Administration and Local government in the 
basic education part of TLMP in Zanzibar?
• What is the actual number of TLMs produced for Zanzibar? For the Mainland secondary 
schools?
Project Implementation
• How and who decided who would get the textbooks?
• What was the problem with the legal office and the potential delay in publishing?
• What was the nature of your relationship with the Peace Corps? The World Bank?
• What criteria were used in school/classroom observation?
• What research was conducted on the entire project process? What lessons were learned? 
What best practices? All noted are a repeat of the first/earlier reports; nothing is new.
• What were the process and instruments used by EMAC in approving the textbooks that 
were produced in Zanzibar and adapted for use on the Mainland?
• While the quarterly reports and other documents report on the outputs, please explain the 
actual work that the textbook developers did in the first phase and how the materials were 
changed for use on the Mainland? Who of the SCSU did what? Who of the MOE/TIE did 
what? What teachers were involved? How were they chosen to be a part of the writing 
team? Who were the subject matter specialists in each team? How did they work with the 
teachers?
Evaluation of TLMP – TLMP in Tanzania 27
• What was the process used to include “culturally relevant” items in the textbooks? Who 
determined cultural relevance?
• How did the Tanzanian and SCSU subject matter experts work together to produce the 
textbooks? What was the process from start to finish?
• Why was the number of textbooks to be produced increased by 40,000?
• What was the process used to develop the teachers’ guides? Who specifically was involved? 
How were curriculum standards incorporated/followed?
• Who was responsible for writing the quarterly and annual reports? Why is there so much 
repetition and so little actual data/information presented?
• What is the system of distributing textbooks at schools? How large are classrooms and 
how many students actually receive textbooks?
Project Impact
• What is SCSU able to do now that it is different from the outset of TLMP? What capacities 
were built by whom? What institutional changes?
• What is the MOE/TIE able to do now that it is different from the outset of TLMP?
• How have teaching practices changed as a result of having access to books?
• How has learning by student improved as a result of using textbooks?
Evaluation of TLMP – TLMP in Tanzania 28
ANNEX C. SCHEDULE OF DATA COLLECTION ACTIVITIES
Date Time Organization Person 
Interviewed/
Research Activity
Title
WHOLE
TEAM
DAR ES 
SALAAM 
2/18 10:00 IBTCI Team 
Meeting
2/18 12:30 MOEVT Chris Kibanga Director-EMAC Local 
TLMP coordinator
2/18 1:00 MOEVT Silvestina Mgisimba; Acting Permanent 
Secretary of Education 
2/18 2:00 MOEVT Dorothy Mwaluko Director, Secondary 
Education-Science & 
Mathematics
2/19 1:00 Uhuru Media Lela Abdala Managing Director
2/19 3:00 USAID Tom LeBlanc;
Laura Kikuli and Abbas 
Nsanzugwanko
Education Manager; 
Program Manager;
M & E Specialist
2/20 2:00 TIE Leonard Akwilapo Acting Director
2/20 5:00 TIE Godson Lemme TIE Subject Matter 
Specialist (Physics
2/21 9:30 MOEVT M. Mukaruka Coast Regional 
Education Officer
2/21 10:30 MOEVT Omari Kisuda;
Alice Msemwa
Kibaha District 
Education Officer;
Assistant District 
Education Officer
2/21 12:00 MOEVT Georges Ngonyani Academic Master 
Mwanalugali Secondary 
School
2/21 1:30 MOEVT Teacher Focus Group Mwanalugali Secondary 
School
WHOLE 
TEAM
COAST
2/21 3:30 MOEVT Aaron Ndunguru Headmaster, Kibaha 
Town Council School
2/22 9:30 MOEVT Bashiry Shellimoh Temeke District 
Education Officer
2/22 10:30 MOEVT Godfrey Mchimbini Head Master;
Tuangoma Secondary 
School
2/22 11:00 MOEVT Various Teachers Tuangoma Secondary 
School
2/22 12-2:00 MOEVT Various teachers 
Classroom 
observations
Tuangoma Secondary 
School
Evaluation of TLMP – TLMP in Tanzania 29
SUB TEAM 
1
MORO-GORO
2/25 8:30 MOEVT Michael Daffa Morogoro Rural 
Council District 
Education Officer
2/25 9:30 MOEVT Mtembeje Kingimali;
Syvanu Konambi
Academic Officer 
Morogoro Municipal 
District Education 
Office; District 
Education Officer
2/25 10:30 MOEVT Rukia Chembe Assistant Academic 
Officer, Morogoro 
Regional Education 
Office
2/25 11:00 MOEVT Tabitha Tusekelege; 
Various Teachers
Head Mistress Kilakala 
Girls Boarding School 
2/25 1:00—4:00 MOEVT Berthea Kulwa
Various Teachers
Students
Head Mistress
Kingolwira Secondary 
School
2/26 2:00 MOEVT Theresa Kamugishu Sekwao Secondary 
School
2/27 9:00-12:30 MOEVT Anna Mmbeya;
Rosemary Sam; 
Various Teachers
Head Mistress, Kirokala 
Secondary School; 
Assistant Academic 
Officer
2/27 1:30-4:30 MOEVT Ibrahim Janguella;
Various Teachers
Head Master,
Mkuyuni Secondary 
School
2/27 8:45-9:15 MOEVT Damari Mchome Meru District Education 
Officer-Secondary 
Education
SUB-TEAM 
2
ARUSHA
2/27 9:30 MOEVT Alex Kitomari Head Master, Poli 
Secondary School
2/28 9:00-2:00 IBTCI Team Team Debrief
2/28 3:00 Oxford University 
Press
Peter Crowder;
Mussa Khamis;
Frederick Sylvester
Acting Managing 
Director; Program 
Manager;
Assistant Program 
managers
WHOLE 
TEAM
DAR ES 
SALAAM
3/1 9:00 MOEVT Silvestina Mgisimba Acting Permanent 
Secretary of Education 
3/1 10:00 USAID Debriefing: Tom 
LeBlanc;
Laura Kikuli and Abbas 
Nsanzugwanko
Education Officer; 
Program Manager;
M & E Specialist
ANNEX D. DATA COLLECTION INSTRUMENTS
In Capital and Adjacent Locations
1) USAID Mission – Education Team
• What has been the mission’s role in implementing TLMP? What types of support activities 
have you provided to the project? How were these different for Phase 1 in Zanzibar and 
Phase 2 on the mainland? 
• Have you supported any other book/learning materials development projects before? What 
indicators were used to track progress? 
• How does TLMP fit in with other USAID education program/priorities in this country? 
How do you see TLMP leveraging other educational development activities in Malawi or 
elsewhere in the future? 
• Did the development of TLMP have any (beneficial) effect on the national curriculum? On 
educational language policy? Has any new emphasis been placed on textbook and learning 
material development? Were all the textbooks and teachers’ guides accepted as part of the 
national curriculum?
• How does USAID support teacher training in this country? How has the TLMP been linked 
to these efforts? What would make these efforts sustainable?
• What specific challenges has TLMP faced in this country?
• How satisfied are you with the way TLMP was managed? What would you change? How 
satisfied are you with the outcomes of TLMP? What would you change?
• Would you say that the model for TLM creation developed by SCSU can be utilized in 
other areas? If not, what would you change?
• If a TLMP-type project emerges/is needed for the future, what would your 
recommendations be for improvement?
• How are you working with the World Bank to extend the creation/publishing/ disseminating 
process of TLMs?
• How are you working with the MOE to develop sustainability plans for the use and 
continuous development of TLMs?
• Do you believe that the textbooks produced for the Mainland were in line with the national 
curriculum? Did they properly address gender, HIV/AIDs and other cross-cutting themes? 
What else should be included?
• Was the level of expertise provided by SCSU and the MOE appropriate to the design of 
textbooks that have current and future applicability? What type of capacity building was 
conducted by SCSU? By the MOE?
• Are you aware of any studies that have documented the changes in national examination 
outcomes that could be linked to the production and distribution of the TLMs?
2) MOEVT/TIE Administrators (triangulate with different administrators at 
each level) 
• What is your current position? How long have you been in this position? For how long 
have you been working in education? In what positions?
• What was your particular involvement in TLMP?
Evaluation of TLMP – TLMP in Tanzania 31
• What is your impression of the TLMP? On a scale of 1-4 with 1 being very good and 4 
being very bad, how would your rank the program? Please explain your reason for this 
choice.
• How did you decide which staff members/departments were to work on the TLMP? Were 
they seconded or were the responsibilities added? Were salaries topped off for 
participation? If yes, by how much?
• How was it decided which schools would receive the TLMs? Which teachers would attend 
the TOT? Which teachers would receive the TLMP cascaded training? How many teachers 
did the MOE train on the use of the TLMs? How did you train them? For how long? Could 
you please provide a list of the teachers who were trained and the schools they 
represented?
• What types of policy change has the MOE instituted regarding textbooks and/or 
supplementary/complementary materials as a result of TLMP? Regarding teacher training?
• What other types of teacher training does the MOE provide? How frequently? 
• How does the district work with teachers to improve their teaching? How was this 
changed after the TLMP teacher training was delivered? How were the TLMs included in 
teacher training (either pre-service or INSET)?
• Did you find that the TLMs produced followed the MOE curricula appropriately? Did they 
incorporate gender? HIV/AIDS? Other cross-cutting themes?
• What was the difference between the TLMs produced for Zanzibar and those produced for 
the Mainland? What process was used in producing the TLMs for the mainland? 
• Do you believe that the way the TLMs were structured helped teachers in becoming more 
learner-centered? If yes, what, specifically, was included to do this? If not, what should be 
included? What other teacher training do you provide in learner-centered methodologies?
• What did the MOE learn from this project about developing textbooks? Is the MOE able to 
reproduce this process? If not, what would the MOE need? Now that the project has 
ended, what is your plan to continue the production of TLMs? What new textbook policies 
have been developed as a result of TLMP?
• How has the material presented in the TLMs been included in preparation for national 
exams? Has performance on exams improved with the use of TLMs?
• How was TLMP monitored by the MOE? What indicators did you use? How often did you 
go to schools to observe the use of TLMs? How was TLM production managed and 
monitored?
• How has the MOE benefited from TLMP? How has it been challenged?
• How satisfied are you with TLMP? If you were to make recommendations to another 
country implementing TLMP, what would you suggest? Why? If you could change anything 
about TLMP, what would it be? Why? If you were to scale up the production and 
distribution of these books, what would you want to be different?
• How satisfied are you with the collaborative relationships established with SCSU? What 
could be improved? 
• Have the textbooks become your national textbooks? If not, why not? What is preventing 
this adoption? If yes, do you have enough for all students and teachers in the country? If 
the answer is no, how many more are needed? What is the MOE plan to make up for this 
shortfall? 
Evaluation of TLMP – TLMP in Tanzania 32
3) Material Developers/Curriculum Specialists
• What is your current position? For how long have you had this position? For how long 
have you been working in this area (e.g., subject matter, curriculum and instruction, grade 
level)?
• How did you become involved in TLMP? What was your specific role at the outset? At the 
end of the project?
• What was the composition of the writing/production team? What types of expertise was 
represented? What other expertise was needed, in your view? How were the members of 
the team compensated for their activities? How many were male and how many females? 
Does this represent the gender distribution of teachers of science and math at the 
secondary level? 
• How often did the two teams meet? What were the results of these meetings?
• What process was used in creating the TLMs? How long did it take for each subject and 
grade level? 
• In developing TLMs, how did you ensure conformity with the national curriculum in terms 
of subject matter and grade level? In terms of ensuring gender neutrality? In terms of 
including such cross-cutting themes as HIV/AIDS? What other cross-cutting themes did you 
include? Were the materials culturally relevant?
• In developing the TLMs, how was the methodology of learner-centered teaching included? 
If so, how did this help teachers improve their teaching? Improve learner outcomes? 
• How did the textbooks for the mainland differ from those produced in Zanzibar? Scope 
and sequence? English language complexity? Illustrations? Cultural issues? Other? 
• What type of local and international review process did the production team have to go 
through? 
• How did you obtain illustrators for the TLMs?
• How satisfied are you with the collaborative production process between yourselves and 
SCSU? What would you change?
• How did you field test what was written? With whom? Where? Did you have a 
rural/urban balance? What was the difference in the results? Did you have to make 
significant changes to allow for appropriate usage in rural areas? What were these?
• How do you think the TLM development process can be improved in the future? 
4) Printers/Publishers – Uhuru Media and Oxford University Press
• Tell me about your operations before you were granted the TLMP contract and how they 
changed as a result of TLMP participation? Who were your clients and what was the nature 
of the work you did before the project and how did this change after the implementation of 
the project?
• How did the contracting occur with the SCSU? What was the content of the contract, i.e., 
what were your responsibilities? Did you have adequate personnel and technical resources 
to fill the order? What was lacking? How did you overcome these? Was the sequencing 
of the materials to be produced in accordance with your capacity, or were there any 
production difficulties that involved obtaining paper, skilled staff, etc.?
• Where were the TLMs printed? What was your particular role in producing the TLMs?
• What, if anything, did you learn through working on TLMP that enabled you to produce 
materials more efficiently and/or less expensively? 
Evaluation of TLMP – TLMP in Tanzania 33
• What instructions were you given on how to distribute the TLMs? From whom? What 
kind of difficulties did you encounter in keeping to the distribution schedule?
• When/how did you distribute the TLMs after they were produced? To whom did you 
distribute them? How many TLMs were delivered to each receiver? What kind of 
tracking/delivery system did you establish? What kind of challenges did you have in 
distributing the materials? How were these overcome? Could you please share your 
distribution list with us?
• How did having the TLMP contract change the way you do business? Improve your 
capacity? What new work are you now able to do that you could not before TLMP? How 
many new employees have you hired? What new equipment have you purchased? How 
much has your income increased? What other inputs would you require to take on more 
textbook production projects?
In Field
5) Regional Education Offices
• What is your current position? How long have you been in this position? For how long 
have you been working in education? In what positions?
• What was your particular involvement in TLMP?
• What is your impression of the TLMP? On a scale of 1-4 with 1 being very good and 4 
being very bad, how would you rank the program? Please explain your reason for this 
choice.
• How many of each TLM did you request for your district/region/province? (Subjects, 
levels?) How did you calculate this number for appropriate grade level students? If you had 
any surplus, what did you do with the materials? If you had any shortfall, what did you do?
• What instructions did you give for distribution to each school? How did you work with the 
distributor of the text and workbooks to ensure that they were properly delivered and 
received? How many of each textbook was made available to each school? Were there 
enough for all children in each class?
• Are all teachers who received the TLM’s required to use them in their teaching? What 
other materials do they have/use? Is there a sufficient number of these materials for all 
children? If not, what does a teacher do?
• What instruction did you give to each school about how the TLMs were to be used? Did 
the MOE provide a TOT? If so, how many teachers in your district attended the TOT? 
How were these teachers chosen? How many male? How many female? Is this division 
representative of the number of men and women teaching math and sciences at the high 
school level? How many of these teachers went on to teach others? How many others 
were trained? 
• How were inspectors instructed on how to evaluate teachers using TLMs? 
• From your point of view, on a scale of 1-4 with 1 being very effective and 4 being not 
effective at all, how would you rank the TLMs produced for this project? What do you 
recommend for materials improvement? Program improvement?
In Schools:
6) Principals/Directors
• What is your current position? How long have you been a principal at this school? How 
long have you been a principal? In how many schools?
Evaluation of TLMP – TLMP in Tanzania 34
• What is the overall economic status of the people in this community? How do they 
generate income? What is the composition of most families/households? How big a 
problem is HIV/AIDS in this community? About what percentage of your students are 
Orphans or Vulnerable Children (OVC)? 
• Do families send their girls to school as often as their boys? What gender-based trends do 
you see in enrollment? Has your school done anything to make teachers or families more 
aware of gender equity? If so, what have the results been?
• What language(s) do children speak when they enter school? Is this language the language 
of instruction? Up until what grade? When does English become the language of 
instruction? Is the level of English used in the TLMs appropriate? If not, what needs to be 
done? 
• What is the average amount of time a teacher remains in your school? What factors cause 
them to leave for work elsewhere? Once teachers have training in different subjects, do 
they generally remain in the school or go elsewhere? 
• How many of each textbook did you request for the school? How many of each textbook 
did you actually receive per grade level? If you had any surplus, what did you do with the 
materials? If you had a shortage, what did you do? When during the term were the books 
received? 
• How many of your teachers received the MOE training on the use of the TLMs? How did 
you choose these teachers? How many were males? How many were females? Does this 
distribution represent the gender division of teaching math and science at the high school 
level? What were the teachers’ reaction to/opinion of the training? Did you attend the 
training yourself? If so, what was your opinion of the training?
• How are the teachers coping with the new materials? For those who have not been 
trained, how are they using the materials? Have the TLMs enabled the teachers to be more 
learner-centered in the methodologies they use? If not, what else do they need?
• What other types of teacher training does the MOE provide? How often? Are those who 
attend expected to pass on what they have learned to their colleagues?
• Do you believe the materials are culturally relevant? Address HIV/AIDs? Address issues of 
gender equity? Follow the curriculum for each subject appropriately? If not, what would 
you change?
• Have the textbooks produced been adopted by the MOE as the core textbooks for each 
subject at each grade level? What other subject matter books do teachers and learners 
have? How would you compare/contrast the two?
• Are the textbooks relevant for the different science labs you have in school? If not, how 
should they be changed?
• Is the language level of the textbooks appropriate for the English language capability at each 
grade for your students? What needs to be changed?
• What is your impression of the TLMP? On a scale of 1-4 with 1 being very good and 4 
being very bad, how would you rank the program? Please explain your reason for this 
choice? What improvements would you make to the TLMP? Why?
7) Classroom and Head Teachers
• What is your current position? How long have you been teaching this subject at this grade 
at this school? How long have you been a teacher? What other classes have you taught 
Evaluation of TLMP – TLMP in Tanzania 35
before? At what grade level? What is the level of education you have achieved? What 
qualifications do you have to be a teacher? (certificate, diploma, degree)
• In this Region, which languages are used for instruction at which grade levels? In which 
language is initial literacy (reading and writing) learned? At what grade do children start 
learning English? At what grade does English become the language of instruction?
• What is your greatest challenge in teaching in English?
• How many students do you have in your classes? Specify class and number of students. 
What is the age range of your students in each class? 
• What non-TLMP textbooks do you have to teach your subject? What do you do when you 
don’t have enough textbooks for each learner? Do you have a teacher’s guide for each of 
the textbooks? If not, what do you use? 
• How would you compare the non-TLMP with TLMP textbooks in terms of usability? Level 
of English language used? Illustrations? Complexity of explanations? Other? How would 
you recommend that they be changed? 
• When did you receive training on the use of TLMs? How long did it last? Who delivered 
the training? On a scale of 1-4 with 1 being very good and 4 being very bad, how would you 
rank the TLMP training you attended? Please explain your reason for this choice? If you did 
not attend any training related to the materials, how did you learn how to use them?
• When did you receive the TLMs for your classes? How many were you provided? From 
whom did you receive them? How did you distribute them to your learners? How many 
learners must share a textbook? 
• Were you able to use the textbooks after the training? Did you feel you needed more 
training? In what? Do you have the appropriate lab equipment to utilize the textbooks? 
What do you do if you do not?
• Do you believe the TLMs were aligned with the curriculum? If not, how should the 
materials be changed? Are the texts culturally relevant? Do they address gender equity? 
Do they address HIV/AIDS? How might the presentation of these ideas be improved? Do 
you believe the TLMs were properly sequenced (go from easiest to hardest)? What would 
need to change if they were not?
• For each class that you teach, how long per day/how many periods per day [per week, per 
month] do you use the TLMs? 
• What, if anything, does “learner-centered teaching” mean to you? Do you think these 
materials help you to be more learner-centered in your teaching? Why/why not? Have the 
materials increased your teaching skills? How?
• What type of difficulties do the learners have in using the materials? Is there a difference 
between girls and boys? What would be needed to overcome these difficulties?
• What changes have you observed and recorded in children’s achievement since the TLMs 
were introduced? Are more girls or boys performing at a higher level in math and sciences 
since using the textbooks? 
• What is your opinion of the TLMs in so far as their attractiveness to students? On a scale 
of 1-4, with 1 being very attractive, and 4 being not very attractive, rank the materials. 
Please explain your reason for this choice.
• What is your opinion of the TLMs in the ways that they depict girls and boys? Do they 
represent girls and boys equally? Do they represent them in non-traditional /traditional 
roles? 
Evaluation of TLMP – TLMP in Tanzania 36
• Is there anything about the TLMs that you would change? What? Why?
• In using the TLMs, what changes have you made in your teaching? How useful is the 
Teacher’s Guide in planning and teaching your lessons? On a scale of 1-4 with 1 being 
extremely helpful and 4 being not helpful at all, please rank the Teacher’s Guide. Please 
explain your reason for this choice.
• What is the greatest challenge your students experience in using the TLMs?
• What do you think is the overall impact of the program on your students? What kind of 
difference does it make in learning for a child to have textbooks?
• What is your impression of the TLMP? On a scale of 1-4 with 1 being very good and 4 
being very bad, how would your rank the program? Please explain your reason for this 
choice. What changes would you make to improve the program: 1) in the textbooks and 
learning materials? 2) In the delivery of the program? 
8) TEACHER OBSERVATIONS OF TLM USE IN THE CLASSROOM
School __________________________ Village/Town/City ________________________
Country _________________________ Type of School: ___Primary; ___Jr. Sec; ___Sec
Teacher Sex _____ M; _____F Grade Level: __________________________ 
No. of Students: ___ M; ___ F; ___Total No. of Books _________________________
Age range of Students: _____M; _____ F
Mother Tongue of Students ________________________________________________
No. & type of learning materials _____________________________________________
Indicator Observed Not 
Observed
Teaching Using TLMs 
Instruction
1. The teacher has prepared materials on the use of the TLMs for the 
class period.
2. Students have the appropriate TLMs and are ready to use them in 
class activities.
3. The teacher explains the goal and purpose of the class lesson to the 
students. 
4. The teacher identifies, pronounces and defines any difficult 
vocabulary before teaching the lesson.
5. The teacher begins the class activity with questions that review 
previous activities using the TLMs and draws on the prior knowledge of 
the students. 
6. The teachers uses learning aids/materials produced by TLMP
7. The teacher can read and explain TLM content to the students
8. The teacher uses TLMs throughout the class period
9. Students use TLMs throughout the class period
10. Students can read and understand the subject matter in the TLMs.
Evaluation of TLMP – TLMP in Tanzania 37
Indicator Observed Not 
Observed
11. Students are actively and interactively engaged with the teacher in 
the use of TLMs (Q&A, group work, workbook practice, continuous 
assessment)
12. The teacher makes sure learners make connections of TLMP content 
to daily lives
13. The teacher gives and corrects homework using the TLMs.
14. The teacher shows evidence of having used the Teacher’s Guide in 
presenting the lesson
15. Students and teachers use mother tongue/English (French) when 
asking and responding about TLMs (circle which language)
TLMs/Artifact Inventory 
16. Lesson objectives are written on the board in either local language or 
English (French)
17. Learning aids/materials are posted in the classroom (TLMP produced 
and others). 
18. TLMs are locked up in the cupboard.
19. Word walls display key words in local languages and English
20. To demonstrate language experience, sentences appear on the 
chalkboard or on a chart in the classroom
21. Students write words and sentences in their exercise books 
(demonstrating evidence of having pencils/pens and exercise books)
22. There is evidence that teachers mark exercise books in a process of 
continuous assessment
Comment_____________________________________________________________________ 
_____________________________________________________________________________ 
9) Learners’ Reading Competency
Note: Because the TLMP textbooks for secondary Mathematics and Science in Tanzania were 
not intended to improve students’ English language reading proficiency, the purpose of 
evaluating students’ reading performance was to evaluate the readability or fit between the 
students’ reading ability and the level of text difficulty presented by the TLMP textbook. In 
addition, because these were secondary school students there was no need to assess them on 
the elements of early grades reading—letter recognition, familiar word reading, and decoding. 
The protocol applied is provided below. 
EVALUATION OF LEARNER READING COMPETENCY 
School ___________________________ Village/Town/City ________________________
Country _________________________ 
Grade Level ______________________ Class Subject Matter _____________________ 
Student Age _____________ Sex ____M; ____F 
Home Language/Mother Tongue of Learner: ____________________________________
Evaluation of TLMP – TLMP in Tanzania 38
Language of Testing: _____________________________________________________
Title/Type of TLM used in Class: ______________________________________________
Reading Aloud:
Instructions: Identify 2 short passages (3 to 5 sentences) from one of the TLMs for the 
appropriate grade level and make a clean copy of each passage. [Please save passages to 
accompany evaluation results. They should preferably be cut from within stories or texts so 
that memorization can be ruled out. One passage should be easier than the other, so one can 
be taken from the beginning of the book and one from the middle, depending on how far into 
the school year they are.] 
Have the student read the more advanced passage aloud and note the results below. If s/he is 
successful, stop after this.
If the advanced passage is too difficult, provide the easier passage and conduct the exercise 
again, noting the results. 
Reading Fluency 
Passage 1: ____________________ (Rate and comment)
1) Fluent _______________________________________________________________ 
2) Little difficulty (specify words) ___________________________________________ 
3) Very haltingly (specify words) ____________________________________________ 
4) Can read only a few words (specify words) __________________________________
5) Cannot read at all ____________________________________________________
Passage 2: ____________________ (Rate and comment)
1) Fluent _______________________________________________________________ 
2) Little difficulty (specify words) ___________________________________________ 
3) Very haltingly (specify words) ____________________________________________ 
4) Can read only a few words (specify words) __________________________________
5) Cannot read at all ______________________________________________________
Reading Comprehension/Oral Reading 
Ability of learner to paraphrase what s/he has read silently:
1) Can put into own words all that has been read without difficulty _____________________
2) Can put into own words only selected points __________________________________
3) Has difficulty putting passage into own words __________________________________
4) Cannot put passage into own words _________________________________________ 
10) TLM Analysis Instrument – Criteria Used for Analysis of Textbooks Generated 
for Mainland Tanzania
 
Subject _________________________ Grade Level _________________________
STUDENT TEXTBOOKS
Evaluation of TLMP – TLMP in Tanzania 39
1. Curriculum Coverage
• How balanced is the presentation of topics as reflected by the table of contents?
2. Readability
• What are the general features of sentence length and vocabulary?
• What is the readability level as measured by the Flesch Readability formula?
3. Text Structure
• How is content organized within paragraphs (e.g., main idea/details; lists, definitions, 
characteristics, etc.)?
• How are paragraphs linked using linguistic markers and content information?
 
4. Text Elements
• How does the text use bold-face and italic fonts to highlight key information?
• How does the text use labels, captions, and subheadings to highlight important 
information? 
5. Graphics
• What kinds of graphics are used in the textbook (photographs, drawings, charts, maos, 
diagrams etc.)?
• What is the quality of these graphic elements?
6. Additional Features
• Does the textbook contain a chapter overview and review?
• Does the textbook include a glossary or list of important formula?
• Does the textbook include self-checking review questions?
TEACHER’S GUIDES
1. Lesson Objective
• Are lesson objectives clearly identified for each lesson?
• Are objectives linked /cross-referenced to the national curriculum?
2. Support for Planning
• Does the teacher’s guide suggest the amount of time needed to complete a unit?
• Does the teacher’s guide identify recommended/required instructional resources and 
equipment? 
3. Background information
• Does the teacher’s guide include material that provides additional content material for 
the teacher?
• Does the teacher’s guide contain suggestions for a variety of teaching techniques?
4. Meeting the Needs of Diverse Students
• Does the teacher’s guide contain suggestions for adapting instruction and content to 
meet the needs of diverse learners?
Evaluation of TLMP – TLMP in Tanzania 40
5. Overall appearance
• Does the appearance of the text seem dense and overwhelming?
• Are key resources easily identified with text graphics?
• Is there a table of contents, glossary and index to simplify usage?
Evaluation of TLMP – TLMP in Tanzania 41
ANNEX E. MATHEMATICS EXAMPLE OF ADAPTATION PROCESS
A report of the review process for Basic Mathematics Forms 3 and 4 indicates that a six-person 
Tanzanian team and the lead mathematics writer from SCSU adapted these materials. The 
books were examined to address concerns in the following areas:
• Conformity to the syllabus
• Organization and presentation of content
• Correctness of facts, concepts and figures
• Conformity to socio-cultural values of Tanzania
• Integration of life skills and cross-cutting issues
• Adaptability and applicability to local and global settings
• Relevance and appropriateness to learners and teachers
• Use of quality instruction
• Ability to appeal to learners of different abilities
• Presence of enjoyment index
• Assessment of learners’ understanding
• Appropriateness of language and communication aspects
The findings of their review led to changes in the following areas:
• Supplying introductions for some subtopics/content of chapters, where necessary 
• Reorganizing the content of some chapters
• Adding examples/exercises that integrate cross-cutting issues and life skills
• Revising/adding class activities to chapters, where necessary
• Introducing/including elements to increase the enjoyment index of mathematics
• Revamping/rewriting some concepts that were felt to be inadequately presented
• Suggesting changes/improvements in figures, terminology, notation, position of figures, etc.
• Proving a variety of examples and problems requiring critical thinking and applications of 
real-life situations throughout the text
A similar review process was undertaken for all textbooks produced in Zanzibar for use on the 
mainland due to more clearly defined learning parameters of the curriculum. A similar process 
was undertaken for each of the teacher’s guides. 
Evaluation of TLMP – TLMP in Tanzania 42
ANNEX F. TEXTBOOK REVIEWS
In general, the textbooks appeared well-designed, with a number of structural elements to 
support learning. Chapters were generally brief and were subdivided into content chunks of a 
half to a full page. This would make the reading task more manageable for a second/foreign 
language user. The subdivisions were clearly labeled with bold fonts to identify the main idea of 
the topic of the section. Much of the content within each section was presented as bulleted 
lists. This structure seemed to reflect the tradition of rote memorization and recall of factual 
information. Other study aids in the student textbook included chapter openers which 
identified key concepts and learning objectives and chapter closers that helped students review 
the critical content presented in the chapter. An abundant use of color photographs, charts and 
other graphics seemed especially helpful for guiding all students but especially valuable for 
second/foreign language learners.
However, when the content was presented as connected text, readability formulas (based on 
sentence length and syllable counts) suggested that the writing was generally above grade level 
for native English speakers. This level of text difficulty would be extremely challenging for 
second/foreign language readers, as the informal reading assessment indicated. 
A review of the Teacher’s guides revealed distinct differences between the two publishers. The 
Guides prepared by Uhuru offered considerable “extra” information for the teacher. These 
extras included:
• The identification of learning objectives for each lesson and explanation of how these 
objectives were linked to the national curriculum
• A review of various teaching techniques 
• A list of recommended materials for each lesson (which would be especially useful when 
planning science activities)
• Practice assessments modeled after the national assessment formats
• Background content information providing additional facts related to lesson content
• A suggested block plan detailing the number of days to allow for each chapter
• Specific teaching suggestions for each lesson 
The Guides produced by Macmillan were adequate but less detailed. They focused on learning 
objectives and answers to questions in student textbooks. 
Overall, both types of Guide seemed to be an exceptional resource in a context that suffers 
from a dearth of professional materials. It is unfortunate that most teachers in the schools we 
visited were unaware of these resources or had not made use of them. Unfortunately there 
were no indications that the publishers asked classroom teachers to review these guides or 
pilot them at any time during the development/adaptation of the TLMs. 
Evaluation of TLMP – TLMP in Tanzania 43
ANNEX G. CONFLICT OF INTEREST STATEMENTS
Evaluation of TLMP – TLMP in Tanzania 44
Evaluation of TLMP – TLMP in Tanzania 45
Evaluation of TLMP – TLMP in Tanzania 46
I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure 
form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to 
Name Dr. Richard W. Chediel
Title
Team Member
Organization IBTCI
Evaluation Position Education Evaluation Specialist
Evaluation Award Number (contract or 
other instrument)
AID-OAA-TO-12-00054
USAID Project(s) Evaluated (Include 
project name(s), implementer name(s) and 
award number(s), if applicable)
Textbooks & Learning Materials Project (Chicago State University, Alabama 
A&M, South Carolina State Univ., Elizabeth City State University, Univ. of Texas 
San Antonio
I have real or potential conflicts of 
interest to disclose.
 No 
If yes answered above, I disclose the 
following facts:
Real or potential conflicts of interest may 
include, but are not limited to:
1. Close family member who is an employee of
the USAID operating unit managing the
project(s) being evaluated or the
implementing organization(s) whose
project(s) are being evaluated.
2. Financial interest that is direct, or is
significant though indirect, in the
implementing organization(s) whose projects
are being evaluated or in the outcome of the
evaluation.
3. Current or previous direct or significant
though indirect experience with the
project(s) being evaluated, including
involvement in the project design or previous
iterations of the project.
4. Current or previous work experience or
seeking employment with the USAID
operating unit managing the evaluation or
the implementing organization(s) whose
project(s) are being evaluated.
5. Current or previous work experience with an
organization that may be seen as an
industry competitor with the implementing
organization(s) whose project(s) are being
evaluated.
Evaluation of TLMP – TLMP in Tanzania 47
protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the 
information for any purpose other than that for which it was furnished.
Signature
Date March 01, 2013
6. Preconceived ideas toward individuals, 
groups, organizations, or objectives of the 
particular projects and organizations being 
evaluated that could bias the evaluation. 
Evaluation of TLMP – TLMP in Tanzania 48